A pipeline’s origin is an S3 bucket. Error records are configured to “Send Response to Origin.” What exactly happens to the error records in this instance?
Solved
Error Records: Send Response to Origin
- Discovered Fame
Best answer by dima
It means it's misconfigured. Send Response to Origin only applies when you're creating a microservice pipeline. Such a pipeline must have either the REST Service, WebSocket Server, or WebSocket Client stage as its origin.
- StreamSets Employee
https://docs.streamsets.com/portal/datacollector/latest/help/datacollector/UserGuide/Pipeline_Design/ErrorHandling.html#concept_kgc_l4y_5r indicates the following:
Send Response to Origin
The pipeline passes error records back to the microservice origin to be included in a response to the originating REST API client. Data Collector includes the records in error record counts and metrics. Use in microservice pipelines only.
- Discovered Fame
I read that too, but what does it mean in the context of an S3 origin? Please distill it down for me.
- StreamSets Employee
- Answer
It means it's misconfigured. Send Response to Origin only applies when you're creating a microservice pipeline. Such a pipeline must have either the REST Service, WebSocket Server, or WebSocket Client stage as its origin.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.