Solved

Error Records: Send Response to Origin

  • 5 January 2022
  • 3 replies
  • 16 views

A pipeline’s origin is an S3 bucket. Error records are configured to “Send Response to Origin.” What exactly happens to the error records in this instance?

icon

Best answer by dima 6 January 2022, 13:19

View original

3 replies

Userlevel 1

@mblahay ,

https://docs.streamsets.com/portal/datacollector/latest/help/datacollector/UserGuide/Pipeline_Design/ErrorHandling.html#concept_kgc_l4y_5r indicates the following:

 

Send Response to Origin

The pipeline passes error records back to the microservice origin to be included in a response to the originating REST API client. Data Collector includes the records in error record counts and metrics. Use in microservice pipelines only.

 

I read that too, but what does it mean in the context of an S3 origin? Please distill it down for me.

Userlevel 2
Badge

It means it's misconfigured. Send Response to Origin only applies when you're creating a microservice pipeline. Such a pipeline must have either the REST Service, WebSocket Server, or WebSocket Client stage as its origin. 

Reply