Skip to main content

hi, i am using in one pipeline as origin HTTP Server with value Max Request Size = 100MB. 

I start to getting errors like this: 
CONTAINER_0001 - com.streamsets.pipeline.api.ext.io.OverrunException: Reader exceeded the read limit '100000000'

Is there anything from configuration perspective how to fix it, to allow bigger records?

Thanks

 

 

 

 

@thirty2 Glad to hear it worked. Size is per request, which normally is one request per record.

To configure larger values you need to set property

 

com.streamsets.httpserverpushsource.maxrequest.mb
 

in your SDC startup script, like 

-Dcom.streamsets.httpserverpushsource.maxrequest.mb=200

Thanks, it is working. 
Just to be sure, is it a size per record?

To configure larger values you need to set property

 

com.streamsets.httpserverpushsource.maxrequest.mb
 

in your SDC startup script, like 

-Dcom.streamsets.httpserverpushsource.maxrequest.mb=200

Reply


Terms & Conditions