Skip to main content

hi, i am using in one pipeline as origin HTTP Server with value Max Request Size = 100MB. 

I start to getting errors like this: 
CONTAINER_0001 - com.streamsets.pipeline.api.ext.io.OverrunException: Reader exceeded the read limit '100000000'

Is there anything from configuration perspective how to fix it, to allow bigger records?

Thanks

 

 

 

 

To configure larger values you need to set property

 

com.streamsets.httpserverpushsource.maxrequest.mb
 

in your SDC startup script, like 

-Dcom.streamsets.httpserverpushsource.maxrequest.mb=200

To configure larger values you need to set property

 

com.streamsets.httpserverpushsource.maxrequest.mb
 

in your SDC startup script, like 

-Dcom.streamsets.httpserverpushsource.maxrequest.mb=200

Thanks, it is working. 
Just to be sure, is it a size per record?

@thirty2 Glad to hear it worked. Size is per request, which normally is one request per record.

Reply


Terms & Conditions