Solved

HTTP Server - Max Request Size

  • 13 January 2023
  • 3 replies
  • 79 views
T
Rank

hi, i am using in one pipeline as origin HTTP Server with value Max Request Size = 100MB. 

I start to getting errors like this: 
CONTAINER_0001 - com.streamsets.pipeline.api.ext.io.OverrunException: Reader exceeded the read limit '100000000'

Is there anything from configuration perspective how to fix it, to allow bigger records?

Thanks

 

 

 

 

icon

Best answer by Dimas Cabré i Chacón 13 January 2023, 12:32

View original

3 replies

Dimas Cabré i Chacón
Rank
Userlevel 2
Badge

@thirty2 Glad to hear it worked. Size is per request, which normally is one request per record.

T
Rank

To configure larger values you need to set property

 

com.streamsets.httpserverpushsource.maxrequest.mb
 

in your SDC startup script, like 

-Dcom.streamsets.httpserverpushsource.maxrequest.mb=200

Thanks, it is working. 
Just to be sure, is it a size per record?

Dimas Cabré i Chacón
Rank
Userlevel 2
Badge

To configure larger values you need to set property

 

com.streamsets.httpserverpushsource.maxrequest.mb
 

in your SDC startup script, like 

-Dcom.streamsets.httpserverpushsource.maxrequest.mb=200

Reply


Terms & Conditions