Hi!
I have problem with size 'Request Data' in HTTP Client.
This step have ‘Required Fields’ in which comes xml different size.
And when the size for the 'Request Data' is exceeded, the pipeline doesn’t work.
Parameters:
HTTP Method: POST
Request Data: <?xml version="1.0" encoding="utf-8"?>
<request>
<internal_id>${record:value("/xml_value")}</internal_id>
</request>
Default Request Content Type: application/xml
Data format: Text with Max Line Length → ${512 * 1024}
ERROR Failed executing stage 'HTTPClient_02': com.streamsets.pipeline.api.base.OnRecordErrorException: HTTP_01 - Error fetching resource. HTTP-Status: 400 Reason: Bad Request null PipeRunner *admin 0 ProductionPipelineRunnable-TEST4ed56edcc-3dbb-47c1-8295-bbec1ab0ebb6-TEST4
com.streamsets.pipeline.api.base.OnRecordErrorException: HTTP_01 - Error fetching resource. HTTP-Status: 400 Reason: Bad Request null
at com.streamsets.pipeline.stage.processor.http.HttpProcessor.processResponse(HttpProcessor.java:793)
at com.streamsets.pipeline.stage.processor.http.HttpProcessor.process(HttpProcessor.java:389)
at com.streamsets.pipeline.api.base.SingleLaneProcessor.process(SingleLaneProcessor.java:95)
at com.streamsets.pipeline.api.base.configurablestage.DProcessor.process(DProcessor.java:35)
at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:299)
at com.streamsets.pipeline.api.impl.CreateByRef.call(CreateByRef.java:40)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:244)
at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:311)
at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:221)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:864)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$executeRunner$3(ProductionPipelineRunner.java:908)
at com.streamsets.datacollector.runner.PipeRunner.acceptConsumer(PipeRunner.java:207)
at com.streamsets.datacollector.runner.PipeRunner.forEachInternal(PipeRunner.java:152)
at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:132)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.executeRunner(ProductionPipelineRunner.java:907)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:885)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runPollSource(ProductionPipelineRunner.java:609)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.run(ProductionPipelineRunner.java:400)
at com.streamsets.datacollector.runner.Pipeline.run(Pipeline.java:535)
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:112)
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:75)
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startInternal(StandaloneRunner.java:762)
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:755)
at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:157)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:34)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:226)
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:34)
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:222)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:100)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Please tell me what setting I need to change to increase the size for the field 'Request Data'.
I use StreamSets Data Collector 3.19.1.
