Technical Service Bulletin 2022-02-25 (TSB) - StreamSets Data Collector 4.4.0 - HTTP Client processor Output Field is not overwritten if the field already exists

StreamSets Support Team would like to inform you of a regression when upgrading to StreamSets Data Collector 4.4.0 involving the HTTP Client processor.

Products Affected: StreamSets Data Collector

Releases Affected: StreamSets Data Collector 4.4.0

Users Affected: StreamSets Data Collector using HTTP Client processor stage

Severity: High/Medium

Description:

The Output Field property specified in the HTTP Client processor stage will not be overwritten if the field already exists. For example, if the field "/output" has a value of "testValue0" before reaching the processor and the processor defines the Output Field property as "/output", then the value will remain "testValue0" regardless of the response returned by the Resource URL specified in the processor.

Immediate action required: Yes

Workaround: 

1. Specify an Output Field which does not already exist
2. If the Output Field already exists and must be overwritten, then a temporary Output Field can be specified and the Expression Evaluator processor used to overwrite the original field. The Field Remover processor can then be used to remove the temporary Output Field. For example:
   1. Field "/output" exists before HTTP Client processor stage
   2. HTTP Client processor specifies Output Field property as "/output1"
   3. Expression Evaluator processor specifies the Field Expression:
      1. Output Field: /output
      2. Field Expression: ${record:value('/output1')}
   4. Field Remover processor removes "/output1"

Fix: A fix for this issue will be available in our next patch release, StreamSets Data Collector 4.4.1.

Resolution: If utilizing a workaround is not possible, then postpone upgrading StreamSets Data Collector until version 4.4.1 is released.

For your questions, please contact our Support Team via a support request or at support@streamsets.com.