S3 Destination WARN "No content length specified..."

  • 26 October 2021
  • 1 reply

Userlevel 6
Badge +3
  • Senior Technical Evangelist and Developer Advocate at Snowflake
  • 67 replies

When writing to Amazon S3 destination why are these “WARN” messages showing up in the logs? Does this have any impact on the running pipeline?


No content length specified for stream data. Stream contents will be buffered in memory and could result in out of memory errors.



Best answer by Dash 26 October 2021, 21:24

View original

1 reply

Userlevel 6
Badge +3

These messages are coming directly from the AWS client library code. Basically when streaming data to/from S3, they don't send a content-length header along with the payload; since it’s streaming mode.

Note that these are just warnings and not errors so this should not impact the running pipeline.