When utilizing the HTTP Client stages to connect to resources we can sometimes face issues which default logging within the pipeline may not encompass. However, the HTTP Client stage contains a fantastic feature to help provide context to the issues you might encounter. This feature is “HTTP Request Logging”.
From the HTTP Client Stage select “Show Advanced Options”
Selecting “Show Advanced Options” will unlock the advanced tab titled “Logging”
From here check the box at the top of the section to ensure request logging has been enabled. Now that you have enabled request logging we need to ensure that it is configured correctly. The picture below shows the settings you should utilize specifically in order for the logging of verbose request and response information. In order for this setting to work in this context you must ensure that Log Level: INFO and that your Max Entity Size is set high enough to fully encompass the contents which you are attempting to log. Keep in mind when configuring the Max Entity Size that the setting handles size in bytes, and so it should be scaled accordingly for large payloads. Lastly, for the debugging selection regarding Verbosity you will want to select PAYLOAD_ANY. Consider this an additional tool in your tool belt to use alongside things like CURL and Postman.
Now when you attempt to run your pipeline you will see detailed information within the log stream pertaining to your connection and payload.
For example, let’s say you are attempting to send a JSON ARRAY of Objects payload to your API. However, for some reason your pipeline errors out and you are unable to deliver your payload. You might try:
- To test your logic with Postman to ensure that it works in an isolated situation.
- CURL the server to ensure that the API endpoint you are trying to reach is accepting connections.
- Now in addition you can view request and response data between those nodes for additional insight.