How to debug SSL connections from SDC

If you ever encounter an error related to the SSL connections from SDC to the various other endpoints we work with, where the stack-trace can have lines like these:

at sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:464) ~[?:1.8.0_345]
at sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:165) ~[?:1.8.0_345]
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:109) ~[?:1.8.0_345]
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1397) ~[?:1.8.0_345]
at sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1305) ~[?:1.8.0_345]
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440) ~[?:1.8.0_345]

And you want to get more information about the SSL connection. You can enable the -Djavax.net.debug option on the JVM arguments.

This page has a lot of useful information about it: https://colinpaice.blog/2020/04/05/using-java-djavax-net-debug-to-examine-data-flows-including-tls/. 

With -Djavax.net.debug you can use this options:

all turn on all debugging
ssl turn on ssl debugging

The following can be used with ssl:
  record       enable per-record tracing
  handshake    print each handshake message
  keygen       print key generation data
  session      print session activity
  defaultctx   print default SSL initialization
  sslctx       print SSLContext tracing
  sessioncache print session cache tracing
  keymanager   print key manager tracing
  trustmanager print trust manager tracing
  pluggability print pluggability tracing

handshake debugging can be widened with:
  data         hex dump of each handshake message
  verbose      verbose handshake message printing

record debugging can be widened with:
  plaintext    hex dump of record plaintext
  packet       print raw SSL/TLS packets

For example to get debug logs for the SSL handshake in verbose form you can put 

-Djavax.net.debug=ssl:handshake:verbose on the JVM options. You follow this documentation page to change the JVM options on Legacy DataCollector.

These logs will be outputted into the /tmp/engine-*.out file. This file will grow pretty quickly, so ideally you should enable this option for 5-15 minutes, capture the problematic connection and then disable it. Then you can use a log viewer program like Console on Mac OS to analyse them.