After successfully upgrading Data Collector using Cloudera Manager from a previous version to 5.x you notice that the log screen does not get any log entries and there is no sdc.log file generated in the $SDC_LOG directory.
Cloudera Manager uses CSD (custom service descriptor) files to determine the configuration, monitoring and file distribution of a custom service (in this case SDC) that will be managed by Cloudera Manager. You can learn more about this in Cloudera’s documentation.
As part of the SDC upgrade steps you need to install the newest CSD file. This has always been a requirement but in previous versions, not updating it caused little to no problems. For this reason some users may have skipped this step for previous SDC upgrades. For upgrades of older SDC versions to 5.x, not updating the CSD file will result in no logs being generated.
SDC 5.x uses log4j 2.x while older versions of SDC used log4j 1.x which are not compatible with each other. If the old CSD file is kept in CM, it will distribute the old log4j 1.x files to SDC 5.x which no longer knows how to use this library.
Updating the CSD file to the correct version and restarting cloudera-scm-server should correct the issue.
***Please note, you can only have one SDC CSD file installed at the same time. Having more than one can cause a crash in cloudera-scm-server.