How to Troubleshoot: NullPointerException at getClasspathUrls(BootstrapMain.java: 579)

Issue:  SDC will not start - we get this exception:

Exception in thread "main" java.lang.NullPointerException
at com.streamsets.pipeline.BootstrapMain.getClasspathUrls(BootstrapMain.java:579)
at com.streamsets.pipeline.BootstrapMain.getStageLibrariesClasspaths(BootstrapMain.java:555)
at com.streamsets.pipeline.BootstrapMain.bootstrap(BootstrapMain.java:307)
at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:111)

Troubleshooting: 

When we have an issue such as this, SDC may run for a moment then crash at startup with this exception. 

In the case we were troubleshooting - ZD-21124 - This indicated that there was an error in one of the following libraries - 

'-configDir' /Users/bob/SDC/streamsets-datacollector-3.17.1/etc
'-mainClass' com.streamsets.datacollector.main.DataCollectorMain
'-apiClasspath' /Users/bob/SDC/streamsets-datacollector-3.17.1/api-lib/*.jar
'-containerClasspath' /Users/bob/SDC/streamsets-datacollector-3.17.1/etc:/Users/bob/SDC/streamsets-datacollector-3.17.1/container-lib/*.jar
'-streamsetsLibrariesDir' /Users/bob/SDC/streamsets-datacollector-3.17.1/streamsets-libs
'-streamsetsLibrariesExtraDir' /opt/sdc-extras/
'-userLibrariesDir' /opt/sdc-user-libs/

The first step is to review the directory and file ownership and permissions for the entire tree for all the diectories listed above.  Verify that the jar files have the correct permissions.  755 and 644 both work when the directory/file is owned by the same user who is trying access the files (in this case, sdc was the service account that's running DataCollector).

In case #211124  - the issue was in the user-libraries - in this case the lib subdirectory was a regular file, and of course there were no jars.

From the exception, we can identify that we're in the bootstrap phase of starting SDC.  This exception tells us little regarding the root cause of the issue and how to move towards a resolution.  We can learn more by enhancing the debugging level in the BootstrapMain.java class with the following parameter added to the SDC_JAVA_OPTS.

-Dstreamsets.bootstrap.debug=true

This will generate a vast amount of debugging to stdout.   if we're unclear how to access stdout in the customer's environment we can capture it by going to libexec/_sdc and finding the statement that starts sdc - it looks like this: 

else
status=88
while [ $status = 88 ]; do
${JAVA} -classpath ${BOOTSTRAP_CLASSPATH} ${SDC_JAVA_OPTS} com.streamsets.pipeline.BootstrapMain \
-mainClass ${SDC_MAIN_CLASS} -apiClasspath "${API_CLASSPATH}" -containerClasspath "${CONTAINER_CLASSPATH}" \
-streamsetsLibrariesDir "${STREAMSETS_LIBRARIES_DIR}" -userLibrariesDir "${USER_LIBRARIES_DIR}" -configDir ${SDC_CONF} \
-libsCommonLibDir "${LIBS_COMMON_LIB_DIR}" ${EXTRA_OPTIONS}
status=$?
done
fi

after ${EXTRA_OPTIONS}   just add a redirect for stdout eg: 

>/tmp/sdc.sdtout

The file will be 10000 - 15000 lines - a few megabytes.

Looking at it from the top, there is the configurations information, then information regarding the specific classes that are found and are being loaded. 

Resolution:

We removed the offending libraries.  If there is any useful content, you might want to rena it to another directory and get the bad directories out of the library path.