SDC startup fails with unsupported JDK error

Scenario 1:

After trying to start SDC service, you may see an error like below when checking the status; preventing SDC service from coming up :

$ sudo systemctl status sdc -l 
● sdc.service - StreamSets Data Collector (SDC) 
Loaded: loaded (/usr/lib/systemd/system/sdc.service; static; vendor preset: disabled) 
Active: failed (Result: exit-code) since Sat 2018-01-20 08:38:14 EST; 13min ago 
Process: 25583 ExecStart=/opt/streamsets-datacollector/bin/streamsets dc -verbose (code=exited, status=1/FAILURE) 
Main PID: 25583 (code=exited, status=1/FAILURE)
Jan 20 08:38:13 bdlingprd05 systemd[1]: Started StreamSets Data Collector (SDC). 
Jan 20 08:38:13 bdlingprd05 systemd[1]: Starting StreamSets Data Collector (SDC)... 
Jan 20 08:38:13 bdlingprd05 streamsets[25583]: Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseG1GC" to $SDC_JAVA_OPTS 
Jan 20 08:38:14 bdlingprd05 systemd[1]: sdc.service: main process exited, code=exited, status=1/FAILURE 
Jan 20 08:38:14 bdlingprd05 systemd[1]: Unit sdc.service entered failed state. 
Jan 20 08:38:14 bdlingprd05 systemd[1]: sdc.service failed.

And trying to start the same using StreamSets command shows the following:

-bash-4.2$ ./streamsets dc -verbose
Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseG1GC" to $SDC_JAVA_OPTS
WARN: Detected unsupported JDK vendor: IBM J9 VM
ERROR: Please install supported JDK or explicitly allow unsupported JDKs via SDC_ALLOW_UNSUPPORTED_JDK=true

Looking further on the Java components may show something like,

-bash-4.2$ rpm -qa | grep java
python-javapackages-3.4.1-11.el7.noarch
java-1.8.0-ibm-1.8.0.4.10-1jpp.3.el7.x86_64
javapackages-tools-3.4.1-11.el7.noarch
-bash-4.2$ java -version java version "1.8.0" Java(TM) SE Runtime Environment (build pxa6480sr4fp10-20170727_01(SR4 FP10)) IBM J9 VM (build 2.8, JRE 1.8.0 Linux amd64-64 Compressed References 20170722_357405 (JIT enabled, AOT enabled) J9VM - R28_20170722_0201_B357405 JIT  - tr.r14.java_20170722_357405 GC  - R28_20170722_0201_B357405_CMPRSS J9CL - 20170722_357405) JCL - 20170726_01 based on Oracle jdk8u144-b01

Goal:

To be able to bring up the SDC service.

Solution:

Based on the above example, it is evident that SDC is failing because of the incompatible IBM Java version.

SDC requires Oracle JDK / Open JDK (from SDC 3.0.0.0+). If this was a functional SDC environment (in the past, before this issue), a likely candidate that could have brought about this breakage is an activity like a Linux patching on the SDC host machine. Then, most likely, some automation or orchestration tool (like Chef, Puppet) could have overwritten the Java installation.

The issue can be easily resolved once that (installing the latest Oracle JDK) is sorted out.

Scenario 2:

The same as the previous but with Cloudera Manager

Goal:

Running a new instance of SDC in Cloudera Manage parcels.

Solution:

Cloudera Manager wasn't able to access the $JAVA_HOME directory due to permissions.
Set permissions individually in the $JAVA_HOME host directory, where the problem occurs, may solve the issue.