Issue: While deploying SDC in kubernetes cluster via provisioning agent. SDC deployment is up and running but data collector doesn’t show up in control hub.
To troubleshoot such issue check the provisioning agent logs
$kubectl logs $<provisioning-agent-pod>if you see the error like below
KubernetesDeploymentCoordinator - Syncing Deployment Status
2022-05-14 03:29:17,810 1thread:Deployment Status Refresher] DEBUG AbstractPodManager - Handling Bootstrap New Pod http://172.24.51.2:18630
2022-05-14 03:29:17,811 1thread:Deployment Status Refresher] ERROR SDCCredentialsDeploymentAssistant - Error accessing ping path for SDC at address http://172.24.51.2:18630
2022-05-14 03:29:17,811 1thread:Deployment Status Refresher] DEBUG SDCCredentialsDeploymentAssistant - SDC Not Accessible Error {}
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at com.streamsets.lib.security.http.RestClient$Response.getStatus(RestClient.java:185)
at com.streamsets.cluster.manager.coordinator.SDCCredentialsDeploymentAssistant.isSdcAccessible(SDCCredentialsDeploymentAssistant.java:77)
at com.streamsets.cluster.manager.kubernetes.pod.AbstractPodManager.handleNewPod(AbstractPodManager.java:70)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.lambda$handlePodActions$6(KubernetesDeploymentCoordinator.java:313)
at java.util.Optional.ifPresent(Optional.java:159)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.handlePodActions(KubernetesDeploymentCoordinator.java:304)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.lambda$syncDeploymentStatus$2(KubernetesDeploymentCoordinator.java:237)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.syncDeploymentStatus(KubernetesDeploymentCoordinator.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
This error indicates that provisioning agent is unable to communicate with the newly launched data collector pod.
To verify this connectivity run the following command from the provisioning-agent
$kubectl exec -it <provsioning-agent-pod> -- bash
$wget -S http://172.24.51.2:18630
Solution: Make sure provisioning-agent can communicate with the newly launched data-collector pod.