SDC deployment is not working [Data Collector not showing up in SCH]


Userlevel 4
Badge
  • StreamSets Employee
  • 96 replies

Issue: While deploying SDC in kubernetes cluster via provisioning agent. SDC deployment is up and running but data collector doesn’t show up in control hub.

 

 

 

To troubleshoot such issue check the provisioning agent logs

$kubectl logs $<provisioning-agent-pod>

if you see the error like below 

KubernetesDeploymentCoordinator - Syncing Deployment Status
2022-05-14 03:29:17,810 [thread:Deployment Status Refresher] DEBUG AbstractPodManager - Handling Bootstrap New Pod http://172.24.51.2:18630
2022-05-14 03:29:17,811 [thread:Deployment Status Refresher] ERROR SDCCredentialsDeploymentAssistant - Error accessing ping path for SDC at address http://172.24.51.2:18630
2022-05-14 03:29:17,811 [thread:Deployment Status Refresher] DEBUG SDCCredentialsDeploymentAssistant - SDC Not Accessible Error {}
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:607)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558)
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at sun.net.www.http.HttpClient.New(HttpClient.java:339)
at sun.net.www.http.HttpClient.New(HttpClient.java:357)
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1226)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162)
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1056)
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1570)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at com.streamsets.lib.security.http.RestClient$Response.getStatus(RestClient.java:185)
at com.streamsets.cluster.manager.coordinator.SDCCredentialsDeploymentAssistant.isSdcAccessible(SDCCredentialsDeploymentAssistant.java:77)
at com.streamsets.cluster.manager.kubernetes.pod.AbstractPodManager.handleNewPod(AbstractPodManager.java:70)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.lambda$handlePodActions$6(KubernetesDeploymentCoordinator.java:313)
at java.util.Optional.ifPresent(Optional.java:159)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.handlePodActions(KubernetesDeploymentCoordinator.java:304)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.lambda$syncDeploymentStatus$2(KubernetesDeploymentCoordinator.java:237)
at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1384)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
at com.streamsets.cluster.manager.kubernetes.KubernetesDeploymentCoordinator.syncDeploymentStatus(KubernetesDeploymentCoordinator.java:237)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

 

This error indicates that provisioning agent is unable to communicate with the newly launched data collector pod. 

 

To verify this connectivity run the following command from the provisioning-agent

$kubectl exec -it <provsioning-agent-pod> -- bash

$wget -S http://172.24.51.2:18630

 

Solution: Make sure provisioning-agent can communicate with the newly launched data-collector pod.


0 replies

Be the first to reply!

Reply