- During SDC startup by default SDC try connecting to repository manifest URL
2022-09-12T21:38:54,103 [user:] [pipeline:] [runner:] [thread:ManifestFetcher] [stage:]INFO StageLibraryUtil - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/5.0.0/tarball/repository.manifest.json- Now, if your SDC doesn’t have internet connectivity it will failed with the below error, However, SDC will start without any issue
2022-09-12T21:38:56,714 [user:] [pipeline:] [runner:] [thread:ManifestFetcher] [stage:] ERROR StageLibraryUtil - Failed to read repository manifest json
javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: connect timed out
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[jersey-common-2.25.1.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[jersey-common-2.25.1.jar:?]
at org.glassfish.jersey.internal.Errors.process(Errors.java:228) ~[jersey-common-2.25.1.jar:?]
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444) ~[jersey-common-2.25.1.jar:?]
at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:311) ~[jersey-client-2.25.1.jar:?]
at com.streamsets.datacollector.stagelibrary.StageLibraryUtil.getRepositoryManifestFile(StageLibraryUtil.java:93) ~[streamsets-datacollector-container-5.0.0.jar:?]
at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.getRepositoryManifestList(ClassLoaderStageLibraryTask.java:1101) ~[streamsets-datacollector-container-5.0.0.jar:?]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_332]
Caused by: java.net.SocketTimeoutException: connect timed out
at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_332]
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_332]
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_332]
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_332]
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_332]
at java.net.Socket.connect(Socket.java:607) ~[?:1.8.0_332]
at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~[?:1.8.0_332]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~[?:1.8.0_332]
at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~[?:1.8.0_332]
at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~[?:1.8.0_332]
at sun.net.www.http.HttpClient.New(HttpClient.java:339) ~[?:1.8.0_332]
at sun.net.www.http.HttpClient.New(HttpClient.java:357) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1228) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1046) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1044) ~[?:1.8.0_332]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_332]
at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:784) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1043) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1572) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:92) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1492) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1490) ~[?:1.8.0_332]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_332]
at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:784) ~[?:1.8.0_332]
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1489) ~[?:1.8.0_332]
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~[?:1.8.0_332]
at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399) ~[jersey-client-2.25.1.jar:?]
at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285) ~[jersey-client-2.25.1.jar:?]
... 13 more - You might see this ERROR trace multiple time as SDC retry to connect this link.
How to verify wether SDC instance has connectivity ?
Run the below command, you can get the exact URL depending upon the SDC version from SDC logs itself.
$curl -ivk http://archives.streamsets.com/datacollector/5.0.0/tarball/repository.manifest.json
How do I suppress this error, if it is expected in our env to not have any internet connectivity for SDC?
Please add the following line to the log config. And restart the SDC
For SDC version >= 5.0.0
logger.l5.name=com.streamsets.datacollector.stagelibrary.StageLibraryUtil
logger.l5.level=FATAL
For SDC version <= 5.0.0
log4j.logger.com.streamsets.datacollector.stagelibrary.StageLibraryUtil=FATAL