Skip to main content
  • During SDC startup by default SDC try connecting to repository manifest URL 
2022-09-12T21:38:54,103 0user:] :pipeline:] :runner:] :thread:ManifestFetcher] rstage:]INFO  StageLibraryUtil - Reading from Repository Manifest URL: http://archives.streamsets.com/datacollector/5.0.0/tarball/repository.manifest.json
  • Now, if your SDC doesn’t have internet connectivity it will failed with the below error, However, SDC will start without any issue
    2022-09-12T21:38:56,714 ,user:] epipeline:] nrunner:] ethread:ManifestFetcher] hstage:] ERROR StageLibraryUtil - Failed to read repository manifest json
    javax.ws.rs.ProcessingException: java.net.SocketTimeoutException: connect timed out
    at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:287) ~8jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:252) ~5jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:684) ~8jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.client.JerseyInvocation$1.call(JerseyInvocation.java:681) ~8jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~1jersey-common-2.25.1.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~9jersey-common-2.25.1.jar:?]
    at org.glassfish.jersey.internal.Errors.process(Errors.java:228) ~2jersey-common-2.25.1.jar:?]
    at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:444) ~4jersey-common-2.25.1.jar:?]
    at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:681) ~8jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:411) ~1jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.client.JerseyInvocation$Builder.get(JerseyInvocation.java:311) ~1jersey-client-2.25.1.jar:?]
    at com.streamsets.datacollector.stagelibrary.StageLibraryUtil.getRepositoryManifestFile(StageLibraryUtil.java:93) ~9streamsets-datacollector-container-5.0.0.jar:?]
    at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.getRepositoryManifestList(ClassLoaderStageLibraryTask.java:1101) ~0streamsets-datacollector-container-5.0.0.jar:?]
    at java.lang.Thread.run(Thread.java:750) ~5?:1.8.0_332]
    Caused by: java.net.SocketTimeoutException: connect timed out
    at java.net.PlainSocketImpl.socketConnect(Native Method) ~o?:1.8.0_332]
    at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~5?:1.8.0_332]
    at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~0?:1.8.0_332]
    at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~8?:1.8.0_332]
    at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~9?:1.8.0_332]
    at java.net.Socket.connect(Socket.java:607) ~0?:1.8.0_332]
    at sun.net.NetworkClient.doConnect(NetworkClient.java:175) ~7?:1.8.0_332]
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:463) ~6?:1.8.0_332]
    at sun.net.www.http.HttpClient.openServer(HttpClient.java:558) ~5?:1.8.0_332]
    at sun.net.www.http.HttpClient.<init>(HttpClient.java:242) ~4?:1.8.0_332]
    at sun.net.www.http.HttpClient.New(HttpClient.java:339) ~3?:1.8.0_332]
    at sun.net.www.http.HttpClient.New(HttpClient.java:357) ~5?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1228) ~2?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1162) ~6?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1046) ~4?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection$6.run(HttpURLConnection.java:1044) ~4?:1.8.0_332]
    at java.security.AccessController.doPrivileged(Native Method) ~o?:1.8.0_332]
    at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:784) ~8?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1043) ~4?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:990) ~9?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1572) ~7?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.access$200(HttpURLConnection.java:92) ~9?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1492) ~9?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection$9.run(HttpURLConnection.java:1490) ~9?:1.8.0_332]
    at java.security.AccessController.doPrivileged(Native Method) ~o?:1.8.0_332]
    at java.security.AccessController.doPrivilegedWithCombiner(AccessController.java:784) ~8?:1.8.0_332]
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1489) ~8?:1.8.0_332]
    at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) ~8?:1.8.0_332]
    at org.glassfish.jersey.client.internal.HttpUrlConnector._apply(HttpUrlConnector.java:399) ~9jersey-client-2.25.1.jar:?]
    at org.glassfish.jersey.client.internal.HttpUrlConnector.apply(HttpUrlConnector.java:285) ~8jersey-client-2.25.1.jar:?]
    ... 13 more
     
  • You might see this ERROR trace multiple time as SDC retry to connect this link.

 

How to verify wether SDC instance has connectivity ?

Run the below command, you can get the exact URL depending upon the SDC version from SDC logs itself.

$curl -ivk http://archives.streamsets.com/datacollector/5.0.0/tarball/repository.manifest.json 

 

 

How do I suppress this error, if it is expected  in our env to not have any internet connectivity for SDC? 

 

Please add the following line to the log config. And restart the SDC

For SDC version >= 5.0.0

logger.l5.name=com.streamsets.datacollector.stagelibrary.StageLibraryUtil
logger.l5.level=FATAL

 

For SDC version <= 5.0.0

log4j.logger.com.streamsets.datacollector.stagelibrary.StageLibraryUtil=FATAL

 

Be the first to reply!

Reply