Hi, I need to know more about the sdc.log files which are available there :
Administration/Logs/Downloads
I need to know :
Then, I would like to know if the logs are lost after a Restart Data collector (manually)
Thanks, Flo
Best answer by AkshayJadhav
You would find the latest logs in the sdc.log then sdc-9.log then sdc-8.log. Here is the last log entry on all the files which explains the same.
#tail -n1 sdc-1.log
2023-04-03 21:18:50,445 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelConverterContextImpl - resolveProperty [collection type; class java.util.List, contains [simple type, class com.streamsets.datacollector.restapi.rbean.lang.BaseMsg]]
#tail -n1 sdc-2.log
2023-04-03 21:18:51,771 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelResolver - Can't check class [simple type, class java.lang.Object], java.lang.Object
#tail -n1 sdc-3.log
2023-04-03 21:18:51,996 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelConverterContextImpl - resolve [simple type, class com.streamsets.datacollector.restapi.bean.AntennaDoctorMessageJson]
#tail -n1 sdc-4.log
2023-04-03 21:18:52,238 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelConverterContextImpl - resolveProperty [collection type; class java.util.List, contains [simple type, class com.streamsets.datacollector.restapi.rbean.lang.BaseMsg]]
#tail -n1 sdc.log
2023-04-03 22:27:57,211 [user:*?] [pipeline:-] [runner:] [thread:managerExecutor-pool-3-thread-1] [stage:] DEBUG StandaloneAndClusterPipelineManager - Triggering previewer cache cleanupIt is happening because of the SizeBasedTriggeringPolicy policy and max files are set to 10.
Hope this helps.
Thank you - AkshayJ
Hi
You can control the logs file size and the total log files by updating the sdc-log4j.properties file. The same can be available via SDC UI » Administration » Log Config » Logs section.
appender.streamsets.policies.size.size=2MB ==> Log file size.
appender.streamsets.strategy.max = 4 ===> Total files.
The logger would not delete any log files from the data collector log directory.
Thank you - AkshayJ
Hello again, I have an additional question.
With the following configuration :
This means that whenever an sdc.log file reaches 256MB :
And this happens because of the parameter MaxBackupIndex, right ?
*The files I am talking about are presented in the first comment in the first picture.
You would find the latest logs in the sdc.log then sdc-9.log then sdc-8.log. Here is the last log entry on all the files which explains the same.
#tail -n1 sdc-1.log
2023-04-03 21:18:50,445 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelConverterContextImpl - resolveProperty [collection type; class java.util.List, contains [simple type, class com.streamsets.datacollector.restapi.rbean.lang.BaseMsg]]
#tail -n1 sdc-2.log
2023-04-03 21:18:51,771 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelResolver - Can't check class [simple type, class java.lang.Object], java.lang.Object
#tail -n1 sdc-3.log
2023-04-03 21:18:51,996 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelConverterContextImpl - resolve [simple type, class com.streamsets.datacollector.restapi.bean.AntennaDoctorMessageJson]
#tail -n1 sdc-4.log
2023-04-03 21:18:52,238 [user:] [pipeline:] [runner:] [thread:main] [stage:] DEBUG ModelConverterContextImpl - resolveProperty [collection type; class java.util.List, contains [simple type, class com.streamsets.datacollector.restapi.rbean.lang.BaseMsg]]
#tail -n1 sdc.log
2023-04-03 22:27:57,211 [user:*?] [pipeline:-] [runner:] [thread:managerExecutor-pool-3-thread-1] [stage:] DEBUG StandaloneAndClusterPipelineManager - Triggering previewer cache cleanupIt is happening because of the SizeBasedTriggeringPolicy policy and max files are set to 10.
Hope this helps.
Thank you - AkshayJ
Hi
If there no further question, please mark the comment as a best answer that will close the thread.
Thank you,
Akshay Jadhav
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
