Solved

Logging more details to Data Collector Errors ?

  • 9 March 2022
  • 2 replies
  • 79 views
robert.bernhard
Rank

I am quite new with solace and wonder if there is any info anyone can share to better find out details for errors. The errors are often not very speak able in user interface and therefore i look for more information like:

  • Where to find the detailed logs (with stack trace and steps before and after) from pipeline runs ?
    In a gcp hosted environment i found something on: /var/log/sdc but on a self managed docker the folder even does not exist.

    I am aware when a step fails you can see the errors in data collector UI but
    i got now a null pointer exception and even don’t see where it failed. 
  • Connection Errors:
    Where can i see more details to errors shown on connection wizard ?
    I had one error telling driver could not be loaded and finally the problem was a driver from
    a different database. (oracle driver error made postgres connection fail)

Many thanks in advance for any help on that.

icon

Best answer by Giuseppe Mura 10 March 2022, 11:21

View original

2 replies

Giuseppe Mura
Rank
Userlevel 3
Badge

hi @robert.bernhard 

 

the first question has a simple answer - the Data Collector running in Docker will generate logs in /logs you’ll find sdc.log in there.

Whether the SDC is running Docker or your GCP machine, you should be able to see the logs directly in the UI (Engines → Select the specific engine → Logs)

The second question should find an answer in the first - when creating / editing a connection, you have an authoring data collector performing the connection to the data source, any errors will be generated and logged by that SDC, and will be found in the UI (as above) or in the logs directory on the machine.

I hope this helps!

robert.bernhard
Rank

Yes it helped but i am not clear how to find the most detailed log info.
I checked for a null pointer exception the log and i set the log4j settings to DEBUG.
Still what i get there is not sufficient to understand the problem.

It seems problem is in generaion of query so it would be helping what query was tried to create.
So in error scenarios i would log out as much as possible to understand the error case.
That does not flood the logs as errors hopefully occur seldom. 

java.lang.NullPointerException
    at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:878)
    at com.google.common.base.Joiner.toString(Joiner.java:452)
    at com.google.common.base.Joiner.appendTo(Joiner.java:106)
    at com.google.common.base.Joiner.appendTo(Joiner.java:152)
    at com.google.common.base.Joiner.join(Joiner.java:195)
    at com.google.common.base.Joiner.join(Joiner.java:185)
    at com.streamsets.pipeline.lib.jdbc.JdbcUtil.generateQuery(JdbcUtil.java:1253)

 

I opened for that now a support ticket:
Request Id: 24464 and Pedro Mora is assigned to it.

Reply


Terms & Conditions