Skip to main content
Question

Retreaving more metrics fom Jobs(SDK)

  • February 17, 2025
  • 1 reply
  • 29 views
E
Fan

Hello!!

Currently, I’m using the SDK to extract metrics from some jobs. However, instead of having only {run_count, input_count, output_count, total_error_count} from sch.job.metrics, I would like to have some other metrics as well, such as counting each record for each step of my job. Is this possible? How can I achieve this?

I’m also looking to get the last received record (for each pipeline) that we can see in the real-time summary. How can I get this metric from ControlHub using the SDK?

1 reply

palmeida123
Fan

Hello,

 

I’m with the same question. Trying to mount a data pipeline monitoring from many tools, and give a simple overview of the landscape. So i’m consuming API’s from the respective tools to mount a unified dashboard. But i’m not able to find how to collect the last received record from a job or pipeline on streamsets. Any sugestion ?! 

 

Reply


Terms & Conditions