Scenario:
User can see live statistics during the lifetime of the pipeline, when it is active and running. The same visibility is lost once the pipeline is stopped.
Goal:
To be able to view the metrics associated with the pipeline once it has been stopped. How to find them?
Solution:
Pipeline metrics JSON will be stored in the PipelineHistory.json file in <SDC-DIST>/data/runinfo/<PipelneID>/0/pipelineStateHistory.json file after the pipeline has been stopped.
It can also be retrieved using the REST API,
http://<sdc_host>:<sdc_port>/rest/v1/pipeline/<pipelineId>/history