Skip to main content

Hello,

I am going through the Data Ops Platform Fundamentals course, and currently on Lab: Deploy a Transformer Engine with a Sample Pipeline. 

I have made sure my deployment is up and running, transformer engine is up. Created a pipeline by using sample pipeline as per lab steps. 

I am getting these validation errors (attached) - Error: Directory path does not exist: file:///data/store_retail

 

Am I missing any steps?

 

Some more info:

I have not changed any parameter settings.  

The runtime env for the pipeline runtime conf - It shows origin_directory=/data/store_retail

The origin shows: Directory Path = file://${origin_directory}

The transformer engine is running. It does not show any directory /data/store_retail

bash-5.0# hostname
46ab0f17ee4f
bash-5.0# ls /data/
transformer
bash-5.0# ls /data/transformer/
blobstore                      meteringSnapshot.json          pipelines                      runInfo                        stats.json
control-hub-pushed.properties  opt-stats.json                 runHistory                     sdc.id                         statsSnapshot.json
bash-5.0# 

 

 

Hi Dhanashri,

 

When you completed the “Setup a Transformer Deployment” lab, did you run the script at the end?

If you don’t run the ./update-nodes.sh  script, the containers won’t talk to each other in Strigo.  That is likely the cause of your error.

 

Please run that script and let us know if that fixes the issue?  

Thanks! 

Hi @john.durkin 

I had run the command earlier.

To try again, I Stopped and started the deployments, verified that the engines are running. 

I have 2 data collectors and 1 transformer engine running. 

ran the ./update_nodes.sh , but still getting the same error in pipeline validation / draft run.

 

@Dhanashri_Bhate 

I think there are some issues in existing template .You can do the setup the pipeline manually by referring the below link.

 

https://github.com/streamsets/pipeline-library/tree/master/transformer/sample-pipelines/pipelines/Tx%20Retail%20Inventory%20-%20Join%20Agg%20Repartition

 

Thanks & Regards

Bikram_

Yes, I moved on with the next lab and built a similar pipeline from scratch without any issues. 

@Dhanashri_Bhate 

 

Nice . Happy learning.

Reply


Terms & Conditions