Skip to main content
Solved

Error: Directory path does not exist: file:///data/store_retail , Lab: deploy transformer pipeline using sample

  • November 9, 2022
  • 5 replies
  • 180 views

Hello,

I am going through the Data Ops Platform Fundamentals course, and currently on Lab: Deploy a Transformer Engine with a Sample Pipeline. 

I have made sure my deployment is up and running, transformer engine is up. Created a pipeline by using sample pipeline as per lab steps. 

I am getting these validation errors (attached) - Error: Directory path does not exist: file:///data/store_retail

 

Am I missing any steps?

 

Some more info:

I have not changed any parameter settings.  

The runtime env for the pipeline runtime conf - It shows origin_directory=/data/store_retail

The origin shows: Directory Path = file://${origin_directory}

The transformer engine is running. It does not show any directory /data/store_retail

bash-5.0# hostname
46ab0f17ee4f
bash-5.0# ls /data/
transformer
bash-5.0# ls /data/transformer/
blobstore                      meteringSnapshot.json          pipelines                      runInfo                        stats.json
control-hub-pushed.properties  opt-stats.json                 runHistory                     sdc.id                         statsSnapshot.json
bash-5.0# 

 

 

Best answer by Bikram

@Dhanashri_Bhate 

I think there are some issues in existing template .You can do the setup the pipeline manually by referring the below link.

 

https://github.com/streamsets/pipeline-library/tree/master/transformer/sample-pipelines/pipelines/Tx%20Retail%20Inventory%20-%20Join%20Agg%20Repartition

 

Thanks & Regards

Bikram_

View original
Did this topic help you find an answer to your question?

5 replies

john.durkin
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 24 replies
  • November 9, 2022

Hi Dhanashri,

 

When you completed the “Setup a Transformer Deployment” lab, did you run the script at the end?

If you don’t run the ./update-nodes.sh  script, the containers won’t talk to each other in Strigo.  That is likely the cause of your error.

 

Please run that script and let us know if that fixes the issue?  

Thanks! 


Hi @john.durkin 

I had run the command earlier.

To try again, I Stopped and started the deployments, verified that the engines are running. 

I have 2 data collectors and 1 transformer engine running. 

ran the ./update_nodes.sh , but still getting the same error in pipeline validation / draft run.

 


Bikram
Headliner
Forum|alt.badge.img+1
  • Headliner
  • 486 replies
  • Answer
  • November 10, 2022

@Dhanashri_Bhate 

I think there are some issues in existing template .You can do the setup the pipeline manually by referring the below link.

 

https://github.com/streamsets/pipeline-library/tree/master/transformer/sample-pipelines/pipelines/Tx%20Retail%20Inventory%20-%20Join%20Agg%20Repartition

 

Thanks & Regards

Bikram_


Yes, I moved on with the next lab and built a similar pipeline from scratch without any issues. 


Bikram
Headliner
Forum|alt.badge.img+1
  • Headliner
  • 486 replies
  • November 10, 2022

@Dhanashri_Bhate 

 

Nice . Happy learning.


Reply