Skip to main content
Solved

Error: Directory path does not exist: file:///data/store_retail , Lab: deploy transformer pipeline using sample

  • November 9, 2022
  • 5 replies
  • 181 views

Hello,

I am going through the Data Ops Platform Fundamentals course, and currently on Lab: Deploy a Transformer Engine with a Sample Pipeline. 

I have made sure my deployment is up and running, transformer engine is up. Created a pipeline by using sample pipeline as per lab steps. 

I am getting these validation errors (attached) - Error: Directory path does not exist: file:///data/store_retail

 

Am I missing any steps?

 

Some more info:

I have not changed any parameter settings.  

The runtime env for the pipeline runtime conf - It shows origin_directory=/data/store_retail

The origin shows: Directory Path = file://${origin_directory}

The transformer engine is running. It does not show any directory /data/store_retail

bash-5.0# hostname
46ab0f17ee4f
bash-5.0# ls /data/
transformer
bash-5.0# ls /data/transformer/
blobstore                      meteringSnapshot.json          pipelines                      runInfo                        stats.json
control-hub-pushed.properties  opt-stats.json                 runHistory                     sdc.id                         statsSnapshot.json
bash-5.0# 

 

 

Best answer by Bikram

@Dhanashri_Bhate 

I think there are some issues in existing template .You can do the setup the pipeline manually by referring the below link.

 

https://github.com/streamsets/pipeline-library/tree/master/transformer/sample-pipelines/pipelines/Tx%20Retail%20Inventory%20-%20Join%20Agg%20Repartition

 

Thanks & Regards

Bikram_

View original
Did this topic help you find an answer to your question?

5 replies

john.durkin
StreamSets Employee
Forum|alt.badge.img
  • StreamSets Employee
  • 24 replies
  • November 9, 2022

Hi Dhanashri,

 

When you completed the ā€œSetup a Transformer Deploymentā€ lab, did you run the script at the end?

If you donā€™t run the ./update-nodes.sh  script, the containers wonā€™t talk to each other in Strigo.  That is likely the cause of your error.

 

Please run that script and let us know if that fixes the issue?  

Thanks! 


Hi @john.durkin 

I had run the command earlier.

To try again, I Stopped and started the deployments, verified that the engines are running. 

I have 2 data collectors and 1 transformer engine running. 

ran the ./update_nodes.sh , but still getting the same error in pipeline validation / draft run.

 


Bikram
Headliner
Forum|alt.badge.img+1
  • Headliner
  • 486 replies
  • Answer
  • November 10, 2022

@Dhanashri_Bhate 

I think there are some issues in existing template .You can do the setup the pipeline manually by referring the below link.

 

https://github.com/streamsets/pipeline-library/tree/master/transformer/sample-pipelines/pipelines/Tx%20Retail%20Inventory%20-%20Join%20Agg%20Repartition

 

Thanks & Regards

Bikram_


Yes, I moved on with the next lab and built a similar pipeline from scratch without any issues. 


Bikram
Headliner
Forum|alt.badge.img+1
  • Headliner
  • 486 replies
  • November 10, 2022

@Dhanashri_Bhate 

 

Nice . Happy learning.


Reply