Solved

Gettting error zomato directory does not exist

  • 3 November 2021
  • 8 replies
  • 998 views

Userlevel 2

I’m following the DataOps Platform Fundamentals course and the Build a Pipeline chapter has you enter “/zomato” as a files directory in the configuration of a Directory source.

However, when validating the pipeline I get the error: SPOOLDIR_12-Directory ‘/zomato’ does not exist: conf.spoolDir.

Any solution for this?

icon

Best answer by dima 3 November 2021, 16:31

View original

8 replies

Userlevel 3
Badge

Make sure the Directory exists. Keep in mind that if you're running Data Collector in Docker, the path would be inside the container.

Userlevel 2

Thank you.

So there are crucial steps missing from the course. They could have at least mentioned to create the zomato directory in the container and also download the actual csv dataset file from the zomato website.

I’m completely new to streamsets and also to the devops role to be honest, and as beginners, we expect courses to have the complete information needed for the hands on part to actually work.

Just a quick mention, in the course, of creating the directory and downloading the dataset file would have been enough to point me in the right direction to make the pipeline work.

Userlevel 2
Badge

SebastienM - Thanks for the feedback.  We’ll review the lab instructions for clarity and make the necessary edits.  We incorporate feedback as we receive it so we appreciate you taking the time to document it.

Userlevel 2
Badge

Added the tag “DataOps Platform Fundamentals Course” 

Userlevel 2
Badge

I found the source of the error.

This error occurs if you don’t follow the prior “Setup a Deployment” lab exactly.  

 

Step 9 shows:
 

  1. Critical Step - Run the following script. Copy and paste it as is from this document into your lab environment and run it. It may take a few seconds to run:

 

sudo docker container ls -n 3 | tail -n +2 | awk '{ print $1 }' | while read line; do sudo docker network connect streamsets-core $line; sudo docker network connect streamsets-integrations $line; sudo docker network connect streamsets-cooked $line ; sudo docker cp training-dependencies/zomato/ $line:zomato; sudo docker update --restart=always $line; echo "set up complete for $line"; done

 

That script sets up your Zomato directory so that you can complete later exercises.  Hope that helps!

 

Userlevel 2

I found the source of the error.

This error occurs if you don’t follow the prior “Setup a Deployment” lab exactly.  

 

Step 9 shows:
 

  1. Critical Step - Run the following script. Copy and paste it as is from this document into your lab environment and run it. It may take a few seconds to run:

 

sudo docker container ls -n 3 | tail -n +2 | awk '{ print $1 }' | while read line; do sudo docker network connect streamsets-core $line; sudo docker network connect streamsets-integrations $line; sudo docker network connect streamsets-cooked $line ; sudo docker cp training-dependencies/zomato/ $line:zomato; sudo docker update --restart=always $line; echo "set up complete for $line"; done

 

That script sets up your Zomato directory so that you can complete later exercises.  Hope that helps!

 

 

FYI, I followed every step of the tutorial from the start, and copy, pasted and executed those parts of the tutorial that had terminal commands.

Even after running the above script , “Zomato folder is showing after running the bash command cd   training-dependencies  and than ls  command“   but still same issue is getting  while previewing the pipeline

I found the source of the error.

This error occurs if you don’t follow the prior “Setup a Deployment” lab exactly.  

 

Step 9 shows:
 

  1. Critical Step - Run the following script. Copy and paste it as is from this document into your lab environment and run it. It may take a few seconds to run:

 

sudo docker container ls -n 3 | tail -n +2 | awk '{ print $1 }' | while read line; do sudo docker network connect streamsets-core $line; sudo docker network connect streamsets-integrations $line; sudo docker network connect streamsets-cooked $line ; sudo docker cp training-dependencies/zomato/ $line:zomato; sudo docker update --restart=always $line; echo "set up complete for $line"; done

 

That script sets up your Zomato directory so that you can complete later exercises.  Hope that helps!

After running the set of given scripts,  it is showing “ training-dependencies/zomato no such file or directory ”

 

Reply