Solved

Gettting error zomato directory does not exist

  • 3 November 2021
  • 6 replies
  • 127 views

Userlevel 2

I’m following the DataOps Platform Fundamentals course and the Build a Pipeline chapter has you enter “/zomato” as a files directory in the configuration of a Directory source.

However, when validating the pipeline I get the error: SPOOLDIR_12-Directory ‘/zomato’ does not exist: conf.spoolDir.

Any solution for this?

icon

Best answer by dima 3 November 2021, 16:31

View original

6 replies

Userlevel 2
Badge

Make sure the Directory exists. Keep in mind that if you're running Data Collector in Docker, the path would be inside the container.

Userlevel 2

Thank you.

So there are crucial steps missing from the course. They could have at least mentioned to create the zomato directory in the container and also download the actual csv dataset file from the zomato website.

I’m completely new to streamsets and also to the devops role to be honest, and as beginners, we expect courses to have the complete information needed for the hands on part to actually work.

Just a quick mention, in the course, of creating the directory and downloading the dataset file would have been enough to point me in the right direction to make the pipeline work.

Userlevel 1

SebastienM - Thanks for the feedback.  We’ll review the lab instructions for clarity and make the necessary edits.  We incorporate feedback as we receive it so we appreciate you taking the time to document it.

Userlevel 1

Added the tag “DataOps Platform Fundamentals Course” 

Userlevel 1

I found the source of the error.

This error occurs if you don’t follow the prior “Setup a Deployment” lab exactly.  

 

Step 9 shows:
 

  1. Critical Step - Run the following script. Copy and paste it as is from this document into your lab environment and run it. It may take a few seconds to run:

 

sudo docker container ls -n 3 | tail -n +2 | awk '{ print $1 }' | while read line; do sudo docker network connect streamsets-core $line; sudo docker network connect streamsets-integrations $line; sudo docker network connect streamsets-cooked $line ; sudo docker cp training-dependencies/zomato/ $line:zomato; sudo docker update --restart=always $line; echo "set up complete for $line"; done

 

That script sets up your Zomato directory so that you can complete later exercises.  Hope that helps!

 

Userlevel 2

I found the source of the error.

This error occurs if you don’t follow the prior “Setup a Deployment” lab exactly.  

 

Step 9 shows:
 

  1. Critical Step - Run the following script. Copy and paste it as is from this document into your lab environment and run it. It may take a few seconds to run:

 

sudo docker container ls -n 3 | tail -n +2 | awk '{ print $1 }' | while read line; do sudo docker network connect streamsets-core $line; sudo docker network connect streamsets-integrations $line; sudo docker network connect streamsets-cooked $line ; sudo docker cp training-dependencies/zomato/ $line:zomato; sudo docker update --restart=always $line; echo "set up complete for $line"; done

 

That script sets up your Zomato directory so that you can complete later exercises.  Hope that helps!

 

 

FYI, I followed every step of the tutorial from the start, and copy, pasted and executed those parts of the tutorial that had terminal commands.

Reply