I get this error, following the steps in the lecture “DataOps Platform Fundamentals”, chapter “Build a Pipeline”, lab “Lab: Build a Pipeline”.
When clicking on the error, it takes me to the Stream Selector module, Configuration, General tab. There is nothing highlighted in that section and all seems in working order to me.
Could you please advise?
Hi
The issue is because you have declared multiple output streams and not all of them are connected to a destination or another processor or executor
The output streams I have declared are as per the steps in the lecture. The lecture says to have two outputs: Trash and Local FS. And then have a Stream Selector with a condition to only keep records from the source that match the condition.
I have followed all the steps in the lectures so far and it results in that error. So there are crucial steps missing from the lectures, namely that of selecting an output after the Local FS?
Hi
The problem is related to how the stages are connected, can you please attach a screenshot of your pipeline?
This is the screenshot of my pipeline, which matches the setup in the lecture:
The error points to the Stream Selector but I see no issue there and the lecture does not mention to do anything in the General tab:
Ok, so you can see that the 1st output stream is empty, probably the “Conditions” tab of your Stream Selector has an extra row adding one more output stream
Yes, I added a condition to only accept values from the “Rating text” column of the csv dataset file that are of string record type:
I was under the assumption that conditions entered in the Conditions tab had a nested behaviour. Did not know it expected a separate output, especially that there is no setting/field in the Conditions tab to specify where to direct the output of each condition to. Also the error icon (the warning question mark) points to the General tab, not the Conditions tab. So I had no indication that the issue was at a conditions level.
Because without that condition I get another error:
I checked the dataset and there is no “\u2018” value in the “/Rating text” column of the csv file downloaded from zomato. So my solution was to add another condition to only accept values from the “Rating text” column of the csv dataset file that are of string record type.
It looks like your quotes in the condition (not the dataset) that works are straight quotes ('), but the one that does not work uses a ‘LEFT SINGLE QUOTATION MARK' (U+2018) (‘). Try replacing the quotes with straight quotes and make sure each opening quote has a matching closing quote.
It looks like your quotes in the condition (not the dataset) that works are straight quotes ('), but the one that does not work uses a ‘LEFT SINGLE QUOTATION MARK' (U+2018) (‘). Try replacing the quotes with straight quotes and make sure each opening quote has a matching closing quote.
I copy-pasted the expression for this condition from the lecture itself at https://academy.streamsets.com/courses/dataops-platform-fundamentals/lessons/build-a-pipeline/topic/lab-build-a-pipeline/:
I have now corrected the quotes and get a different error:
That should be:
${record:value('/Rating text') == 'Very good' || record:value('/Rating text') == 'Excellent' }
That should be:
${record:value('/Rating text') == 'Very good' || record:value('/Rating text') == 'Excellent' }
Corrected the ‘:’ in ‘record:value’. I was then able to validate the pipeline and preview the results. Thank you.
It seems the expression in the lecture needs correcting.
Added the tag “DataOps Platform Fundamentals Course”
*Tags will be distinguish which course within this category.
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
