Solved

Getting error: VALIDATION_0011 - Stage has open output streams

  • 4 November 2021
  • 11 replies
  • 215 views

Userlevel 2

I get this error, following the steps in the lecture “DataOps Platform Fundamentals”, chapter “Build a Pipeline”, lab “Lab: Build a Pipeline”.

When clicking on the error, it takes me to the Stream Selector module, Configuration, General tab. There is nothing highlighted in that section and all seems in working order to me.

Could you please advise?

icon

Best answer by sam 4 November 2021, 16:38

View original

11 replies

Userlevel 2
Badge

Hi @SebastienM,

The issue is because you have declared multiple output streams and not all of them are connected to a destination or another processor or executor

Userlevel 2

The output streams I have declared are as per the steps in the lecture. The lecture says to have two outputs: Trash and Local FS. And then have a Stream Selector with a condition to only keep records from the source that match the condition.

I have followed all the steps in the lectures so far and it results in that error. So there are crucial steps missing from the lectures, namely that of selecting an output after the Local FS?

Userlevel 2
Badge

Hi @SebastienM,

The problem is related to how the stages are connected, can you please attach a screenshot of your pipeline?

@Dash are you aware of any known issues with the lectures?

Userlevel 2

This is the screenshot of my pipeline, which matches the setup in the lecture:

 

The error points to the Stream Selector but I see no issue there and the lecture does not mention to do anything in the General tab:

 

Userlevel 2
Badge

Ok, so you can see that the 1st output stream is empty, probably the “Conditions” tab of your Stream Selector has an extra row adding one more output stream

Userlevel 2

Yes, I added a condition to only accept values from the “Rating text” column of the csv dataset file that are of string record type:

 

I was under the assumption that conditions entered in the Conditions tab had a nested behaviour. Did not know it expected a separate output, especially that there is no setting/field in the Conditions tab to specify where to direct the output of each condition to. Also the error icon (the warning question mark) points to the General tab, not the Conditions tab. So I had no indication that the issue was at a conditions level.

Because without that condition I get another error:

 

I checked the dataset and there is no “\u2018” value in the “/Rating text” column of the csv file downloaded from zomato. So my solution was to add another condition to only accept values from the “Rating text” column of the csv dataset file that are of string record type.

 

It looks like your quotes in the condition (not the dataset) that works are straight quotes ('), but the one that does not work uses a ‘LEFT SINGLE QUOTATION MARK' (U+2018) (‘). Try replacing the quotes with straight quotes and make sure each opening quote has a matching closing quote.

Userlevel 2

It looks like your quotes in the condition (not the dataset) that works are straight quotes ('), but the one that does not work uses a ‘LEFT SINGLE QUOTATION MARK' (U+2018) (‘). Try replacing the quotes with straight quotes and make sure each opening quote has a matching closing quote.

 

I copy-pasted the expression for this condition from the lecture itself at https://academy.streamsets.com/courses/dataops-platform-fundamentals/lessons/build-a-pipeline/topic/lab-build-a-pipeline/:

 

I have now corrected the quotes and get a different error:

 

That should be:

${record:value('/Rating text') == 'Very good' || record:value('/Rating text') == 'Excellent' }

Userlevel 2

That should be:

${record:value('/Rating text') == 'Very good' || record:value('/Rating text') == 'Excellent' }

 

Corrected the ‘:’ in ‘record:value’. I was then able to validate the pipeline and preview the results. Thank you.

It seems the expression in the lecture needs correcting.

Userlevel 5
Badge

Added the tag “DataOps Platform Fundamentals Course” 

*Tags will be distinguish which course within this category. 

Reply