Skip to main content
Solved

How can I set a schema without having to run the preview?

  • February 10, 2022
  • 1 reply
  • 10 views

Drew Kreiger
Rock star
Forum|alt.badge.img
  • Senior Community Builder at StreamSets
  • 95 replies

How can I set a schema without having to run the preview?

Best answer by Drew Kreiger

Data Collector performs a schema-on-read, so you should preview the data at least once to capture the schema. There is an option in the Preview Configuration to save the preview record schema which Determines whether the preview record schema is saved in the pipeline configuration. By default, the schema is not saved. When you close and then reopen the pipeline, the Schema tab for each pipeline stage is empty. You must run preview again to update the input and output schema. When selected, the schema is saved after the pipeline is closed.

View original
Did this topic help you find an answer to your question?

1 reply

Drew Kreiger
Rock star
Forum|alt.badge.img
  • Author
  • Senior Community Builder at StreamSets
  • 95 replies
  • Answer
  • February 10, 2022

Data Collector performs a schema-on-read, so you should preview the data at least once to capture the schema. There is an option in the Preview Configuration to save the preview record schema which Determines whether the preview record schema is saved in the pipeline configuration. By default, the schema is not saved. When you close and then reopen the pipeline, the Schema tab for each pipeline stage is empty. You must run preview again to update the input and output schema. When selected, the schema is saved after the pipeline is closed.


Reply