Build a Slowly Changing Dimension with Transformer for Snowflake

  • 12 March 2022
  • 0 replies
  • 20 views

Userlevel 1
  • StreamSets Employee
  • 0 replies

This pipeline is populating and updating a Slowly Changing Dimension for the NYC Citibike stations using Transformer for Snowflake. The pipeline is using StreamSets’ Slowly Changing Dimension stage that handles all of the logic for Type 1 and Type 2 dimensions. The SCD stage then produces the correct output along with the metadata that the Snowflake Destination uses to perform the merge.


0 replies

Be the first to reply!

Reply