Question

Sorting a specific column and writing it to a new table

  • 27 July 2023
  • 0 replies
  • 17 views

Badge

Use Case:-

We have a dataset, in which we have columns as follows:-

FIRST_NAME, LAST_NAME, EMAIL, PHONE, GENDER, DEPARTMENT, JOB_TITLE, YEARS_OF_EXPEREIENCE, SALARY. 

 

Lets sort the column SALARY in ascending order and write to a new table with just 4 columns, FIRST_NAME, LAST_NAME, YEARS_OF_EXPEREIENCE, SALARY.

 

Pipeline Design:-

  1. Snowflake Table (origin)
  2. Sort (Processor)
  3. Column Remover (Processor)
  4. Snowflake Table (Destination)

 

Pipeline Working:-

  1. Snowflake Origin will fetch the table and columns and pass the records to Sort processor
  2. Sort processor will sort the data based on the configuration and pass it to Column Remover. (SALARY column, Ascending order)
  3. Column Remover will keep or remove the columns based on the configuration.
  4. Snowflake Table Destination will write the data to a new snowflake table.

 

 


0 replies

Be the first to reply!

Reply