- - Knowledge base
Product Updates
Groups
Join Build It With Brenna

Build it with Brenna: Transformer for Snowflake

(Wed, 31 May, 17:00)

Home
Community overview
Platform Q&A
StreamSets Academy

StreamSets Academy

StreamSets Academy course topics and questions

67 Topics
187 Replies

When you subscribe we will email you when there is a new topic in this category

67 Topics

Recently active

Most replies Most views Newest first

AnkurRoadie

asked in StreamSets Academy

Connect to Dynamic 365

Hi,As per the requirement I need to connect Streamsets to Dynamic 365 db.DO we have this connector and if not how we can connect it? Regards,Ankur

4 months ago

lakshmi_narayanan_tDiscovered Fame

asked in StreamSets Academy

datadrift

Difference in data rule vs data drift rule with some scenario, use case?

4 months ago

Santhosh KumarDiscovered Fame

asked in StreamSets Academy

Reattempt Test White Belt

Hi Team, Good Morning, as I participated in the test(White belt), unfortunately I am under scored so, i had another test within the month ,so I requested you to provide a coupon code or detailed information about the procedure for reattempt.

5 months ago

lakshmi_narayanan_tDiscovered Fame

asked in StreamSets Academy

data collector vs transformer

Anyone can provide a quick overview of pros and cons of data-collector vs transformer apart from streamsets document also.

5 months ago

AnkurRoadie

asked in StreamSets Academy

Santhosh KumarDiscovered Fame

asked in StreamSets Academy

Kafka connection not working ...

Hi Team,i using docker its work on JDBC connection but not working on a Kafka connection.

5 months ago

lakshmi_narayanan_tDiscovered Fame

asked in StreamSets Academy

how to saw the local fs in command terminal

below i post the pic ,anybody fix the issue .i dont known whats going wrong in pipeline or command terminal

5 months ago

SUDHANSHOOFan

asked in StreamSets Academy

Build a CDC Pipeline, unable to find - https://academy.streamsets.com/wp-content/uploads/2021/11/externalResources.zip

Hi Streamsets Team,I was using Streamsets Tutorial - Build a CDC Pipeline, Where its mentioned to use External Resource but unable to find, because this hyperlink does not map correctly.Could you please check and update.

5 months ago

SUDHANSHOOFan

asked in StreamSets Academy

java.lang.IllegalArgumentException: File not found: '/resources/transformer/db_token'

HI Team,Trying a sample Transformer pipeline - “Scala User-Defined Function”. And after creation I found this error with pipeline not with component whichever its used.CREATION_005 - Could not resolve implicit EL expression '${runtime:loadResource('db_token', false)}': com.streamsets.pipeline.api.el.ELEvalException: CTRCMN_0100 - Error evaluating expression ${runtime:loadResource('db_token', false)}: java.lang.IllegalArgumentException: File not found: '/resources/transformer/db_token'I just open the sample pipeline, didn't do anything with it. Does any one have idea regarding this?

5 months ago

tejas_surveFan

asked in StreamSets Academy

Related With REST API

I was working on StreamSet Learning Labs and after completing some learning tracks, currently I am stuck in REST API part and lab videos and instructions are not enough for me to understand it properly.So please guide me regarding these same things

5 months ago

chrisnrublocFan

asked in StreamSets Academy

SCD behaviour options missing !

HIi am trying to do the snowflake transformer course but the lab wants me to use an option in the scd processor thats not available !behaviour for new fields “Keep for inserted records and populate values from change data for updated records” all i see are remove from change and keep for newest and set null for previous ! is this the replaced version or is something different going to happen ?

5 months ago

SUDHANSHOOFan

asked in StreamSets Academy

Build a Transformer pipeline: org.apache.spark.sql.AnalysisException: Partition column `` not found in schema

Hi Team,While building a Transformer pipeline exercise, I was joining (Inner Join) two MySQL Tables from classicmodels database, employees & customers tables using condition “employees_employeeNumber = customers_salesRepEmployeeNumber”its showing below error: org.apache.spark.sql.AnalysisException: Partition column `` not found in schema struct<employees_lastName:string,employees_firstName:string,employees_email:string,employees_employeeNumber:int,employees_officeCode:string,customers_customerNumber:int,customers_salesRepEmployeeNumber:int,customers_creditLimit:decimal(10,2)>;

5 months ago

SUDHANSHOOFan

asked in StreamSets Academy

In Transformer, joining two small table but preview does not show any output for join processor

Hi All,in Transformer I’ve created a simple pipeline join 2 small MySQL table but join processor doesn’t show any output records, but previews show actual number of records to be saved in destination.While doing the same things with MySQL in backend actual record count is matched what previewed record is showing.

5 months ago

MelanieDiscovered Fame

asked in StreamSets Academy

JDBC Connection not working (Zomato)

HiI am trying to establishing a connection to jdbc:mysql://mysqldb:3306/zomato but it is not working.I tried 2 differents approach:at first, I did not want to use Strigo so I was not limited to the 8 hours. So I run my engines in containers within my local machine. The engines are running fine on the Control Hub and I can create and run some pipelines. However, I would like to use the connection cited in one of the lab with mysqldb and zomato. I am not sure how to “install” the zomato db and reviews table locally? I have a container running mysql but from there i dont know what to. If i try to create the connection using my local engine, i get then the below error.JDBC_00 - Cannot connect to specified database: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. I could see a post with the same error message but it does

5 months ago

Santhosh KumarDiscovered Fame

asked in StreamSets Academy

streamsets White Belt certification

How many attempts using the white belt certification.

5 months ago

Dhanashri_BhateOpening Band

asked in StreamSets Academy

Is it possible to use Regex in attribute name?

Hello,Can I use regex patterns to filter records based on a field name pattern? Or use them in Field Type converter? For example: If a record has field names containing “cost” in them e.g. “Average Cost for two”, “Cost range”, “Parking Costs” etc. I want to send these records to a certain destination? Or if the field names including “cost” in them, use field type converter to convert the values to integer.I did not find any specific information on this in Docs. Played around with a few patterns. But even if some passed validation, they failed in preview. -Dhanashri

6 months ago

loveStremSetsFan

asked in StreamSets Academy

Strigo Lab - Streamsets Academy:DataOps Platform Fundamentals/Lab Build a JDBC Pipeline

I am following the instructions in the Lab: Build a JDBC Pipeline. As per the instructions, I go on the Right Hand Pane and click on Set Up → Engines and then click on the Engine(s) running on Strigo and then on the top tab I click on “External Resources” and then clicking on the side of the new left panel, I click on “External Libraries” and then the Plus (+) icon, for me to Upload the External Library, I select from the pull-down menu the Stage Library for JDBC and then it asks to Browse File. At that point the video states in a pop-up block that “Files required to complete this exercise are provided in the lab guide”.I could not find the file which in the video is named “mysql-connector-java-8.0.25.jar”. I search the Windows environment through the Folder Explorer under the “This PC” for that jar file, using *.jar as the search term, with only the securitypack jar file showing up.I then googled that I could download directly that named jar file from the MySQL website and downloade

6 months ago

yash4b4Fan

asked in StreamSets Academy

Streamsets Academy login issues

Hi Team,My login credentials are not working. getting the following error on the webpage. Your connection to this site isn't secureDon't enter any sensitive information on this site (for example, passwords or credit cards). It could be stolen by attackers.

6 months ago

mr_maniiRoadie

posted in StreamSets Academy

Streamset whitebelt certification

Hi, Im planning to take up exams for Streamset whitebelt certification in the coming days. Is there any inputs that anyone can provide around how to attempt for the same?

6 months ago

suresh_venkataFan

asked in StreamSets Academy

Unable to pause Strigo "The workspace cannot be paused"

Hi Team,I’m Unable to pass the strigo last 2hours. its Consumed 2hours time Already Please Help me on this.

6 months ago

Dhanashri_BhateOpening Band

asked in StreamSets Academy

Unable to locate this UI option for support bundle

Hello,I am going through the troubleshooting training material on the StreamSets Academy website. Course link: https://academy.streamsets.com/courses/troubleshooting-and-support/lessons/generating-a-support-bundle-in-the-streamsets-cloud-control-hub/Unable to locate this UI (in the screenshot) shown in the training video at 1:00 onwards till 3:11. Is it not accessible during training?

6 months ago

Dhanashri_BhateOpening Band

asked in StreamSets Academy

Error: Directory path does not exist: file:///data/store_retail , Lab: deploy transformer pipeline using sample

Hello,I am going through the Data Ops Platform Fundamentals course, and currently on Lab: Deploy a Transformer Engine with a Sample Pipeline. I have made sure my deployment is up and running, transformer engine is up. Created a pipeline by using sample pipeline as per lab steps. I am getting these validation errors (attached) - Error: Directory path does not exist: file:///data/store_retail Am I missing any steps? Some more info:I have not changed any parameter settings. The runtime env for the pipeline runtime conf - It shows origin_directory=/data/store_retailThe origin shows: Directory Path = file://${origin_directory}The transformer engine is running. It does not show any directory /data/store_retailbash-5.0# hostname46ab0f17ee4fbash-5.0# ls /data/transformerbash-5.0# ls /data/transformer/blobstore meteringSnapshot.json pipelines runInfo stats.jsoncontrol-hub-pushed.properties opt-stats.json

6 months ago

Page 2 / 3

Badge winners

AkshayJadhavhas earned the badge Eager to help
john.durkinhas earned the badge Eager to help
samhas earned the badge Eager to help
saleempothiwalahas earned the badge Eager to help
Bikramhas earned the badge Community Champion 🏅

Show all badges

Terms & Conditions

Sign up

Already have an account? Login

Social Login

Username *

E-mail address *

What I do... *

Data Leader Data Architect Data Engineer Data Scientist Other

Company *

Country *

Zip Code *

Marketing Communications

Yes No

Password *

I have read and Agree to the Website Terms of Service and I have read and acknowledged the Privacy Policy.

loginBox.register.email_repeat

Login to the community

No account yet? Create an account

Social Login

Username or Email

Password

Remember me

Forgot password?

Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.

Username or e-mail

Back to overview

Scanning file for viruses.

Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.

This file cannot be downloaded

Sorry, our virus scanner detected that this file isn't safe to download.