StreamSets Academy

StreamSets Academy course topics and questions

  • 85 Topics
  • 215 Replies

85 Topics

M
Discovered Fame
MelanieDiscovered Fame
asked in StreamSets Academy

JDBC Connection not working (Zomato)

HiI am trying to establishing a connection to jdbc:mysql://mysqldb:3306/zomato   but it is not working.I tried 2 differents approach:at first, I did not want to use Strigo so I was not limited to the 8 hours. So I run my engines in containers within my local machine. The engines are running fine on the Control Hub and I can create and run some pipelines. However, I would like to use the connection cited in one of the lab with mysqldb and zomato. I am not sure how to “install” the zomato db and reviews table locally? I have a container running mysql but from there i dont know what to.         If i try to create the connection using my local engine, i get then the below error.JDBC_00 - Cannot connect to specified database: com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. I could see a post with the same error message but it does

16
R
StreamSets Employee
9 months ago
Praveen P
Fan
Praveen PFan
asked in StreamSets Academy

how to write avro file in wholefile format from kafka to local fs?

i trying to write avro format as wholefile format from kafka to localfs but i got error.below  i share the pipeline ,can you please suggest how to done this 

13
Praveen P
Fan
3 months ago
SebastienM
Roadie
SebastienMRoadie
asked in StreamSets Academy

Getting error: VALIDATION_0011 - Stage has open output streams

I get this error, following the steps in the lecture “DataOps Platform Fundamentals”, chapter “Build a Pipeline”, lab “Lab: Build a Pipeline”.When clicking on the error, it takes me to the Stream Selector module, Configuration, General tab. There is nothing highlighted in that section and all seems in working order to me.Could you please advise?

11
Drew Kreiger
Rock star
1 year ago
D
Opening Band
Dhanashri_BhateOpening Band
asked in StreamSets Academy

Where can I find Transpose processor documentation?

Hello,I am looking for documentation for the transpose processor in transformer pipelines.I searched here : https://docs.streamsets.com/portal/platform-transformer/latest/search.html?searchQuery=transpose but did not find it.Can someone share the link to the relevant documentation?-Dhanashri 

10
Bikram
Headliner
6 months ago
SebastienM
Roadie
SebastienMRoadie
asked in StreamSets Academy

Gettting error zomato directory does not exist

I’m following the DataOps Platform Fundamentals course and the Build a Pipeline chapter has you enter “/zomato” as a files directory in the configuration of a Directory source.However, when validating the pipeline I get the error: SPOOLDIR_12-Directory ‘/zomato’ does not exist: conf.spoolDir.Any solution for this?

8
Masoom
Fan
1 year ago
S
Discovered Fame
Santhosh KumarDiscovered Fame
asked in StreamSets Academy

file transfer from origin to destination

hi,I want to transfer multiple files from origin to  destination like any other destination ,how can I achieve these use case .any one can suggest  .

8
Bikram
Headliner
7 months ago
S
Discovered Fame
Santhosh KumarDiscovered Fame
asked in StreamSets Academy

active file ,file tail

hi,I want to use the file tail origin but i didn’t known the active file path usage  in sdc. anyone can provide the configuration of file tail and active file path . 

7
S
Discovered Fame
7 months ago
lakshmi_narayanan_t
Discovered Fame
lakshmi_narayanan_tDiscovered Fame
asked in StreamSets Academy

directory not able ready xml file .

ERROR LIKEin my directory xml file but not able to fetch records and also i increased max record length-but file length is less than 1mb but i don’t know why unable to fetch my xml data any one can find solution and share it. 

6
saleempothiwala
Headliner
1 month ago
A
Roadie
akshayp2Roadie
asked in StreamSets Academy

Subscriptions created for create incident for failed job in ServiceNow has not triggered.

I have created subscription as suggested in related topic but that subscription is not triggered.  I am able to create incident at pipeline level when pipeline fails, using the notification tab by giving webhook details for incident table.Also, I want some information about job failure in incident table for e.g. Short description like job_name is ‘abc’ failed and their error details. Is this possible using subscription? If yes then please help me on the same.

6
XavierV
StreamSets Employee
4 months ago
loveStremSets
Fan
loveStremSetsFan
asked in StreamSets Academy

Strigo Lab - Streamsets Academy:DataOps Platform Fundamentals/Setup a Deployment

When I am in the Strigo Lab simulator environment, it has 2 simultaneous sessions running currently (a Windows desktop and a “Streamset Cluster” [Ubuntu shell]). When I am doing Setup a Deployment part of the training, it requires me to copy and paste the into the Ubuntu shell a docker run command to Setup the Environment.  This docker command is quite long (several rows) as it includes the Deployment ID and Tokens.  However I can not copy paste from the Strigo Windows environment into the Ubuntu shell as it is two separate environments. The only way would be to MANUALLY type each and every character which would not work if I did a single typo.  Is there any work around?

5
loveStremSets
Fan
1 year ago
D
Opening Band
Dhanashri_BhateOpening Band
asked in StreamSets Academy

Error: Directory path does not exist: file:///data/store_retail , Lab: deploy transformer pipeline using sample

Hello,I am going through the Data Ops Platform Fundamentals course, and currently on Lab: Deploy a Transformer Engine with a Sample Pipeline. I have made sure my deployment is up and running, transformer engine is up. Created a pipeline by using sample pipeline as per lab steps. I am getting these validation errors (attached) - Error: Directory path does not exist: file:///data/store_retail Am I missing any steps? Some more info:I have not changed any parameter settings.  The runtime env for the pipeline runtime conf - It shows origin_directory=/data/store_retailThe origin shows: Directory Path = file://${origin_directory}The transformer engine is running. It does not show any directory /data/store_retailbash-5.0# hostname46ab0f17ee4fbash-5.0# ls /data/transformerbash-5.0# ls /data/transformer/blobstore                      meteringSnapshot.json          pipelines                      runInfo                        stats.jsoncontrol-hub-pushed.properties  opt-stats.json                

5
Bikram
Headliner
10 months ago
lakshmi_narayanan_t
Discovered Fame
lakshmi_narayanan_tDiscovered Fame
asked in StreamSets Academy

jdbc tee processor not write data in microsoft sql server for control hub 3.x

hi,I am facing issue , to write  a data in  Microsoft Sql server database  by  using JDBC tee processor but its not work .I tried the identity insert on and off in database level .if it Possibile or not .suggest any processor to  achieve this .ERRORdemo pipelinejdbc tee configuration  

5
lakshmi_narayanan_t
Discovered Fame
7 months ago
loveStremSets
Fan
loveStremSetsFan
asked in StreamSets Academy

Strigo Lab - Streamsets Academy:DataOps Platform Fundamentals/Lab Build a JDBC Pipeline

I am following the instructions in the Lab: Build a JDBC Pipeline. As per the instructions, I go on the Right Hand Pane and click on Set Up → Engines and then click on the Engine(s) running on Strigo and then on the top tab I click on “External Resources” and then clicking on the side of the new left panel, I click on “External Libraries” and then the Plus (+) icon, for me to Upload the External Library, I select from the pull-down menu the Stage Library for JDBC and then it asks to Browse File.  At that point the video states in a pop-up block that “Files required to complete this exercise are provided in the lab guide”.I could not find the file which in the video is named “mysql-connector-java-8.0.25.jar”.  I search the Windows environment through the Folder Explorer under the “This PC” for that jar file, using *.jar as the search term, with only the securitypack jar file showing up.I then googled that I could download directly that named jar file from the MySQL website and downloade

4
M
Discovered Fame
9 months ago
M
Fan
MadhumithaFan
asked in StreamSets Academy

Any references for preparation of the Streamsets White belt Exam

Can I get any references for the preparation for Streamsets white belt exam other than the Streamsets Academy content…?

4
Praveen P
Fan
6 months ago
deepikamani007
Fan
deepikamani007Fan
asked in StreamSets Academy

field type converter

If i converting string type date to date type means the output show wrong. 

4
deepikamani007
Fan
6 months ago
D
Opening Band
Dhanashri_BhateOpening Band
asked in StreamSets Academy

Is it possible to use Regex in attribute name?

Hello,Can I use regex patterns to filter records based on a field name pattern? Or use them in Field Type converter? For example: If a record has field names containing “cost” in them e.g. “Average Cost for two”, “Cost range”, “Parking Costs” etc. I want to send these records to a certain destination? Or if the field names including “cost” in them, use field type converter to convert the values to integer.I did not find any specific information on this in Docs. Played around with a few patterns. But even if some passed validation, they failed in preview.  -Dhanashri 

4
D
Opening Band
9 months ago
R
Fan
rohit6488Fan
asked in StreamSets Academy

Missing Fragment used in Academy Course for CDC from Mysql to Snowflake

Hi All,i am looking for Fragment which has been used in Academy Course for CDC Pipeline Demo, they have mentioned they will share with us. But i cant find it anywhere. Can someone help me to share that fragment. Regards,Rohit

4
R
Fan
3 months ago
lakshmi_narayanan_t
Discovered Fame
lakshmi_narayanan_tDiscovered Fame
asked in StreamSets Academy

how to implement the sql parser processor

how to execute the sql parser processor in pipeline ,anyone can provide the pipeline.

3
saleempothiwala
Headliner
7 months ago
ufmurphy
Opening Band
ufmurphyOpening Band
asked in StreamSets Academy

DataOps Platform Fundamentals Course - Data Drift missing

Hey guys, In the DataOps Platform Fundamentals Course, the video on Data Drift is missing.https://academy.streamsets.com/courses/dataops-platform-fundamentals/lessons/data-drift-2/ Same with CI/CD:https://academy.streamsets.com/courses/dataops-platform-fundamentals/lessons/implementing-pipeline-ci-cd/ Is there a video, or should I mark it complete and move on? Jim

3
ufmurphy
Opening Band
1 year ago
ufmurphy
Opening Band
ufmurphyOpening Band
asked in StreamSets Academy

Runtime parameters: reset origin and start

Hey Community, I see the option in the Streamsets canvas:Start pipeline Reset origin and start Start with parameters Is it possible to combine the last 2 options?Reset origin and start with parameters

3
ufmurphy
Opening Band
1 year ago
M
Roadie
mr_maniiRoadie
posted in StreamSets Academy

Streamset whitebelt certification

Hi, Im planning to take up exams for Streamset whitebelt certification in the coming days. Is there any inputs that anyone can provide around how to attempt for the same?

3
saleempothiwala
Headliner
10 months ago
D
Opening Band
Dhanashri_BhateOpening Band
asked in StreamSets Academy

Unable to locate this UI option for support bundle

Hello,I am going through the troubleshooting training material on the StreamSets Academy website. Course link: https://academy.streamsets.com/courses/troubleshooting-and-support/lessons/generating-a-support-bundle-in-the-streamsets-cloud-control-hub/Unable to locate this UI (in the screenshot) shown in the training video at 1:00 onwards till 3:11. Is it not accessible during training?  

3
D
Opening Band
10 months ago
S
Fan
suresh_venkataFan
asked in StreamSets Academy

Unable to pause Strigo "The workspace cannot be paused"

Hi Team!I’m Unable to Pause My Strigo . Iam Facing Error like this“The workspace cannot be paused while labs are still be provisioned. Once the labs are ready, you will be able to pause your workspace.”Requesting To You Please help me on this 

3
john.durkin
StreamSets Employee
6 months ago
D
Opening Band
Dhanashri_BhateOpening Band
asked in StreamSets Academy

Email notifications for alerts are not working

Hello,I have a pipeline that reads a log file and sends alerts when CRITICAL category message is seen.The alert works fine, I get notification on Control Hub UI, I am also getting Webhook notifications.But the email notification does not seem to work.Settings done in the Rules section for email: In the Data Rule - Send Email = true, In the notifications tab a valid email id is given.Is there any other setting that overrides this? Did not see any user level preference setting option for this. Thanks,Dhanashri 

3
Bikram
Headliner
9 months ago
S
Discovered Fame
Santhosh KumarDiscovered Fame
asked in StreamSets Academy

Reattempt Test White Belt

Hi Team, Good Morning, as I participated in the test(White belt), unfortunately I am under scored so, i had another test within the month ,so I requested you to provide a coupon code or detailed information about the procedure for reattempt.

3
Bikram
Headliner
8 months ago

Badge winners

Show all badges
Terms & Conditions