Community Articles and Got a Question?

Hi All, i am trying to start the multiple jobs, but throwing an error, but stopping of multiple jobs working as expected. Stop: /jobrunner/rest/v1/jobs/stopJobs ==> Workingstart: /jobrunner/rest/v1/jobs/startJobs ==> failling

I have a pipeline that gathers records from an HTTP client. At one point in the pipeline I have a stream selector that filters out records that I don’t want and sends them to trash, while remaining records gets written to an S3 destination. At the end is a pipeline finisher executor.The issue is when there are no remaining records after the stream selector, ie. the data contained no relevant records. In this case the pipeline job remains running indefinitely, and I have to stop it manually. How can I stop the pipeline when there are no records left after the stream selector?

Hi Team, I am using Strigo lab environment for my training and it suddenly stopped working . I have attached the error screenshot. Please help me resolve the same. 

Hi I am trying my lab to connect to My Sql I did the following In Deployment menu, I configured all the JDBC LibraryIn the engine I downloaded mysql-connector jar and restarted the engine In the connection I gave the connection string, username and password, however when I test connection I am getting the following error JDBC_00 - Cannot connect to specified database: java.sql.SQLException: No suitable driver found for jdbc:mysql://mysqldb:3306/zomatoIs there anything I am missing here.? 

I am trying to fetch all the records from the API using http client and selecting pagination option.The processor is not giving any error but its only giving the records for 1st page only.Attaching the screenshot for refrence.value list contains 1000 records but the page size of the api is 100. I need to fetch all the records.

We have Streamsets pipeline for generating Postgres incremental records using Logical replication slots.It ran successfully for few days and generated the files.Then after few days Streamsets are running and are in active state, but the incremental files are not getting generated. We dont see any errors in Streamsets and Postgres logs. The replication slots are getting filled up in the backend.Do, someone knows the rootcause of this issue and any solution to fix this.Please suggest. Thank you,Jose Kattakayam 

I have a pipeline that uses a JDBC Query Consumer, running in incremental mode. The source table is in MSSQL.My offset column is in datetime type, and is never null (although nullable).While the job is running, I can retrieve the offset values in the GUI. However, after the job finishes successfully, when I try to retrieve the offset values, the result is empty.In the data table, there are about 250k records, belonging to 5 distinct dates. I can’t see why the latest offset cannot be saved.In the select query, I have this → SELECT * FROM dbo.PHIN_Bankruptcy_Load WHERE ReportDate > '${OFFSET}' ORDER BY ReportDateI would appreciate any tip that can help me overcome this problem - how to make the pipeline save the offset at the end of the run?

When I create a new fragment I get an error message as “Invalid Fragment. No open Input or Output lanes”. Anything that I miss here? 

Hi, I am getting SNOWFLAKE_56 - Key fields not specified for table 'ORCL_EMP' errorWhen Table Auto create is enabled, CDC also enabled and given table key column as ID column 

I am using kafka consumer as an origin and the event is expected at particular time. I want to send the notification if the event is not published at the expected time. How can this be achieved in streamsets? 

I have a SQL Script with multiple drop & create DDL(Create tables As Select *), I want to run them at one go. I am quite new to informatica powercenter, can some one provide the process of using SQL transformation for BigQuery in informatica.Sample Query:- informatica transformation

Hi All,Is there any way to get the response body for other http code in HTTP client processor? The API (POST method) I am calling, it gives error reason as response for error records with http code  400 in postman. In streamsets I want to get the error reason as response but getting 400 and record is going in error files.Thanks in advance for your help. Regards,Anjalee

How to get the data collector 3.22.3 version's activation key ?

Our requirement is to collect all records and do some processing. That we have done using JavaScript array. We want to write that array to the output processor post completion of all batch. sdc.output.write() function is not working in the destroy block. Is there anyway to achieve the above requirement? Is there any way to hold writing output and only do output once all batches are finished.

I have input coming from source as records. Record-1 is xyz, record 2 is abc…I need to merge all these values into a list by checking few conditions in groovy.How do I achieve that. Finally output should come in groovy when reading all records is over.like [xyz,abc,pqr] But I am getting code output asOutput for record-1 [xyz]output for record2 -[abc] and so onCan someone please help me how its possible?

How we can capture AWS S3 Events in streamsets on cloud,which service we can use here? 

Hi All, I have created the installation script and tried running the docker image in strigo environment. However, it instantly throws the below error. docker: Got permission denied while trying to connect to the docker daemon socket  Could someone please let me know what is that I am doing it wrongly? Any remedy? Thank you!Yuvaraj Thiyagarajan

My JDBC producer stage is giving the error java.lang.NoClassDefFoundError: org/apache/commons/lang3/StringUtils Has anyone had a similar issue before?

I want my pipeline to automatically trigger only when the source file is placed on the server. How can we achieve this in streamsets?

Hi Team,As, I am new to Streamsets want to have more information about Rest Service.We have databases on on-prem, We run procedure on database that initiate the rest API call to tomcat(service published on this) web server on port 8908 redirectedNow we receive one url which contain different parameter. I need to configure the same in Streamsets.Can you please let me know, How we can configure same in rest service.  

Hi team,I am using field renamer for mssql column name rename from origin mssql “regionid” to target snowflake “RegionID”.Attchment 1 is my configure for Field Rename configuration. When I ran “previewer”, did not show any errors.Attachment 2 is log.Cannot figured out what is the exact reason for the failure from the log.Can you suggest?

Hi, community,  I am using Streamset data collector for data migration from PostgreSQL to Cassandra But while testing the Cassandra connection I am encountering the above error. 

How to initiate the emr cluster through streamset

When checked at path Computer Configuration -> Administrative Templates -> Windows Components -> Windows Update, I don't see any detail for WSUS. Please tell what's the other way to check WSUS server detailsCyberArk Vault 

Has anybody used GraphQL as data source? I’ve been tried to connect it using StreamSets, but, no luck!