30-Day Free Trial: It’s Never Been Easier To Get Started With StreamSets
Environment:Streamsets Data Collector v4.2.1. Kafka Multi-topic consumer origin. Issue:It has been observed that Kafka origin is failing with the following error message when “produce single records” configuration is on + data is in JSON format.2021-01-10 15:31:12,193 [user:test] [pipeline:test_kafka_working] [runner:] [thread:Kafka pipeline-thread1] [stage:KafkaMultitopicConsumer_01] INFO MultiKafkaSource - Multi kafka thread halted unexpectedly: KAFKA_29 - Error fetching data from Kafka: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0java.util.concurrent.ExecutionException: com.streamsets.pipeline.api.StageException: KAFKA_29 - Error fetching data from Kafka: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at com.streamsets.pipeline.stage.origin.multikafka.MultiKafkaSource.produce(MultiKafkaSource.java:163) at com.streamsets.pipeline.ap
Hi,We need to export jar files/libraries of specific engine of one control hub to another control hub using Python SDK. How can we achieve it? Thanks.
Today is the last day to participate in the 2023 Wisdom of Crowds mid-year research process and receive a complimentary report of study findings and a gift from StreamSets as a thank you. Dresner Advisory Services is inviting all business and IT users to participate in its annual examination of the state of the BI, analytics, and information infrastructure marketplaces focusing on drivers, usage, and products. The 2023 mid-year survey covers Analytical Data Infrastructure (ADI), as well as Embedded BI, Data Engineering, Workforce Planning and Analysis, and Supply Chain Planning and Analysis. Participants only need to fill in areas that are relevant to their current usage and experience. Users in all roles and throughout all industries are invited to contribute their insight via an online survey. The corresponding Dresner Advisory Services research will be published beginning in 4Q23 and throughout 1Q24. Qualified survey participants will receive complimentary copies of the findings.
The Speakatoo API functions correctly in POSTMAN, but it isn't working as expected in my integration for Text To Speech conversion. What might be the issue?
we upgraded data collector to v5.6.0 from v5.5.0 recently… but since then getting our s3 pipelines are failing with below ERROR.. we are using `Instance Profile` as authentication method..Also data collector with v5.5.0 which are deployed same (AWS acc)environment are working fines with s3 pipelines & instance_profile authentication method. anyone else faced similar kind of problem?RETRY: S3_21 - Unable to write object to Amazon S3, reason : com.amazonaws.services.s3.model.AmazonS3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method. (Service: Amazon S3; Status Code: 403; Error Code: SignatureDoesNotMatch; Request ID: 3VFHG02TG4GR3KTR; S3 Extended Request ID: PePeTM0p51ZFcECJ/Mw6PU9nK4Km6AWNEaWhcykqj2ovDW/IbGcMtcaK+xcRxlMoPuRlCyxh2vo=; Proxy: null), S3 Extended Request ID: PePeTM0p51ZFcECJ/Mw6PU9nK4Km6AWNEaWhcykqj2ovDW/IbGcMtcaK+xcRxlMoPuRlCyxh2vo=
My csv file has only Two Record But it shows ,extra records in stream sets Environment.this is my csv record data but Building Information column had a paragraph data.In stream sets 5 records Showing .anyone can suggest how to handle this file.
How can I change the type of a new added field in the Expression Evaluator? Because if I add a field like this: Then the field is defined as a string. But I need to have it as DATETIMEI know I could just not add this field but my goal is to make the pipeline creating not existing tables if needed. So I need to declare all attributes, even they are empty. Thanks
Previously on all data collector below 5.X version , The behaviour was that all SDC logs were included in the Support Bundle. Now only the current log(sdc.log) is included and sdc-*.log logs are missing.The solution for the fix was applied post 5.4 version onwards.
We can see the list of jars that has been loaded during SDC start up by below configuration,If using Cloudera Manager, Please add the following to the Data Collector Advanced Configuration Snippet (Safety Valve) for sdc-env.sh configuration and restart SDC: export SDC_JAVA_OPTS="-Dstreamsets.bootstrap.debug=true ${SDC_JAVA_OPTS}" This will tell us what jars are being picked up in the classpath when SDC starts. The output will appear in stdout.log of the Data Collector process. If we are using standalone SDC, we can add the above snippet directly to sdc-env.sh ($SDC_HOME/libexec)and restart the sdc.
Our goal is to sync between Oracle and Snowflake with CDC Oracle. Some of our source tables in Oracle are defined as DATE. So we created this tables also in Snowflake with the same type, DATE. But now I get this error message: But in this case the field ACTIVITY_TIME is DATE on Oracle and Snowflake.Later I noticed that we don’t have any problem if I convert all DATE fields into DATETIME attributes on Snowflake. But looking in the documentation, there is nothing mentioned that we have to use DATETIME in such cases. The type DATE is explicitly mentioned as supported type. Does anyone had the same issues? I mean it’s not so unusual to use the DATE type in a database
Hi Team, Does StreamSets support HL7 data format? I know there is a python way of doing this. Thanks,Meher
Issue:-While adding a json data to the MongoDB collection, following error has been observed.MONGODB_13 - Error serializing record: java.lang.UnsupportedOperationException: BSON Converter cannot convert java.util.ArrayList to BSON DocumentSolution:- MongoDB atlas was reading the record as a list. As the JSON data was enclosed with the [].We have to set the JSON Content to JSON array of objects at the Origin side.
Hi,I am trying to load data into snowflake in SDC. But, I am getting below error. Can you please tell me where I am doing wrong because it is working fine in Transfomer for snowflake for the same snowflake account url. In SDC it is not working if I am using the locator on account tab. SNOWFLAKE_11 - Could not create SQL DataSource: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: JDBC driver encountered communication error. Message: Exception encountered for HTTP request: Certificate for <kxxxxxx.eu-north-1.snowflakecomputing.com> doesn't match any of the subject alternative names: [*.prod3.us-west-2.snowflakecomputing.com, *.us-west-2.snowflakecomputing.com, *.global.snowflakecomputing.com, *.snowflakecomputing.com, *.prod3.us-west-2.aws.snowflakecomputing.com]. Thanks,Malathi
can you please suggest any other configuration or proseccor to achieve output.i reading my file in directory origin origin is :configuration my destination is localfs:configuration after execution of my pipeline :-out put file like - But my requirement is like how to acive my fie format ,by using any processor jython evaluvator or whole file transformer.
Become a leader!
Already have an account? Login
No account yet? Create an account
Enter your username or e-mail address. We'll send you an e-mail with instructions to reset your password.
Sorry, we're still checking this file's contents to make sure it's safe to download. Please try again in a few minutes.
Sorry, our virus scanner detected that this file isn't safe to download.
