Skip to main content

Welcome

Welcome to the StreamSets Community

105 Topics
A
Anonymous
published in Welcome Guide
Last Day! Wisdom of Crowds End-User Survey - Participate Now

Today is the last day to participate in the 2023 Wisdom of Crowds mid-year research process and receive a complimentary report of study findings and a gift from StreamSets as a thank you. Dresner Advisory Services is inviting all business and IT users to participate in its annual examination of the state of the BI, analytics, and information infrastructure marketplaces focusing on drivers, usage, and products.  The 2023 mid-year survey covers Analytical Data Infrastructure (ADI), as well as Embedded BI, Data Engineering, Workforce Planning and Analysis, and Supply Chain Planning and Analysis. Participants only need to fill in areas that are relevant to their current usage and experience. Users in all roles and throughout all industries are invited to contribute their insight via an online survey. The corresponding Dresner Advisory Services research will be published beginning in 4Q23 and throughout 1Q24. Qualified survey participants will receive complimentary copies of the findings.  

160
A
1 year ago
A
Anonymous
published in Welcome Guide
Write a review & receive a $10 gift cardBlog

Hi StreamSets Community! We want to hear from you and ask if you’d share your experience using StreamSets on Capterra. When you write an honest review of your experience on Capterra, the first 30 qualifying reviewers will receive a $10 gift card as a thank you. Your review will help users of technology products and services consider us as their partner for solving their data integration challenges.We appreciate your honest feedback and thank you for your time.Submit your review now

600
A
1 year ago
A
Anonymous
published in Events & Webinars
New Course: Agile Reporting with StreamSets

We are proud to announce a new, free, and completely self-paced course focused on real world scenarios: Agile Reporting with StreamSets.Learn how to streamline financial reporting through modern, resilient, and repeatable data integration with StreamSets.  Quickly and securely integrate and migrate data from internal and external sources, legacy systems, and files into modern data platforms such as Snowflake to accelerate financial reporting.This self-paced hands-on training course teaches you how to overcome several common challenges experienced by organizations attempting to accelerate reporting capabilities:"hard to access" data in legacy systems data silos and data sprawl heavy reliance on IT and a lack of self-service access moving data from legacy to modern data environments integrating internal and external data sources including filesAudienceThe course is designed for data engineers and anyone who is interested in learning how to use StreamSets to solve reporting challenges.Enr

910
A
1 year ago
Drew Kreiger
Rock star
Drew KreigerSenior Community Builder at StreamSets
posted in Welcome Guide
New to the StreamSets Community? Everything to get you started. 🚀

​New to the StreamSets Community? If so, this is the place. Welcome!  First thing first: We are here for YOU. If you are a first-time StreamSets user, customer, partner or a StreamSets Rock star, we have you covered. The StreamSets community is the go-to place to learn, share, and grow your knowledge with other like-minded professionals. If you want to go quickly, go alone. If you want to go far, go together. - African proverb Take a tour of the general welcome community posts.-Please head over to our community guidelines page to learn:Who we are a company and a community team Unacceptable Behavior/ Guidelines (Do’s and Dont’s) Code Of Conduct Communication Guidelines Reporting Harassment or Other IssuesOther Platform related helpful topics:  5 tips to ask your question the right way Question vs Conversation Meet the Community | Introduce yourself Community Platform Theme, Bages, and Ranks DocumentationLearn how to make the most of StreamSets with user guides and tutorials here StreamS

8363
sean
StreamSets Employee
3 years ago
Drew Kreiger
Rock star
Drew KreigerSenior Community Builder at StreamSets
posted in Welcome Guide
StreamSets Community Guidelines

Welcome to the StreamSets Community! Please read over the following Community Guidelines and Code of Conduct below. Please also view StreamSets work culture Blog and Privacy policy. Who we areThe StreamSets Community Is a global community of data engineers who set out to learn, share, and expand their DataOps skills.  “If you want to go fast, go alone. If you want to go far, go together” – African Proverb GuidelinesThe Do’s and Don’ts Do’s: Share helpful docs/resources, tips & tricks, and useful questions Network with other StreamSets users Help answer questions via the forum  Challenge and push boundaries Be positive, Be kindDon’ts: Share personal information Attack/Call out others Degrade other members based on their knowledge level  Promote a personal brand or company Code Of Conduct The StreamSets Community is dedicated to providing a harassment-free, equitable and inclusive community experience for everyone. We do not tolerate harassment of members in any form. We take this po

2321
Drew Kreiger
Rock star
3 years ago
aimè leon dore
Fan
aimè leon doreFan
posted in  Show us your Pipelines
The Intersection of Streetwear and High Fashion

In the ever-evolving world of fashion, few brands have made as significant an impact in such a short amount of time as Aimé Leon Dore. Founded in 2014 by Teddy Santis, the New York-based label has carved out a unique space at the intersection of streetwear and high fashion. Known for its timeless designs, cultural references, and sophisticated appeal, Aimé Leon Dore has quickly become a name synonymous with elevated, luxury street style.The Vision Behind Aimé Leon DoreTeddy Santis, the creative force behind Aimé Leon Dore, sought to create a brand that transcended the typical streetwear mold. While many labels focus on bold logos and oversized designs, Santis envisioned a line that celebrated heritage and craftsmanship while embracing modernity. Aimé Leon Dore’s collections are heavily inspired by New York City, blending the city's vibrant street culture with refined, minimalist silhouettes. It’s a celebration of the American experience, drawing influences from classic Americana, sport

90
aimè leon dore
Fan
8 days ago
L
Fan
lotihig137Fan
asked in  Show us your Pipelines
getting Error: [Local FS 1 - Directory Template] Base directory path could not be created

I am trying to build a pipeline by following the steps mentioned in the streamsets docs- https://docs.streamsets.com/portal/platform-controlhub/controlhub/UserGuide/GettingStarted/Try.html#task_q3r_p2x_k4b but getting an issue while writing to the local fils system.the issue resides in the directory template [Directory Template info/HDFS_output/${YYYY()}-${MM()}-${DD()}-${hh()}-${every(5,mm())}codeHADOOPFS_41 - Base directory path could not be created]Can anyone help? Do we have a set of configuration that needs to be installed before running streamsets?

90
L
Fan
18 days ago
J
Fan
jojodoFan
posted in  Show us your Pipelines
MongoDB Atlas Connection seems to have a bug.

I was trying to add a MongoDB Atlas connection and on clicking the test connection button, I am getting an error “[unauthorized] not authorized on local to execute command”. I tested the connection string and credentials in Mongo Compass and there it was working fine. So, I could not understand why the test connection is failing. After many tries, I just thought of saving the connection as it is and try it in the pipeline. Magically when used in the pipeline I was able to fetch the data from MongoDB. Then again when I tried the test connection and was getting the same error.

90
J
Fan
1 month ago
J
Fan
jojodoFan
posted in  Show us your Pipelines
Oracle CDC client pipeline keep running without processing any records, but in DBeaver same account can get data from V$LOGMNR_CONTENTS

I build a pipeline using Oracle CDC client, it is a very simple pipeline , have attached my exported pipeline. I currently using sysdba access configured in Streamsets and using this account I ran below in DBeaver can get records from  V$LOGMNR_CONTENTS, please refer to attached screenshot "DBeaver_logmnr_screeashot.png"", however the streamsets cdc pipeline keeps running without any input and output, I also attached the sdc.log from server.from the log I can see the pipeline has gotten the timestamp of the starting SCN operation , however it cannot get records from LOGMNR and then insert into destination.  

30
J
Fan
1 month ago
L
Fan
lanek92292Fan
posted in  Show us your Pipelines
50 Cápsulas de Amor Propio PDF Gratis Sara Espejo

Libro 50 Cápsulas de Amor Propio PDF Gratis, 50 Cápsulas de Amor Propio PDF Gratis, 50 Cápsulas de Amor Propio PDF Descargar Gratis. Descubre el Libro 50 Cápsulas de Amor Propio PDF.. ➤➤➤➤ DESCARGAR 50 CAPSULAS DE AMOR PROPIO PDF GRATIS CLIC AQUI .El libro ’50 Cápsulas de Amor Propio’ ofrece una guía integral para redescubrir el sentido de la vida. Al embarcarnos en este viaje de autodescubrimiento, es fundamental reconocer la importancia de encontrar un propósito y una pasión que nos impulsen a seguir adelante. El amor propio juega un rol crucial en este proceso, ya que nos permite entender nuestras necesidades y deseos más profundos, y así orientar nuestras acciones hacia una vida más significativa.Encontrar propósito no es una tarea sencilla, pero ’50 Cápsulas de Amor Propio’ nos proporciona herramientas y reflexiones para identificar nuestras pasiones y metas. El libro nos ayuda a conectarnos con nuestro yo interior y a escuchar nuestra intuición. Este autoconocimiento es esencial

309590
L
Fan
1 month ago
K
Fan
kebig59243Fan
posted in  Show us your Pipelines
El Codigo De La Abundancia PDF Gratis

El Codigo De La Abundancia PDF Gratis, El Codigo De La Abundancia Jorge Muñoz Parral PDF Gratis, Libro El Codigo De La Abundancia PDF Gratis.En busca de la prosperidad económica y el bienestar emocional, muchas personas se han preguntado cuál es el secreto de las personas más ricas del mundo. ¿Cómo han logrado acumular tanto dinero y vivir en abundancia?. ➤➤➤➤ EL CODIGO DE LA ABUNDANCIA PDF GRATIS CLIC AQUI .La respuesta puede encontrarse en el libro «El Código de la Abundancia«. Este libro revela información que ha sido utilizada por las personas más ricas a lo largo de la historia para atraer el dinero y crear una vida llena de abundancia y bienestar.El Código de la Abundancia es un método probado que ha sido utilizado desde hace miles de años. Se basa en la idea de que todos tenemos el potencial de desbloquear nuestra prosperidad y crear una vida llena de abundancia en todas las áreas.. ➤➤➤➤ EL CODIGO DE LA ABUNDANCIA PDF COMPLETO + BONOS GRATIS CLIC AQUI . 

1310
K
Fan
1 month ago
H
Fan
haceya8112Fan
posted in  Show us your Pipelines
Te lo Ocultaron PDF Gratis

Libro Te lo Ocultaron PDF Gratis, Ebook Te lo Ocultaron PDF Gratis, Te lo Ocultaron PDF Descargar Gratis.Libro «Te lo Ocultaron»: Descubre la verdad oculta.. ➤➤➤➤ LIBRO TE LO OCULTARON PDF GRATIS CLIC AQUI .Te lo Ocultaron es tu billete a un universo de secretos vetados y conspiraciones entrelazadas. Explora las teorías de conspiración y los eventos históricos alterados.4 increíbles bonos gratis con Te lo Ocultaron: Ellos Gobiernan, Contacto Encubierto, Silenciados y Tu energía ¿A quién se la das?Prepárate para un viaje emocionante que desafiará tus conocimientos y te invitará a pensar críticamente.. ➤➤➤➤ LIBRO TE LO OCULTARON PDF + BONOS GRATIS CLIC AQUI . 

7290
H
Fan
1 month ago
T
Fan
tewoho3642Fan
posted in  Show us your Pipelines
Desbloquea Tu Mente Millonaria PDF Gratis

Desbloquea Tu Mente Millonaria PDF Gratis, Libro Desbloquea Tu Mente Millonaria PDF Gratis, Desbloquea Tu Mente Millonaria PDF Descargar Gratis. Desbloquea tu mente millonaria: Pensar, actuar y crear riqueza como los millonarios.. ➤➤➤➤ DESBLOQUEA TU MENTE MILLONARIA PDF GRATIS CLIC AQUI .Descubre cómo cambiar tu mentalidad hacia el dinero y alcanzar el éxito financiero con el libro ‘Desbloquea tu mente millonaria‘.Obtén 5 bonus exclusivos gratis y únete a una comunidad en línea de personas comprometidas con su crecimiento personal y financiero. ¡Transforma tu vida y alcanza la abundancia y la prosperidad que deseas!Una de las claves principales que se revela en «Desbloquea tu mente millonaria» es la importancia de cambiar nuestra mentalidad hacia el dinero. Muchas personas tienen creencias limitantes sobre la riqueza y el éxito financiero, lo que les impide alcanzar su máximo potencial.El libro Desbloquea tu mente millonaria te guiará a través de ejercicios prácticos y reflexiones prof

23300
T
Fan
1 month ago
SeduccionRemasterizada
Fan
SeduccionRemasterizadaFan
posted in  Show us your Pipelines
Seduccion Remasterizada PDF Gratis

Libro Seducción Remasterizada: La obra maestra respaldada por expertos en seducción, persuasión y psicología aplicada.. ➤➤➤➤ SEDUCCION REMASTERIZADA PDF GRATIS CLIC AQUI .Si estás buscando mejorar tus habilidades sociales y dominar el arte de la seducción, el libro «Seducción Remasterizada» es tu mejor opción. «Seducción Remasterizada» ha sido creado por expertos en el campo de la seducción, la persuasión y la psicología aplicada, y te brindará todas las herramientas necesarias para seducir mujeres de manera efectiva.Aprende cómo desarrollar tu carisma y ser más atractivo ante las mujeres.Descubre cómo seducir a cualquier mujer sin importar tu apariencia física o tu situación económica.Y mucho mas…. ➤➤➤➤ SEDUCCION REMASTERIZADA PDF + BONOS GRATIS CLIC AQUI . 

12360
SeduccionRemasterizada
Fan
1 month ago
Jdididis18
Fan
Jdididis18Fan
asked in  Show us your Pipelines
hi there I have a question about a new one for the and

Hi there I have a question about a new one for the and then you can always come over and watch the game with the and then you can always come over and watch the game with the and then you can always come over and 

30
Jdididis18
Fan
1 month ago
T
Fan
TRUCOSPSICOLOGIAFan
posted in  Show us your Pipelines
Trucos Psicologia Oscura PDF Gratis

La psicología oscura es una disciplina fascinante que puede ayudarnos a mejorar nuestras habilidades de comunicación, detectar mentiras y engaños, y influir en las decisiones de los demás.. ➤➤➤➤ TRUCOS PSICOLOGIA OSCURA PDF + BONOS GRATIS CLIC AQUI .Si estás interesado en aprender más sobre la psicología oscura, te recomendamos Trucos de Psicología Oscura.Recuerda que el conocimiento es poder, y mientras más aprendas sobre este tema, más efectivo serás en su aplicación.  

1580
T
Fan
1 month ago
B
Fan
bozo16Fan
asked in  Show us your Pipelines
"What are pipelines?" box gone.

Doing the tutorial, fragments section (https://academy.streamsets.com/learn/courses/6/streamsets-platform-fundamentals/lessons/172/lab-create-a-new-fragment) I’m invite to import a pipeline, but I don’t have the box “What are pipelines?” with the import link. Maybe I closed the box at some stage, but how doI get it back?Screenshot of my Pipelines 

372
B
Fan
2 months ago
P
Fan
PratikSinhaFan
asked in  Show us your Pipelines
Error: [Local FS 1 - Directory Template] Base directory path could not be created (HADOOPFS_41)

I am trying to build a pipeline by following the steps mentioned in the streamsets docs- https://docs.streamsets.com/portal/platform-controlhub/controlhub/UserGuide/GettingStarted/Try.html#task_q3r_p2x_k4b but getting an issue while writing to the local fils system.the issue resides in the directory template [Directory Template info/HDFS_output/${YYYY()}-${MM()}-${DD()}-${hh()}-${every(5,mm())}codeHADOOPFS_41 - Base directory path could not be created]Can anyone help? Do we have a set of configuration that needs to be installed before running streamsets?

180
P
Fan
5 months ago
M
Fan
malikhas15Fan
asked in  Show us your Pipelines
Help Needed: Issues Connecting Streamsets JMS Producer to Solace

I have a Streamsets pipeline where I am using the SQL Server CDC component and the JMS Producer component. My connection to SQL Server is working perfectly, and I am receiving data in my pipeline from SQL Server. However, I am encountering issues when trying to connect to Solace through the JMS connector.Initially, I received errors when attempting to connect to Solace. I then used the following parameters:- JMS Initial Context Factory: `com.solacesystems.jndi.SolJNDIInitialContextFactory`- JNDI Connection Factory: `/jms/cf/default`- JMS Provider URL: `tcp://pub:admin@172.17.0.4:55555`- JMS Destination Name: `cdc/demo/sqlserver`- JMS Destination Type: `Topic`- Solace JMS VPN: `test`The pipeline builds successfully, but when I go to the Solace "Try Me" tab and attempt to subscribe to this topic using the client "pub" I do not receive any messages.Additionally, I noticed something peculiar: when I use a client name that does not exist in Solace, the connection still builds successfully.

220
M
Fan
5 months ago
J
JnaneshwariNew Member
posted in  Show us your Pipelines
OBRUNNER_63 - Pipeline status: 'START_ERROR', Pool exhausted'

Hi Team, I am facing bellow error when i execute my pipeline . as a work around we increased ClassLoaders = 100 Still facing same issues  OBRUNNER_63 - Pipeline status: 'START_ERROR', message: 'java.lang.RuntimeException: Could not get a private ClassLoader for 'streamsets-datacollector-azure-lib', for 'com_streamsets_pipeline_stage_metadata_gen2_DataLakeGen2MetadataDExecutor', active private ClassLoaders='50': java.util.NoSuchElementException: Pool exhausted'

70
J
6 months ago
moviepages
Fan
moviepagesFan
asked in  Show us your Pipelines
how to create the pipeline for the xml data where it is taken from sftp

how to create the pipeline for the xml data where it is taken from sftp in the streamsets we have getting only 1 record from sftp in the preview data 

362
roma
Fan
7 months ago
D
Discovered Fame
DolphinDiscovered Fame
asked in  Show us your Pipelines
Cannot Connect StreamSets with Python SDK, but Can connect via CURL command with same Credential ID and Token

Hi Team,I generate Credential ID and Token via StreamSets UI → Manage → API Credential, and with the generated ID and Token I can run Curl command, returned status is “HTTP/1.1 200 OK”, also return me a json format that show my organization id, email, extra.. However, When I want to connect with below code snippet (using same credential ID and Token), it returns me error of 403. paste code and error below:>>> from streamsets.sdk import ControlHub>>> sch = ControlHub(credential_id='absd_myid', token="abcd_mytoken.") Traceback (most recent call last):  File "<stdin>", line 1, in <module>  File "/opt/module/sfsdcvenv/lib/python3.8/site-packages/streamsets/sdk/sch.py", line 141, in __init__    self.api_client = sch_api.ApiClient(  File "/opt/module/sfsdcvenv/lib/python3.8/site-packages/streamsets/sdk/sch_api.py", line 96, in __init__    raise ValueError('Encountered error while decoding auth token: {}'.format(e))ValueError: Encountered error while decoding aut

481
Bikram
Headliner
7 months ago
D
Discovered Fame
DolphinDiscovered Fame
asked in  Show us your Pipelines
How to Config to make Pipeline "Stop Event" not run when pipeline failed

Hi team,we have a pipeline, which configured a stop event, it let a sql statement to run once pipeline finish processing data,  however we found, when the pipeline failed due to some reason, this “Stop Event” still run, which is not expected for us. Can you please let me know if some place can be configured, which let the Pipeline "Stop Event" not run when pipeline failed processing data? I actually found if the pipelien “Start Event” failed, the pipeline will not run which is expected, however the “Stop Event” always run even pipeline failed. 

542
D
Discovered Fame
7 months ago
vishalsdc2024
vishalsdc2024New Member
asked in  Show us your Pipelines
Stream set pipeline is not starting

Hi , i am facing an issue while starting my sdc service and sdc is not coming up. it is prod and need to be up. can someone please help on this at the earliest. Caused by: com.streamsets.datacollector.store.PipelineStoreException: CONTAINER_0206 - Cannot load details for pipeline 'SupplyCha__eed7a3d5-486c-499b-baa1-6eac92aa198b__averydennison.com': java.io.IOException: File '/apps/sdc/data/pipelines/SupplyCha__eed7a3d5-486c-499b-baa1-6eac92aa198b__averydennison.com/pipeline.json-tmp' exists, '/apps/sdc/data/pipelines/SupplyCha__eed7a3d5-486c-499b-baa1-6eac92aa198b__averydennison.com/pipeline.json-old' should exists we have referred link for solution but not sure what can be done as files are not there on those pipelines.  Thanks & Regards,Vishal Verma     

280
vishalsdc2024
8 months ago
C
ColinNew Member
asked in  Show us your Pipelines
Inquiry Regarding StreamSets Product Versions and Feature Differences

As per my understanding, the Community Edition is available for free. Could you please provide insights into the advantages of the Enterprise Edition over the Community Edition? Additionally, what are the limitations of the Community Edition?As per my understanding, the Community Edition is available for free. Could you please provide insights into the advantages of the Enterprise Edition over the Community Edition? Additionally, what are the limitations of the Community Edition?

160
C
8 months ago

Badge winners

Show all badges
Terms & Conditions