Solved

PostgreSQL Metadata processor does not accept preconfigured connection?

  • 9 August 2022
  • 4 replies
  • 36 views
P
Rank

Hi, I am new to StreamSets so I may miss a thing here, if this is the case please apologize!

Wanted to configure the PostgreSQL metadata processor and found that the only way to connect is to directly configure the JDBC connection string and credentials. There seems no way to use a connection previously configured, like when setting up a JDBC (multitable) consumer or JDBC Producer.

Is this a defect/missing feature or do I misunderstand something?

 

Thank you in advance!

icon

Best answer by Pradeep 10 August 2022, 09:49

View original

4 replies

Pradeep
Rank
Userlevel 3
Badge +1

@pedromanuel If you are designing the pipelines through StreamSet’s control-hub or dataops platform, you can utilize Connections feature for the same. Refer https://docs.streamsets.com/portal/platform-controlhub/controlhub/UserGuide/Connections/GettingStarted.html#concept_fxv_rp5_4mb for more details.

P
Rank

Hi @Pradeep , thank you for your reply! This was exactly what I was trying to do and it is working fine for origins (like the JDBC Multitable Consumer) or destinations (like the JDBC Producer). But for the JDBC metadata processor, which is supposed to use the exact same connection as the JDBC producer, there is no option to select a Connection, i can only fill out the connection details like JDBC url, credentials etc. For clarity, I append two screen shots, one for the producer (a destination) where I can select a connection and one for the processor, where I only can specify the JDBC details but not a connection...

 

 

Does this makes sense?

Pradeep
Rank
Userlevel 3
Badge +1

Hi @pedromanuel, you are correct. Connection can not be used in Postgresql Metadata processor currently. Will be raising a feature request for the same. Lookout for release notes for future releases. I guess you are already updating connection details manually and please continue to use this as a temporary workaround.

P
Rank

Thank you @Pradeep !

Reply


Terms & Conditions