Question

​Use Oracle CDC Client the record value warped UNISTR function

  • 8 May 2023
  • 6 replies
  • 64 views


I used StreamSets Data Collector 5.4.0, Oracle character set is AL32UTF8, In preview mode, I see that the record value is wrapped in a layer of UNISTR functions, convert to unicode

like this :


Oracle CDC Client config :

is it something wrong with my JDBC config or how to set the character set ? 

thanks


6 replies

Userlevel 5
Badge +1

@xagavin 

 

can you please try this and let me know if it works for you .

 

jdbc:oracle:thin:@192.168.100.130:1521/ironman/TMODE=ANSI,CHARSET=UTF8

@Bikram 

Thank you for your reply , i add this “jdbc:oracle:thin:@//192.168.100.130:1521/ironman/TMODE=ANSI,CHARSET=UTF8” and get this error at start pipline  


An exception occurred while running the pipeline, com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0800 - Can't start pipeline due 1 validation error(s). First one: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.error.OracleCDCRecoverableException: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: IO Error: NL Exception was generated	ProductionPipelineRunnable	*admin		ProductionPipelineRunnable-oraclebina07561dd-ec76-45db-82c3-a06caa8a8ccd-oracle-binlog- 130 (1) - < 4ca7317e-388b-4797-95d2-4243b19d02dd>
com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0800 - Can't start pipeline due 1 validation error(s). First one: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.error.OracleCDCRecoverableException: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: IO Error: NL Exception was generated
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:130) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:63) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startInternal(StandaloneRunner.java:757) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:750) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:150) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_352]
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:88) ~[streamsets-datacollector-container-5.4.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_352]

 

Userlevel 4
Badge

@xagavin 

 

it looks like your data contains some chinese characters : gavin-罗江1408.

 

Give the following a try and see if any of the work:

jdbc:oracle:thin:@//192.168.100.130:1521/ironman?useUnicode=true&characterEncoding=utf8

jdbc:oracle:thin:@//192.168.100.130:1521/ironman?useUnicode=true&characterEncoding=AL32UTF8

 

You might have to change the ? to /

thank you for your reply, 

i get this error , does the driver not support parameters?

Can't start pipeline due 1 validation error(s). First one: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.error.OracleCDCRecoverableException: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: IO Error: NL Exception was generated
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:130) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:63) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startInternal(StandaloneRunner.java:757) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:750) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:150) ~[streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~[streamsets-datacollector-stagesupport-5.4.0.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[?:1.8.0_352]
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:88) ~[streamsets-datacollector-container-5.4.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_352]

 

Userlevel 5
Badge +1

@xagavin 

I believe so  and also did some analysis from my end but its saying that UTF-8 should solve the issue.

For work around , can you please try out to decode the output string using Jython processor and check if the string value is as expected.

I will give a try from my end as well and let you know if any luck .

 

Thanks & Regards

Bikram_

 

 

 

 

@Bikram i used regex to decode in my services 

Reply