I used StreamSets Data Collector 5.4.0, Oracle character set is AL32UTF8, In preview mode, I see that the record value is wrapped in a layer of UNISTR functions, convert to unicode
like this :
Oracle CDC Client config :
is it something wrong with my JDBC config or how to set the character set ?
thanks
can you please try this and let me know if it works for you .
jdbc:oracle:thin:@192.168.100.130:1521/ironman/TMODE=ANSI,CHARSET=UTF8
Thank you for your reply , i add this “jdbc:oracle:thin:@//192.168.100.130:1521/ironman/TMODE=ANSI,CHARSET=UTF8” and get this error at start pipline
An exception occurred while running the pipeline, com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0800 - Can't start pipeline due 1 validation error(s). First one: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.error.OracleCDCRecoverableException: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: IO Error: NL Exception was generated ProductionPipelineRunnable *admin ProductionPipelineRunnable-oraclebina07561dd-ec76-45db-82c3-a06caa8a8ccd-oracle-binlog- 130 (1) - < 4ca7317e-388b-4797-95d2-4243b19d02dd>
com.streamsets.datacollector.runner.PipelineRuntimeException: CONTAINER_0800 - Can't start pipeline due 1 validation error(s). First one: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.error.OracleCDCRecoverableException: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: IO Error: NL Exception was generated
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:130) ~:streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:63) ~astreamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startInternal(StandaloneRunner.java:757) ~:streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:750) ~:streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:150) ~:streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~:streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~astreamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~astreamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~:streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~:streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~astreamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~astreamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~:streamsets-datacollector-stagesupport-5.4.0.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~:?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~:?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~:?:1.8.0_352]
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:88) ~astreamsets-datacollector-container-5.4.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~1?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~:?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) ~:?:1.8.0_352]
it looks like your data contains some chinese characters : gavin-罗江1408.
Give the following a try and see if any of the work:
jdbc:oracle:thin:@//192.168.100.130:1521/ironman?useUnicode=true&characterEncoding=utf8
jdbc:oracle:thin:@//192.168.100.130:1521/ironman?useUnicode=true&characterEncoding=AL32UTF8
You might have to change the ? to /
thank you for your reply,
i get this error , does the driver not support parameters?
Can't start pipeline due 1 validation error(s). First one: JDBC_00 - Cannot connect to specified database: com.streamsets.pipeline.stage.origin.jdbc.cdc.oracle.error.OracleCDCRecoverableException: com.streamsets.pipeline.api.StageException: JDBC_06 - Failed to initialize connection pool: com.zaxxer.hikari.pool.HikariPool$PoolInitializationException: Failed to initialize pool: IO Error: NL Exception was generated
at com.streamsets.datacollector.execution.runner.common.ProductionPipeline.run(ProductionPipeline.java:130) ~~streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunnable.run(ProductionPipelineRunnable.java:63) ~~streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.startInternal(StandaloneRunner.java:757) ~~streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.standalone.StandaloneRunner.start(StandaloneRunner.java:750) ~~streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.datacollector.execution.runner.common.AsyncRunner.lambda$start$3(AsyncRunner.java:150) ~~streamsets-datacollector-container-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210) ~~streamsets-datacollector-stagesupport-5.4.0.jar:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~~?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~~?:1.8.0_352]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~~?:1.8.0_352]
at com.streamsets.datacollector.metrics.MetricSafeScheduledExecutorService$MetricsTask.run(MetricSafeScheduledExecutorService.java:88) ~~streamsets-datacollector-container-5.4.0.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~~?:1.8.0_352]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~~?:1.8.0_352]
at java.lang.Thread.run(Thread.java:750) ~~?:1.8.0_352]
I believe so and also did some analysis from my end but its saying that UTF-8 should solve the issue.
For work around , can you please try out to decode the output string using Jython processor and check if the string value is as expected.
I will give a try from my end as well and let you know if any luck .
Thanks & Regards
Bikram_
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.