Skip to main content

Hi, when using the PostgreSQL Metadata processor I get the exception “JDBC_91 - Unsupported type for column 'fdwoptions': _text (jdbc type id 2003) ” when the processor tries to create a database table for a record. The record has a handful of string, date and decimal fields, no null values and none with  the name ‘fdwoptions’ so I have no idea what the processor is actually doing and why it would try to create a column with this name. Also the type ‘_text’ is confusing, when looking at the documentation, String is mapped to varchar, so not sure about the ‘_text’ type in this context.

Any ideas?

Thank you.

Added the stack trace for a little more info:

com.streamsets.pipeline.api.base.OnRecordErrorException: JDBC_91 - Unsupported type for column 'fdwoptions': _text (jdbc type id 2003)	at com.streamsets.pipeline.stage.processor.jdbcmetadata.JdbcMetadataProcessor.process(JdbcMetadataProcessor.java:215)	at com.streamsets.pipeline.api.base.RecordProcessor.process(RecordProcessor.java:52)	at com.streamsets.pipeline.api.base.configurablestage.DProcessor.process(DProcessor.java:35)	at com.streamsets.datacollector.runner.StageRuntime.lambda$execute$2(StageRuntime.java:287)	at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:232)	at com.streamsets.datacollector.runner.StageRuntime.execute(StageRuntime.java:299)	at com.streamsets.datacollector.runner.StagePipe.process(StagePipe.java:209)	at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processPipe(ProductionPipelineRunner.java:859)	at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.lambda$executeRunner$3(ProductionPipelineRunner.java:903)	at com.streamsets.datacollector.runner.PipeRunner.acceptConsumer(PipeRunner.java:195)	at com.streamsets.datacollector.runner.PipeRunner.forEachInternal(PipeRunner.java:140)	at com.streamsets.datacollector.runner.PipeRunner.executeBatch(PipeRunner.java:120)	at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.executeRunner(ProductionPipelineRunner.java:902)	at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.runSourceLessBatch(ProductionPipelineRunner.java:880)	at com.streamsets.datacollector.execution.runner.common.ProductionPipelineRunner.processBatch(ProductionPipelineRunner.java:503)	at com.streamsets.datacollector.runner.StageRuntime$3.run(StageRuntime.java:371)	at java.security.AccessController.doPrivileged(Native Method)	at com.streamsets.datacollector.runner.StageRuntime.processBatch(StageRuntime.java:367)	at com.streamsets.datacollector.runner.StageContext.processBatch(StageContext.java:294)	at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.handlePostBatchAsNeeded(JdbcBaseRunnable.java:412)	at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.generateBatchAndCommitOffset(JdbcBaseRunnable.java:306)	at com.streamsets.pipeline.lib.jdbc.multithread.JdbcBaseRunnable.run(JdbcBaseRunnable.java:167)	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)	at java.util.concurrent.FutureTask.run(FutureTask.java:266)	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)	at java.util.concurrent.FutureTask.run(FutureTask.java:266)	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)	at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214)	at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44)	at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25)	at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210)	at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeRunnable.run(SafeScheduledExecutorService.java:176)	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)	at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.lambda$call$0(SafeScheduledExecutorService.java:214)	at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:44)	at com.streamsets.datacollector.security.GroupsInScope.execute(GroupsInScope.java:25)	at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeCallable.call(SafeScheduledExecutorService.java:210)	at com.streamsets.pipeline.lib.executor.SafeScheduledExecutorService$SafeRunnable.run(SafeScheduledExecutorService.java:176)	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)	at java.util.concurrent.FutureTask.run(FutureTask.java:266)	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)	at java.lang.Thread.run(Thread.java:750)

 

So, I can answer my own question. I forgot to enter the schema name. Seems this causes the processor to take some weird detours. The field which appears in the exception message (fdwoptions) is actually a column in the postgres catalog table pg_catalog.pg_foreign_data_wrapper. No idea what causes the processor to access this table...


Reply