Skip to main content

I’m working with a legacy version of datacollector here v.3.22 and an elastic cluster (7.10.2). I’m getting the following error message:

Record1-Error ELASTICSEARCH_17 - Could not index '10' records: POST https://<ingest_route>:<port>/_bulk: HTTP/1.1 400 Bad Request {"error":{"root_cause":a{"type":"illegal_argument_exception","reason":"Action/metadata line 1] contains an unknown parameter m_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line 1] contains an unknown parameter m_type]"},"status":400}

 

I have tried removing the @metadata/type field with a field remover and tried renaming it but I get the same error message. 

I also noticed the stage lib I’m using is for elasticsearch 7.9

Is this version of streamsets I’m working with just too outdated and can no longer support more recent version of elasticsearch? Is there any workaround I could try to solve this?

 

 

Thank you @vmafonso  for reaching out to the StreamSets Community. would you please share the configuration of the elasticsearch stage.

Hi @AkshayJadhav thank you for the quick reply. 

Here’s the current config:

 

General:

Name: Elasticsearch 1

Description:

Stage Library: Elasticsearch 7.9.0 

Required Fields:

Preconditions: 

On Record Error: Send to Error 

 

Elasticsearch:

HTTP URLs: <ingestion_route>

HTTP Port: 9200

Use Security: YES

Index: <index_name>-${YYYY()}-${MM()}-${DD()}

Mapping: <index_name>

Document ID:

Additional HTTP Params:

Detect Additional Nodes in Cluster: NO

Time Basis: ${time:now()}

Data Time Zone: + 00:00 UTC (UTC)

Parent ID: 

Routing:

Data Charset: UTF-8

Default Operation: INDEX

Unsupported Operation Handling: Discard

Additional Properties: {}

 

Security:

Mode: Basic

User Name: <username>

Password: <password>

Enable SSL: YES

SSL TrustStore Path: <path_to_truststore>

SSL TrustStore Password: <truststore_password>

 

 

 

I have checked with the latest version and I do not see any issues with connectivity. 

You’ve checked with the latest version of elasticsearch? So there should be no incompatibility issues? Did you configure TLS client authentication for elastic? 

Reply


Terms & Conditions