Solved

DataOps Platform - REST API Origin - How to invoke pipeline?


Hi All,

I created a microservice pipeline with REST API Origin using the DataOps platform as per video: https://www.youtube.com/watch?v=wIZWMV1bMl4

Now, I am unable to invoke the pipeline using 3rd party tools like Postman. I have chosen default settings in SDC and headers as (X-SDC-APPLICATION-ID:sdc_microservice & Content-Type:application/json) yet I’m facing issue as “Error: connect ETIMEDOUT xx.xx.xx.xx:8000”.

However, when I use the same URL in a curl command where SDC container is running, I see, 200 OK response. 

curl -i -X GET http://xx.xx.xx.xx:8000/rest/v1/user --header "X-SDC-APPLICATION-ID:sdc_microservice"

There is no connectivity issue. So, can anyone tell me why I am unable to use Postman tool or if I am missing anything here?

Below are the images showing the same:

 

icon

Best answer by AkshayJadhav 4 April 2023, 19:14

View original

12 replies

Userlevel 4
Badge

Thank you @Kartik Medavarapu .

Hi @AkshayJadhav  - Apology for a delayed response. We tried using the “-p 8000:8000” but no luck. 

Due to other priority tasks, I stopped checking on this. However, since we got the response by using curl command, we leveraged that instead. Thank you for all the support !

Userlevel 4
Badge

Hi @Kartik Medavarapu - Were able to perform the api call via postman to the docker host on 8000 port? 

If yes, I would request you to mark the comment as best answer that will close the thread.

 

Userlevel 4
Badge

@Kartik Medavarapu - I was having the same issue with the docker data collector however I was able to make it work by mapping the 8000 port to the docker host machine(macbook).

Just added -p 8000:8000 in the docker startup script and send the request on 8000 port via Postman application.

 

Hope this helps.

Thank you - AkshayJ

Userlevel 4
Badge

@Kartik Medavarapu - Are you able to ping to the docker ip/hostname from the windows laptop?

@AkshayJadhav - I’m running data collector as docker/container in a VM (ubuntu) and using postman from my windows laptop.  

Userlevel 4
Badge

@Kartik Medavarapu  - Are you running data collector as tar ball installation in VM and where is the postman installed? 

 

@AkshayJadhav - Thank you for the inputs. Presently, I don’t see any connectivity issue. However, I will check again and get back. 

Userlevel 4
Badge

@Kartik Medavarapu - I believe the connection is not happening , sometime back I ran the data collector in docker and got the same error message due to my host machine is not able to reach to the docker host. I tried ping command it did not work. Looks like a connection issue to me.

Userlevel 4
Badge

 @Kartik Medavarapu - I tried the same in DataOps with v5.2 tar ball installation and it worked. Here are the screenshots: 

 

From your screenshot, I can see that postman is not able to reach to the IP address. If you are using the data collector in VM then mention the VM ip address and still it does not work then run ping <VM IP address> command to check the connectivity from host to VM is working.

-AkshayJ

Hi @AkshayJadhav,

 

Thank you for quick response. I have mentioned the same headers and tried, but no luck !!

We are using SDC 5.x installed as container in a local VM and SCH is DataOps platform. Any other specific env setting to be considered? 

 

Regards,

Kartik

Userlevel 4
Badge

Hi @Kartik Medavarapu - Thank you for reaching out to the StreamSets Community.

I was able to get perform the api call from the postman application, here are the headers that I used:

 

Let us know how it goes.

 

Thank you - AkshayJ

Reply