Solved

Launching SDC with docker-compose fails

  • 9 February 2022
  • 6 replies
  • 229 views

I’ve been spinning up a few docker containers within a docker-compose.yml and recently this has begun to fail with the following error. (my transformer instance has the same issue)

Creating collector ... done

Attaching to collector

collector      | Engine running in Deployment Mode

collector      | Java 1.8 detected; adding $SDC_JAVA8_OPTS of "-XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Djdk.nio.maxCachedBufferSize=262144" to $SDC_JAVA_OPTS

collector      | bash: line 1: {ISSUES:[code:SSO_01]}: command not found

collector      | Engine Pre Start failed with exit code 127

collector exited with code 127

 

Just executing the docker run command still works fine but sorting out the networking will be a pain so getting this functionality back would be helpful.

 

This used to work, but recently has stopped. I’m bringing up a local database and hopefully an instance of a Vault credential store and having the containers linked in the docker-compose file is convenient for development and testing.

The service in the docker-compose.yml file looks like this.

  collector:

    container_name: collector

    image: streamsets/datacollector:4.3.0

    environment:

      - STREAMSETS_DEPLOYMENT_SCH_URL=https://na01.hub.streamsets.com

      - STREAMSETS_DEPLOYMENT_ID=<my deployment id>

      - STREAMSETS_DEPLOYMENT_TOKEN=<my token>

    ports:

      - "18630:18630"

    links:

      - "postgres:postgres"

 

As I mentioned, this had been working with 4.1.0 but has since stopped. When debugging it I could trace it to a curl command that is reaching back to streamsets.com and I suspect there’s something wrong with the command itself when run as part of docker-compose (white space missing perhaps?)

I’m running this with docker 20.10.12 on an ubuntu 20.04 system.

Thanks,

Travis

icon

Best answer by wilsonshamim 23 February 2022, 03:43

View original

6 replies

Userlevel 2
Badge

@Travis ,

would you please share the `docker run` command that works for you for SDC 4.3.0?

@Travis ,

would you please share the `docker run` command that works for you for SDC 4.3.0?

The docker run command was just the one provided by control hub

docker run -d -e STREAMSETS_DEPLOYMENT_SCH_URL=https://na01.hub.streamsets.com -e STREAMSETS_DEPLOYMENT_ID=<my deployment id> -e STREAMSETS_DEPLOYMENT_TOKEN=<my token> streamsets/datacollector:4.3.0

 

Userlevel 2
Badge +1

Hi @Travis :

As discussed during our cadence call, it did work to me.

Installed latest version of docker-compose https://docs.docker.com/compose/install/

docker-compose --version

docker-compose version 1.29.2, build 5becea4c

created the  yml file with the same content that was shared by you. 

I did test with 1.23.0  and to worked with it as well.

sudo curl -L "https://github.com/docker/compose/releases/download/1.23.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

 

 

Thank you @wilsonshamim,

I went ahead and upgraded my docker-compose to that version (1.29.2) and had a go.

I had the same issue, so there’s something else going on for me.

We’re spinning up an EC2 instance today, I’ll see if it works there, I suspect something local to my environment at this point.

 

I’m using docker 20.10.12 and docker-compose 1.29.2 on an Ubuntu 20.04 system in case the underlying docker version has something to do with it.

 

Userlevel 2
Badge +1

The issue was due to an incorrect token.  Its resolved after correcting the token

Userlevel 5
Badge

Thank you @wilsonshamim for adding the resolution. I will mark it as the best answer. 

Reply