How to deploy SDC in k8s cluster without provisioning agent?

The Ideal approach via provisioning agent:

A Provisioning Agent is a containerized application that runs in a Kubernetes container orchestration framework. The agent communicates with Control Hub to automatically provision Data Collector containers in the Kubernetes cluster in which it runs. Provisioning includes deploying, registering, starting, scaling, and stopping the Data Collector containers. You can configure the Provisioning Agent to provision Data Collector containers enabled for Kerberos authentication. You can find steps here

However, if you want to avoid provisionig agent installation due to any reason. you can follow the below steps for reference.

This is the bare minimum setup required to deploy SDC in any k8s without control-agent.

Step 1: Create a lower-cased UUID and store it in a secret
 

$SDC_ID=`uuidgen | tr "[:upper:]" "[:lower:]"`
$echo "Generated sdc.id "${SDC_ID}
$kubectl create secret generic sdc-id --from-literal=sdc.id=${SDC_ID}

Step 2: Get an SDC auth token from Control Hub and store in k8s secret
 

SCH_AUTH_TOKEN= ${TOKEN}

You can get this from UI --> Engine --> Three vertical dots --> Data Collector component --> generate Token
 

$ kubectl create secret generic sdc-auth-token --from-literal=application-token.txt=${SDC_AUTH_TOKEN}

Step 3: Deploy ConfigMap for dpm.properties
 

$kubectl apply -f dpm-configmap.yaml

Below is my YAML, Please verify `dpm.base.url`  that it point to correct platform

kind: ConfigMap
apiVersion: v1
metadata:
  name: dpm-config
data:
    dpm.properties: |-
      dpm.enabled=true
      dpm.base.url=https://eu02.hub.streamsets.com
      dpm.registration.retry.attempts=5
      dpm.security.validationTokenFrequency.secs=60
      dpm.appAuthToken=@application-token.txt@
      dpm.remote.control.job.labels=
      dpm.remote.control.ping.frequency=5000
      dpm.remote.control.events.recipient=jobrunner-app
      dpm.remote.control.process.events.recipients=jobrunner-app,timeseries-app
      dpm.remote.control.status.events.interval=60000
      http.meta.redirect.to.sso=false

Step 4: Final SDC deployment
 

 $kubectl apply -f sdc.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sdc
  labels:
    app : sdc
spec:
  replicas: 1
  selector:
    matchLabels:
      app: sdc
  template:
    metadata:
      labels:
        app: sdc
    spec:
      containers:
      - name : sdc
        image: streamsets/datacollector:4.3.0
        ports:
        - containerPort: 18630
        env:
        - name: SDC_JAVA_OPTS
          value: "-Xmx2048m -Xms2048m"
        volumeMounts:
        - name: sdc-id
          mountPath: /data/sdc.id
          subPath: sdc.id
        - name: sdc-auth-token
          mountPath: /etc/sdc/application-token.txt
          subPath: application-token.txt
        - name: dpm-config
          mountPath: /etc/sdc/dpm.properties
          subPath: dpm.properties
      volumes:
      - name: sdc-id
        secret:
          secretName: sdc-id
      - name: sdc-auth-token
        secret:
          secretName: sdc-auth-token
      - name: dpm-config
        configMap:
          name: dpm-config

Step 5: Verify SDC engine is available in UI