Solved

E2E integration testing

  • 15 October 2021
  • 1 reply
  • 37 views

We want to create an integration test suite and run it in a self-contained world of headless containers

The high level goal is 

docker-compose to start all containers 

  • streamsets
  • kafka

configure streamsets 

send data to kafka topic 

StreamSets processes the data in the topic and send its to other topics 

Application under tests, processes the data (fails / passes) - the test

tear down  

Question:

How do I configure streamset pipeline without using the UI? 

icon

Best answer by Kirti 15 October 2021, 21:42

View original

1 reply

Userlevel 2
Badge

Hi @haroon.said ,

There is SDK for Python available to configure StreamSets pipeline as well as many other UI operations to do programmatically.

 

Documentation for SDK 

SDK Tutorials

In fact, we use this SDK heavily along with STF (StreamSets Test Framework) for internal E2E integration testing. And, if interested, SDC tests are available here at datacollector-tests repo to take a look.

 

Regards-

Kirti

Reply