We want to create an integration test suite and run it in a self-contained world of headless containers
The high level goal is
docker-compose to start all containers
configure streamsets
send data to kafka topic
StreamSets processes the data in the topic and send its to other topics
Application under tests, processes the data (fails / passes) - the test
tear down
Question:
How do I configure streamset pipeline without using the UI?
Best answer by Kirti
Hi
There is SDK for Python available to configure StreamSets pipeline as well as many other UI operations to do programmatically.
In fact, we use this SDK heavily along with STF (StreamSets Test Framework) for internal E2E integration testing. And, if interested, SDC tests are available here at datacollector-tests repo to take a look.
Regards-
Kirti
Hi
There is SDK for Python available to configure StreamSets pipeline as well as many other UI operations to do programmatically.
In fact, we use this SDK heavily along with STF (StreamSets Test Framework) for internal E2E integration testing. And, if interested, SDC tests are available here at datacollector-tests repo to take a look.
Regards-
Kirti
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
