Repartition Processor Configuration

  • 27 March 2023
  • 6 replies

I am facing issue while configuring repartition method attribute for repartition processor using python. What is the right way to set it.


Best answer by chandrakant_v 27 March 2023, 14:04

View original

6 replies

Userlevel 4

Hello @chandrakant_v  - Thank you for reaching out to the StreamSets Community. Could you please elaborate the issue and share the error/configs with us? 


I’m trying to set repartition method value this way:

repartition.repartition_method = "REPARTITION BY NUMBER (INCREASE OR DECREASE)"

But i’m getting the following error: 

Repartition Method -- CREATION_010 - Configuration value 'REPARTITION BY NUMBER (INCREASE OR DECREASE)' is not a valid 'RepartitionTypeValues' enum value: java.lang.IllegalArgumentException: No enum constant com.streamsets.pipeline.spark.transform.repartition.RepartitionTypeValues.REPARTITION BY NUMBER (INCREASE OR DECREASE)


Userlevel 4

@chandrakant_v please provide the solution so that it helps others. Also, make that solution as the answer to close the post :-)

to set the repartition method, one need to use the following snippet.

Use the modelType attribute of repartition processor object.

repartition.modelType = "Repartition By Number (Increase or Decrease)"


Userlevel 4

Thank you for sharing the resolution @chandrakant_v .

Feel free to reach out to us if you have any further issues. :)