Skip to main content

How to increase JVM heap memory on SDC running in dataops platform?


Jordi S
StreamSets Employee

To increase the java heap size of a datacollector engine created through a DataOps deployment we have to change the configuration of said deployment, which in turn will make the setting chance to be applied to all the datacollectors under that deployment. 

1. First, go to the deployments screen, click on the deployment your SDC is using, and then click on the edit button.

 

NSSEejf9o8Q5FI7AvGPSI78vuDFr8bJEOwHKr4KGymaC14uNhaduKxqFhFF4cT8mkeO0v-su-EEeGDWiucDh7l9Uz7VjAv-r81Sw_WP_Z84OCZcn7fqKcFpGgVOqiDT3hdVXhOD9OUoOOSyVEsWcroyCpoXGAUJ8fNqPrIsWTV2dEgvGa5eeRlgN5uYk_Q

 

2. Skip the define deployment stage by clicking on the save & next button, then on the configure deployment click on the advanced configuration 

 

Sk-PFTrnmG3vdoW4JVfbOCFOrNF5CS2amYMwfpBJ4TkMyjo3o-yrQr56wgHmWK0kdlox_s3cN0WsPYjX1EWM7z4jUFSYd1pIY5efJSfq2Rc5E0wZnjMVweUo9Sk4uuF4GaM-_ROfa_EI_A9V2BjDN-jtik2vfGpbYPLEfOF96Y39RhRVe6-yaJJBwXtK1g


 

3. Navigate to Java Configuration tab on the far right. Now you have two options, percentage and absolute. If you use the absolute option you can specify the size you want, in megabytes. In this case, I used 8gb of space. To avoid constant recalculation of the allocated heap size, set both the minimum and maximum properties to the same value.

 

_wGqRODNI12lN4e9rQ6cS3dxXBh19fuUm0Dm4omCdM2xKZkIASRSzrmrferTOGi5EZiWlg1Dj4kFon45XoMc8KoACtYsyp0IRyHiM3gG4CQq55XuTGFc24UhI5AoEypzYZIAKNpyNpTNcMs_NR0TbNsaW2BBqaYizifiL7FtitOadnfN2Zkh10b5DHtoWg

 

4. For the changes to take effect we have to restart the engine that uses that deployment. Go to the engine tab, click on the engine, and then on the Restart Engine button. This process can take some time depending on the amount of work that engine is running.  

 

0PMdowlW3Pf1mRyFALSD_wqgvTd2RKeEwJ3aEEpZyh60ey75J3yLat5xnQSL6TjvsnWgNJyk78ZIoBxIGgjWpgpuon0ZAsdkBimvvPmoCVVwVEtiLPpXt3XcOw4s9D-LOgdGUt07fwbHGR0hxByHF8owN07bnGxRPHaMjYnkalhpn_hfesYPv5DMJqSvSw

 

0 replies

Be the first to reply!