Hello @lakshmi_narayanan_t - Thank you for reaching out to the StreamSets Community. I ran a test pipeline and was able to validate it without any issue with mongodb_opblog origin. 

This is how I created the Capped collection:

You can also use the following command to create the capped collection with size.

db.createCollection("oplog.test",{capped:true,size:100000})

>1]. https://www.mongodb.com/docs/manual/core/capped-collections/#:~:text=Procedures-,Create%20a%20Capped%20Collection,pre%2Dallocate%20for%20the%20collection.

Could you please try creating the capped collection with above method and let us know how it goes?

Thank you - AkshayJ

This code is working as expected, but we need to refactor it for better performance and readability . We should consider implementing caching mechanisms to reduce database calls. Also, we need to optimize the data structures and algorithms used in this code to improve its efficiency. Finally, we should add more comments and documentation to make the code more readable and understandable for future developers who might work on it.

view software project

@lakshmi_narayanan_t  - Are you still facing an issue?  

@Janeer - Would it be possible for you to reach out to the Account Manager who handles you account from StreamSets side with the use-case details? so that we can discuss this further and help you on this. 

Hi @Janeer and @lakshmi_narayanan_t - I just wanted to reach out to you to check if you have any questions on this.

If no further questions, I would request you to mark the thread as best answer that will close the thread.

Thank you - AkshayJ

HI AkshayJ

Is possible to perform cdc in mongodb atlas using mongodb oplog?

 The mongodb oplog connection string will be satisfied for MongoDB atlas but i got time out error

please give some guidance to clear that error.