Skip to main content

Hi experts,

 

is it possible to delete objects from an s3 bucket using a DataCollector Pipeline? Didn't find anything…

 

Thanks,

 

Marco

Hi @mstoffel Not sure of your full use-case but with Amazon S3 origin you can configure post processing option to delete or archive object

 

On the Post Processing tab, configure the following properties:

Post Processing Property Description
Post Processing Option The action taken after successfully processing an object:
  • None - Keeps the object in place.
  • Archive - Copies or moves the object to another location.
  • Delete - Deletes the object.

 

ref here

@mstoffel 

please try below options in AWS s3 origin advanced option.

 

 

Thanks that works, but how to delete files that do not match the data format?

Hi @mstoffel Could you please let us know your exact use-case?

Reply


Terms & Conditions