Question

Delete S3 object

  • 2 August 2022
  • 4 replies
  • 54 views

Hi experts,

 

is it possible to delete objects from an s3 bucket using a DataCollector Pipeline? Didn't find anything…

 

Thanks,

 

Marco


4 replies

Userlevel 3
Badge

Hi @mstoffel Not sure of your full use-case but with Amazon S3 origin you can configure post processing option to delete or archive object

 

On the Post Processing tab, configure the following properties:

Post Processing Property Description
Post Processing Option The action taken after successfully processing an object:
  • None - Keeps the object in place.
  • Archive - Copies or moves the object to another location.
  • Delete - Deletes the object.

 

ref here

Userlevel 3
Badge +1

@mstoffel 

please try below options in AWS s3 origin advanced option.

 

 

Thanks that works, but how to delete files that do not match the data format?

Userlevel 2
Badge

Hi @mstoffel Could you please let us know your exact use-case?

Reply