Skip to main content

Hi everybody! 
I want to remove files  (whole file data format) from a origin stage (GCS stage) after processing it, but when I activate thge post-processing option and set it to "Delete" and try to start the associated job to the pipeline a have the Error:

GCS_11 - Error validating postprocessing options. If 'Whole File' data format is used, neither 'Delete' nor 'Archive -> Move into ...' can be used.

There are any way to do that directly from a pipeline or the best practice is to add a shell script at the end of the pipeline to remove the files from command line with google CLI commands after processing them?
 

Hola @jarquero,

The error you are obtaining is pretty clear. Unfortunately, what you are trying to do is not possible…

You should try with events & some scripts, bash perhaps.

Thanks
Edy

Reply


Terms & Conditions