Skip to main content

Hi Team,
We are using groovy_scripting origin to read files from s3 bucket. But problem is that we are not able to apply offset in sql query used in code. Can u help us to provide some way to apply offset in sql query. I am attaching code snippet that is working but we are not able to apply offset in it.

 

@himanshu1234567 

I got this below code snippet from google with sql query on s3 object and it works fine for it.

May i know if the code is working fine in your local setup .

#  start client with s3
s3 = boto3.client('s3', region_name='us-east-1')

#  define file location and name
bucket_name = 'MY-BUCKET-NAME'
filename = 'test.csv'

#  create SQL expression to query by date using column index
sql_exp = "SELECT * FROM s3object s LIMIT 1"

#  should we use header names to filter
use_header = False

#  return CSV of unpacked data
file_str = query_csv_s3(s3, bucket_name, filename, sql_exp, use_header)

#  read CSV to dataframe
df = pd.read_csv(StringIO(file_str))

#  print column names
print(list(df.columns))

 

@Bikram where u are using offset here, my problem is that sql query is not supporting offset.Can we use offset in your code?

@Bikram  we need groovy/java code?

@himanshu1234567 

groovy only.

@Bikram above code u pasted is groovy?

Reply


Terms & Conditions