Question

AWS EMR Configuration

I was able to configure EMR Cluster from my transformer pipeline and start the job. But the job does not finish.

It fails with an error

ERROR Client: Application diagnostics message: Shutdown hook called before final status was reported.

Any ideas ?

Thanks !

Regards,

Srinivasa Nanduru

Couple of things I tried for it to work.

Downgraded AWS EMR version 7.X to 6.5 (Not entirely sure if this is the root cause, tried nonetheless).

I opened in the port 19630 for TCP inbound for my AWS EC2 engine, where I have my Transformer Docker Engine installed

I provided public IP address for call back URL in the pipeline configuration, http://ip_address:19630

This has solved the issue.

Sign up