Skip to main content
Question

Is it feasible to set up the transformer pipeline so that it runs on an AWS EMR cluster?

  • May 5, 2023
  • 2 replies
  • 32 views

Chromeheartshoodie42
Fan

I built a primary transformer pipeline that will operate on an EMR cluster. I utilized the essential setup details in Cluster configuration.
I'm having the following problem:
Pipeline
for a Transformer Engine
Transformer
 

2 replies

Bikram
Headliner
Forum|alt.badge.img+1
  • Headliner
  • 486 replies
  • May 5, 2023

@Chromeheartshoodie42 

Yes , the transformer pipeline can be configured on EMR cluster and if you can provide me the error details i can check on it.


saleempothiwala
Headliner
Forum|alt.badge.img

@Chromeheartshoodie42 

 

make sure your transformer url is reachable from AWS EMR. 

If you are using a docker for transformer engine then you will have to make configuration such that the docker is accessible from the internet.