EMR for beginners — Resolving common issues
I started working for a Tech company right out of college. My first ever project assigned to me had all things to do with cloud. It was a bit daunting at first, not going to lie, but stackoverflow solutions got me through. So here I am sharing a few solutions that I found worked perfectly well for me.
1 .EMR terminates without starting
This was one of the very first issues I faced and it’s relatively an easy fix. If you are using spot pricing in your script, make sure that the bid price mentioned in your script is well over the asking price for the instance. You can check the asking price of instances here .
2. Host Key Verification Fail
While trying to log into your EMR via EC2 instance (SSH) you might encounter this quite terrifying error
Simply run ssh-keygen –R emr-ip on your EC2 instance to remove previous key and try logging in again OR sudo ssh-keygen -f “/root/.ssh/known_hosts” -R “<emr-ip>” if the previous command does not work.
3. Vertex Failed- No Route to Host Exception
If you are running HIVE rules on your EMR (spun on a daily basis), you might have come across this error and have yourself scratching your heads.
One solution is to make sure that the table you are referring to is created and exists in the current running EMR (table might be referred in the condition if or where, or you might be selecting data from the table).
These were some of the common issues I came across while dealing with running HQL scripts on EMRs. Hope it helped !
