Microsoft Azure
Cloud Financial Management (FinOps)
Autoscaling applications
Google Cloud
Kubernetes infrastructure optimization
Reserved commitment management
Amazon Web Services
Cloud services for MSPs
Spot Overview
About NetApp
Elastigroup
Ocean
Ocean CD
Spot Connect
Eco
Cost Intelligence
Billing Engine
CloudCheckr
Ocean for Apache Spark
Spot Security
Case Studies
Resource Center
Documentation
Service Status
What Is CloudOps?
How to Operationalize FinOps
Contact Us
Careers
Amazon’s Elastic Map Reduce or EMR, makes it easy to set up, operate and scale big data environments. This enables data scientists and developers to rapidly analyze massive amounts of structured and unstructured data. Combined with Spot by NetApp’s Elastigroup, data scientists can reliably run EMR core and task nodes on highly affordable EC2 spot instances.
Eliminating a single point of failure
EMR’s popularity and broad adoption is due to its powerful parallel processing–where datasets are distributed and processed across multiple compute nodes. All this data processing distribution as well as cluster health checks, is performed by EMR master nodes.
However, if the master node fails, all data processing stops, and the cluster is automatically terminated.
To avoid this, Elastigroup now supports defining multiple master nodes in a single EMR cluster. This ensures that in case of a primary master node failure, it will automatically fail over to a standby master node without impacting data processing.
Note: This is currently available for EMR version 5.23.0 and higher.
Getting started with EMR multiple master nodes
Using Spot by NetApp’s API the following values need to be configured:
- Master group target value should be set to 3 (this is value currently supported by AWS)
- Master group Lifecycle must be ON_DEMAND
- We recommend changing the
terminationprotectionvalue to be false, otherwise you won’t be able to terminate the cluster using Elastigroup.
