Elastigroup – The next generation of auto scaling for big data clusters
Spotinst Elastigroup integrates with AWS EMR Clusters. It automates the management of Spot, On Demand & Reserved Instances. You simply tell Elastigroup how much capacity you need and it does the rest.
Elastigroup understands that Hadoop is different than a Web Server
Big Data workloads are a complete contrast to standard Applications. Elastigroup monitors the pending jobs and tasks in the EMR cluster, and launches the desired capacity to meet the required demand. It also using an aggressive Downscaling to remove idle resources in the cluster.
Elastigroup gets smarter over time, by understanding how long jobs and tasks are running.
Remove idle resources faster than ever
Elastigroup shuts down nodes or the entire cluster without the risk of data loss when there are no more active jobs.
Spot instance lifecycle management
Elastigroup uses its proven prediction algorithm to match Spot Instances that can run for the desired amount of time for the required Task or job.
Mixed instance types
By configuring heterogeneous clusters and mixing nodes of multiple instance types, Elastigroup delivers maximum efficiency to run the relevant job on the right machine type to achieve greater data processing performance.
Optimizing for both pricing model and Instance Type&Size
Elastigroup automates the Instance lifecycle and provides a combination of EC2 Spot, Existing RI reservations and On-Demand based on EC2 Spot capacity, availability & pricing trends
Mixed instance types
By configuring heterogeneous clusters and mixing nodes of multiple instance types, Elastigroup delivers maximum efficiency to run the relevant job on the right machine type to achieve greater data processing performance.
The World’s Leader in management of Spot Instances
Fully utilize EC2 Spot instances for any EMR cluster
Elastigroup balances the required performance, cost, and SLA requirements when launching, scaling and terminating EC2 Spot Instances.
Complete confidence while running Spot instances for production and time-concern jobs
Elastigroup drains nodes, approximately 15 minutes before a Spot termination notification arrives from the cloud provider, in order to gracefully terminate existing Tasks and prevent from scheduling new jobs on these nodes. In parallel, Elastigroup monitors the HDFS/S3 read&write activity to make sure that no data is being lost during the replacement process.