Deploy AWS spot instances with Elastigroup

Up to 90% cost savings on spot instances

AWS spot instances represent AWS’s excess capacity, available for any surge in customer demand. To offset the loss of idle infrastructure, AWS offers this excess capacity at a massive discount to drive usage. With up to 90% in savings when compared to on-demand pricing, spot instance pricing is an affordable option for organizations looking to significantly reduce their cloud bills.

One of the challenges with spot instances, however, is that they can disappear at a moment’s notice. Without automation in place, it’s impossible for AWS users to predict when these instances might shut off and find a suitable replacement in time to minimize interruptions to resources and services. This is where Elastigroup from Spot by NetApp comes in.

 

Understanding Elastigroup by Spot

Elastigroup is a cloud infrastructure automation service in which the user can provision, manage, and scale compute instances to support any elastic application or load-balanced workload, on top of spare and reserved cloud capacity, without compromising availability.

With an enterprise-grade SLA (Service Level Agreement), even mission-critical and production workloads can benefit from up to 90% cost savings.

Elastigroup provides every feature from a standard cluster management platform but raises the bar with predictive approaches to instance selection, auto-recovery, and autoscaling, along with comprehensive dashboards and advanced automation.

 

How to deploy spot instances with Elastigroup

Pre-requisites

1. Create a Spot account

Spot Sign Up
Spot Sign Up

 

2. Connect your Spot account to your AWS account

Connect AWS Account with Spot
Connect AWS Account with Spot

 

3. Deploy/use an existing AWS auto scaling group

Note: We have the following infrastructure in place on our AWS account for this Proof of Concept (PoC):

  • An AWS auto scaling group scaled across two availability zones
  • 2 AWS EC2 on-demand instances running under region us-west-2
AWS Infrastructure
AWS Infrastructure

 

AWS auto scaling group
AWS auto scaling group

 

AWS EC2 Instance
AWS EC2 Instance

 

Get Started

1. Log in to the Spot Console and select Elastigroup (top-left menu)

Spot Elastigroup Groups
Spot Elastigroup Groups

 

2. Go to Groups -> select Import -> Auto Scaling Group

Spot Elastigroup Import Auto Scaling Group
Spot Elastigroup Import Auto Scaling Group

 

3. Import configuration by selecting the appropriate region and auto scaling group and hit Next.

Import Auto Scaling Group Configuration
Import Auto Scaling Group Configuration

 

Spot creates the Elastigroup based on the ASG (Auto Scaling Group) parameters imported from AWS. A summary of the parameters appears with the Review tab open. You can review the list of parameters.

Import Auto Scaling Group - Summary
Import Auto Scaling Group – Summary

 

4. Hit Create and Elastigroup will import the auto scaling group with existing configuration and deploy the desired infrastructure on AWS.

Import Auto Scaling Group - Complete
Import Auto Scaling Group – Complete

 

Elastigroup after Import Completion
Elastigroup after Import Completion

 

Once the import has been completed successfully, you can:

  • Route to the Instances section to identify newly created spot instances now running under your AWS account managed via Elastigroup.
  • Route to the Costs section to get a breakdown of imported clusters by:
    • Instance Lifecycle
    • Cost by Pricing Option
    • Daily Cluster Cost
  • Route to your AWS account to find additional infrastructure spun up by Spot now being managed by Elastigroup.

 

Instances via ASG and Spot Elastigroup
Instances via ASG and Spot Elastigroup

 

5. Scale down the auto scaling group on AWS

Once Elastigroup takes over to deploy instances and run them on spot instances, you can scale down the AWS auto scaling group to zero.

Going forward Elastigroup would be the deployment arm for running spot instances on AWS with more features and capabilities than a traditional auto scaling group, such as:

  • Delivering fully optimized and automated infrastructure scaling
  • Delivering availability and scalability for scale-out applications, at up to 90% lower cost

To scale down, navigate Auto Scaling Groups under AWS. Select the appropriate ASG and go to Actions, Edit, then Update Group Size all to zero.

Updating existing AWS ASG to zero
Updating existing AWS ASG to zero

 

Graceful instance termination post ASG update
Graceful instance termination post ASG update

 

Reverting to AWS Auto Scaling Group

To revert infrastructure deployment to AWS auto scaling groups, follow the steps below:

  • AWS -> Auto Scaling groups -> Select appropriate ASG -> Actions –> Edit –> Update Group Size all to desired level
  • Spot.io -> Elastigroup -> Groups -> Select appropriate Group -> Actions -> Delete Group

Once completed, your desired infrastructure would be again consuming AWS auto scaling to launch EC2 on-demand instances as defined.

Delete Elastigroup
Delete Elastigroup

 

Additional Information

Before selecting the Create option, Elastigroup presents several options as follows:

1. Export the same configuration using JSON (JavaScript Object Notation), CloudFormation, and Terraform

Import Auto Scaling Group – Multiple Export Options
Import Auto Scaling Group – Multiple Export Options

 

2. Review and edit the imported configuration with the desired state before selecting the Create option for the following:

  • General:
    • Downscale Imported ASG – Elastigroup will automatically scale down the existing auto scaling group on AWS as per the desired target.
    • Downscale will only happen once Spot launches new instances on your AWS account that are running healthy.
Import Auto Scaling Group General
Import Auto Scaling Group General

 

  • Compute:
    • Edit the existing configuration with the desired compute parameters, such as changing the VPC (Virtual Private Cloud), Operating System, Subnets, instance types, and spot types.
    • Review compute also allows users to edit/add launch specifications, User Data Startup Script, and/or Shutdown Script.
    • Enable persistent stateful resources such as root volumes, data volumes, and IP addresses.
    • Integrate with Chef, ECS (Elastic Container Service), Beanstalk, Docker, Gitlab, Kubernetes, and more.
Import Auto Scaling Group - Compute
Import Auto Scaling Group – Compute

 

  • Networking:
    • Elastigroup import allows attaching the imported group with load balancers or Spot ITF (Intelligent Traffic Routing).
Import Auto Scaling Group - Networking
Import Auto Scaling Group – Networking

 

  • Predictive rebalancing:
    • Define the capacity workload to Min, Max, and Target
    • Define Optimization Strategies, such as:
      • Failback to On-Demand Instances
      • Utilize Commitment Plans
      • Cluster Orientation
      • Spot Market Scoring
      • Preferred Availability Zones
      • Preferred Spot Types
      • Draining Timeout (seconds)
Import Auto Scaling Group - Predictive rebalancing
Import Auto Scaling Group – Predictive rebalancing

 

  • Scaling – Scaling lets you define the following:
    • Metric List
    • Target Scaling Policies
    • Simple Scaling Policies
    • Termination Policies
    • Scheduling
Import Auto Scaling Group - Scaling
Import Auto Scaling Group – Scaling

 

When you are finished reviewing and making modifications to the configuration, click Create. Spot will create your Elastigroup.

 

Additional resources on Elastigroup