Everything you need to know about autoscaling spot instances

With so many types of instances in AWS available, using as broad a mix of instances and pricing models is a great approach for ensuring maximum availability. AWS auto scaling and more specifically EC2 auto scaling groups as well as Spot by NetApp’s Elastigroup and Ocean can be very helpful in this regard. Let’s review EC2 auto scaling functionality and see how it compares to Elastigroup and Ocean’s support for mixed instance types along with support for all AWS’s pricing models including spot instances, Savings Plans, RIs and on-demand instances.

In this article:

What has changed in EC2 Auto Scaling groups and EC2 Fleet behavior?

You can now use a mixed instances policy in an EC2 Auto Scaling group, to mix on-demand and spot instances, as well as different instance types.

In the case of a scale-up operation, how does EC2 Auto Scaling Group determine what instance type to launch?

This is determined according to the Auto Scaling group’s allocation strategy. You can use the capacity-optimized allocation strategy to launch instances from the most-available capacity pools, or the lowest-price allocation strategy to launch instances with the lowest spot price.

For Spot by NetApp’s Elastigroup and Ocean users, the instance types and pricing model are chosen based on the workload performance requirements, spot instance capacity-pool availability, and price. Moreover, Spot by NetApp’s predictive rebalancing detects spot instances that are likely to be interrupted up to an hour in advance, allowing for graceful draining and instance replacement. Intelligent workload placement ensures that unused AWS Savings Plans and RIs are utilized before launching new spot instances. If there is no available reserved capacity or spot instances, Elastigroup and Ocean will automatically fallback to on-demand to ensure availability, reverting to spot instances as soon as feasible.

For Kubernetes and containerized applications, Ocean provides the additional benefits of container-driven autoscaling which perfectly matches container resource requirements with underlying compute infrastructure, freeing DevOps from managing servers and driving additional cost savings with node bin-packing.

When configuring the usage of Spot Instances in an EC2 Auto Scaling group, is there a guarantee that I always have the right amount of compute capacity?

No. Whenever spot capacity pools are unavailable, AWS’s EC2 Auto Scaling group will not be able to spin up new instances and will wait until spot instances will be available again.

On the other hand, Elastigroup and Ocean will simply fall back to on-demand instances (or any available Savings Plans and RIs) when spot capacity isn’t available, guaranteeing availability. Elastigroup and Ocean will revert back to spot instances as soon as they become available again. In short with Spot by NetApp your compute capacity is guaranteed and backed by an SLA.

Is it possible to prioritize certain instance types in an EC2 Auto Scaling group?

No. The EC2 Auto Scaling group will select instance types according to your allocation strategy (capacity-optimized or lowest-price).

In order to maximize the availability and achieve the lowest possible cost in the spot instance market, it is recommended to select as many instance types as possible but this may cause a performance hit. With Elastigroup and Ocean however, you can prioritize certain instance types over others. Elastigroup and Ocean will launch your preferred instance type if it’s available as a spot instance and only use other instance types when the preferred type is unavailable.

Will EC2 Auto Scaling groups be able to automatically allocate and prioritize my Savings Plans and RI utilization?

No. EC2 Auto Scaling groups will only launch the number of instances you configured for each pricing model.

Elastigroup and Ocean, however, will intelligently utilize your AWS Savings Plans and reserved instances, before launching spot instances. Both Elastigroup and Ocean are aware of your entire reserved capacity purchases and apply the entire business logic that is associated with properly utilizing reserved instances across accounts. Whenever Spot by NetApp launches a new instance, it will determine if there are any vacant Savings Plans or RIs and will automatically provision appropriate instances that will utilize the available reserved capacity. Only after achieving full Savings Plans and RI utilization will spot instances be provisioned.

Will EC2 Auto Scaling group be able to gracefully drain spot instances, before they are terminated?

You can enable capacity-rebalance on your Auto Scaling group, which will automatically react to the rebalance recommendation – a signal that a spot instance is at elevated risk of interruption. AWS states that this signal can arrive earlier than the 2-minute interruption notification but does not guarantee it. If the signal does arrive in time, it can be used to start your instance shutdown procedures, prevent new work from being started or launch replacement instances before terminating the at-risk spot instances. This process will also deregister the terminated instance from Elastic Load Balancing, or run Lifecycle Hooks.

In contrast, Spot by NetApp’s predictive rebalancing algorithms are able to select spot instance with sufficient longevity for your workloads’ specific needs. In addition, spot instance interruptions are detected up to an hour in advance, guaranteeing workload continuity and performance. Whenever an interruption is identified, Elastigroup and Ocean will gracefully drain the at-risk instance, replacing it with either available Savings Plans, RIs or other spot instances, only falling back to on-demand if absolutely needed for maintaining availability. This is all done automatically, without any need for hands-on monitoring or management. Of course, all preemption is done gradually, maintaining TCP/HTTP state as well as container constraints to ensure service uptime.

Here is a comparative table for quick reference:

EC2 Auto Scaling Group Spot by NetApp (Elastigroup and Ocean)
SLA When running spot instances with AWS’s EC2 ASG, there is no capacity availability guarantee and no SLA. With Elastigroup and Ocean your capacity is guaranteed and backed by an SLA.
Prediction You can enable the capacity rebalance feature on your AWS EC2 Auto Scaling group to automatically react to spot instance rebalance recommendation notifications that can be received before the 2-minute interruption, and Auto Scaling groups will relaunch replacement instances. However, AWS does not guarantee that the interruption signal will arrive more than 2 minutes in advance. Predictive rebalancing algorithms detect spot instance interruptions up to an hour in advance and automatically moves workloads to available instances that can be covered by unused Savings Plans or RIs. If none are available, spot instances with the greatest longevity will be selected.
Graceful Draining By enabling EC2 Auto Scaling group capacity rebalance, your instances will be replaced when a spot instance gets a rebalance recommendation, and will adhere to Lifecycle Hooks as well as automatically deregister the terminated instances from Elastic Load Balancing. With prediction of interruptions up to an hour in advance, Spot by NetApp automatically replaces instances to another spot instance capacity pool (only after checking that there are no unused Savings Plans or RIs that can be used). The preemption is done gradually, maintain TCP/HTTP state and containers constraints to ensure service uptime.
Determination of instance types to launch According to the allocation strategy configured in the EC2 Auto Scaling group. The instance types are chosen based on the workload performance requirement. The pricing models are selected intelligently prioritizing usage of unused Savings Plans and RIs before launching new spot instances. Predictive algorithms determine and choose relevant spot instances with the greatest longevity.
AWS Savings Plans and reserved instance utilization EC2 Auto Scaling group will launch exactly the number of instances you configured for each pricing model, without attempting to utilize unused Savings Plans or RIs. Elastigroup makes sure to first utilize unused Savings Plans and RIs, before launching spot or any other on-demand instances.
Instance types & Availability Zone prioritization EC2 Auto Scaling group will attempt to launch an identical number of instances across all the availability zones that you added, and then apply the allocation strategy. You can prioritize certain instance types and AZs over others. Preferred instances will be launched as long as it is available as a spot instance and other instance types will only be used when the preferred type is unavailable.
Containers Autoscaling EC2 Auto Scaling group uses traditional scaling metrics to autoscale containerized applications. You can use orchestrator-specific autoscaling tools like Amazon ECS cluster autoscaling, or Kubernetes cluster-autoscaler. For containerized applications, Ocean’s container-driven autoscaling determines the best blend of instance types from different instance families and sizes (including GPU & FPGA support) to match container resource requirements (CPU, Memory, Network, etc.) in the cluster.