Cybereason Reduces Dev/Test Costs by over 50% with Elastigroup by Spot
Organizations that run their software on public cloud providers enjoy the simple deployments and ease of management but costs often become a concern as a business scales. As Cybereason grew rapidly, so did their AWS bill. Cybereason wanted to focus their investment on product and people, so they created a new mandate of cost-efficiency, and started looking for ways to reduce their operational costs. One of the candidates for cost optimization was the R&D Development workflow.
Cybereason’s backend infrastructure is run on AWS EC2 with Elastic IP’s and additional data volumes using EBS for each instance. They manage many different versions of their product in production and each deployment consists of a minimum of 3 instances. By having to support a wide variety of software versions, their developers will need lab environments deployed to support each version which is costly.
Cybereason started to investigate EC2 Spot Instances – which are excess capacity instances sold at steep discounts up to 90% off, In the first phase, Cybereason tried to develop a spot instance management system by themselves with open source tools, after building an initial solution they faced edge cases such as re-mounting EBS volumes after spot interruptions and attaching the same public IP to the newly launched instances. Eventually, Cybereason approached Spot to test the Elastigroup by Spot‘s stateful solution.
For stateful workloads, availability is crucial for a service to remain functional. Database administrators, for example, need to ensure that they have a certain number of instances online to ensure data integrity. Administrators are apprehensive to using spot instances because they can be interrupted by the cloud provider with little notice. Luckily with Elastigroup by Spot, organizations like Cybereason changed their minds about using spot instances for stateful use cases.
Based on historical and statistical data, Elastigroup by Spot predicts interruptions ahead of time and automatically migrates instances into different machine Types, Zones, or falls back to on-demand to ensure high availability and consistency. This was important for Cybereason because they could use Elastigroup to utilize spot and on-demand instances to ensure availability and reduce costs as much as possible. When they learned that Elastigroup also has support for managing stateful resources on spot instances, they started their evaluation.
When using a stateful Elastigroup, you can choose to have the Elastic IP, Private IP, and data volumes automatically attach to the newly created spot instances. The end result is a new instance configured exactly like the previous one with the same data. This is key for stateful workloads because when the instance comes back online, it will be as if it never left. To further help Cybereason reduce costs, Elastigroup was able to help automatically scale down their infrastructure outside of business hours.
Demand for a service increases and decreases throughout the day. As demand increases, the number of compute instances should increase to meet the demand. When demand decreases and the number of instances remain the same, organizations are wasting money.
Elastigroup by Spot was able to help Cybereason with this problem by being able to schedule scale down operations outside their defined business hours. When it is time to scale down instances, Elastigroup will gracefully pause the instances to meet the minimum and target capacity specified.
The paused instance resources such as EBS volumes and Elastic Network Interfaces, are detached and are kept in an available state until it is time to re-launch. When it is time to scale up again, the paused instances will be brought back online with the existing resources attached to the new instances as they launch.
Here are some of the ways Elastigroup helps businesses reduce costs and operational overhead.
- Seamless Integration – Elastigroup integrates with dozens of applications and services including Kubernetes, Amazon EKS, Elastic Load Balancer, Application Load Balancer, Autoscaling Groups, Amazon ECS, Terraform, AWS CloudFormation, Beanstalk, OpsWorks, EMR, CodeDeploy, Jenkins, and more.
- Intelligent Auto-Scaling – Elastigroup provides tailor-made auto-scaling for your application. Metric-based or event-driven, Elastigroup will always make sure that you have the optimal mix of right-sized VMs running to meet your workloads needs.
- Spot Market – Elastigroup reliably leverages the Spot Market to optimize your underlying infrastructure for cost without compromising availability.
- Always Up – Elastigroup uses predictive algorithms to identify and drain VMs that are about to be preempted. Prior to VM termination, Elastigroup seamlessly launches a new VM and replace it seamlessly. Elastigroup will also make sure to distribute your VMs across different sizes and types to optimize cost reduction and VM longevity.
Elastigroup’s support for stateful applications provided Cybereason with the ability to run their workloads for their lab and production environments on a mix of reserved, on-demand, and spot instances. Cybereason was able to reduce their cloud computing costs by over 50% across development and production environments, with their CI servers running on spot instances as well, and managed by Elastigroup. By being able to utilize on-demand and spot instances, Cybereason was able to reduce costs and ensure availability for their workloads. By using Elastigroup to schedule capacity, the Cybereason team was able to further reduce costs by scaling down unused instances for non-peak hours. Going forward, Cybereason is excited to look at how Elastigroup works with Terraform, Google Cloud Platform, and migrating their Kubernetes KOPS environment to spot instances.