CI/CD on Spot Instances

What is CI/CD?

CI (Continuous Integration) and CD (Continuous Deployment/Delivery) are related concepts designed to improve efficiency in developer teams, featuring both a cultural and technical element. CI/CD aims to improve efficiency by ensuring that code across an entire development team is always integrated with the central line of code.

CD differs from CI through deploying all code to a testing environment to make sure that changes are fully tested and ready for release. There is also a difference between Continuous Deployment and Continuous Delivery, even though both are sometimes referred to as CD. In Continuous Deployment, the moving of code changes from staging to production environments happens automatically, whereas for Continuous Delivery this process must be manually approved.

Cultural and Technical CI/CD

The cultural element of CI/CD is a focus on frequent code integrations across a developer team, often multiple times per day. This is designed to make sure that any bugs or flaws in the code can be found much more quickly. If a developer’s code is frequently integrated then there is a lot less possibly “bad” code to check for issues. Put simply, if you can check every 6 hours of work for errors in code that would make it un-integratable, there’s a lot less code to try and fix if anything goes wrong compared to if you check every 6 days.

The technical element of CI/CD is software which can automatically verify the health of developer code as it is regularly merged into the main line of code. Without an automated element, CI/CD would not be possible as the constant manual health-checking would slow down the developing process to a near stand-still.

Possible Challenges with the CI/CD Pipeline

One issue with the CI/CD for many companies is that, whilst it undoubtedly improve the lives of the developers, it can lead to headaches for company budgets as growth leads to increased expenditure in the test/staging environments.
Increasing number of developers (always a good thing) leads to increasing amount of tests, therefore increasing amount of servers required to run tests and increasing costs of doing so (always a bad thing). This is where companies often face a toss-up between staying Cloud-based to allow ease of scaling or else move to On-Prem environments to allow reduced cost. You can read a detailed case study on this process written by Capriza.

Cost-effectiveness of CI/CD processes

Using Cloud providers spare capacity instances (i.e. Spot Instances on AWS) is a great way to radically increase the cost efficiency of Cloud environments. CI/CD processes are some of the best use-cases for Spot Instances, as testing environments need be intensely scalable whilst being as cost-efficient as possible.
As CI/CD processes are not directly customer-facing, the damage caused by any interruptions to Spot Instances is also much smaller than production or mission-critical environments.

Elastigroup

This is where Elastigroup can be of help to reduce costs for the CI/CD pipeline automation whilst simultaneously allowing a company to expand their testing by utilizing Elastigroup’s ML algorithm to predict instance terminations and prevent 99% of Spot interruptions from actually happening.
With Spotinst providing Spot Instances the same SLA as On-Demand Instances for a greatly reduced price, companies can enjoy all the benefits of running a CI/CD pipeline on Spot Instances without having to worry about the issues that Spot can raise.

Checkout Elastigroup for:

Jenkins

The Elastigroup Jenkins plug-in helps customers do more with a Jenkins setup. It does so by allowing users to configure Jenkins to automatically scale designated Amazon EC2 Spot Instances as Slaves up or down depending on the number jobs pending on the Jenkins queue. Learn More

GitLab

Using Gitlab Runner’s autoscaling feature you can now configure Docker Machine to provision Spot instances on Elastigroup. This gives both the cost savings of Spot instances and the 100% availability that Elastigroup provides. All while retaining full control over your CI/CD with Gitlab Runners. Learn More

Code Deploy

Spotinst Elastigroup integrates with CodeDeploy and acts as the infrastructure layer for your CodeDepoly environments, enabling the launch and management of EC2 Spot Instances according to your configuration and specifications right into your deployment groups with the latest changes.  This enables you to keep your application highly available and benefit the Spot instances low costs, 80% cheaper than On Demand. Learn More