Winning Combo – Kubernetes, Terraform And Elastigroup by Spot

Based in Stockholm, Sweden, [24]7 Predictive Search Bidding (formerly known as Campanja) provides a predictive bidding engine for online advertising allowing their customers to maximize each dollar spent on search advertising (Google AdWords and Bing Ads). [24]7 Predictive Bidding utilizes Real Time Tracking, Predictive Modelling, and High-Frequency Bidding to harness the power of online advertising for their clients. [24]7 was an early adapter of Kubernetes (they have been using Kubernetes since version 1.0) and has been pushing forward with their usage of Kubernetes with spot instances.

High-Frequency Bidding for Google AdWords

In the early days of search engine advertising, we learned that the process was very old-fashioned. Customers would go to an agency and have a new campaign for Google AdWords. The agency would manually place bids for advertising on Google but since this was all done by hand for each keyword they would only be able to place at maximum a couple hundred bids a day for an AdWords account. Eventually, [24]7, then Campanja came up with the concept of a “High-Frequency Bidder” through automation. With the High-Frequency Bidder, they can place millions of bids per day with different aspects such as keywords and cost per click. They can determine the appropriate cost per click based on each query to ensure that their clients are not paying too much when placing bids. If your ad is going to show up in the search you have to know when is the optimal time and day for you to show up on the query. Sometimes it’s not worth it to you to show up because it’s too expensive for the conversion you will get. Other times, it’s worth it and you should bid higher to make sure you get on the search page. The high-frequency bidder can place bids much faster than a human can do, this ensures that bids are placed at the appropriate price at any given time and usually ensures that the company is paying the right amount at the optimal time to get a conversion. Since you get much better value from people clicking on links to your sites it makes sense to go with this method vs impressions (paying per viewing of the link). [24]7 Predictive Search tracks clicks to their client’s websites and can calculate exactly what each click costs, how much the client thinks it is worth, and what the bid should be. This type of automation around online advertising is the future and is like Nielsen and an ad bidding partner all in one.

Terraform and Kubernetes

[24]7 Predictive Search utilizes separate environments that are defined by Terraform. Typically these environments include production, development, and staging. Environments for dev and staging are frequently torn down when not in use and provisioned again when they are needed. Another advantage of using Terraform is the ability to separate the environment into individual modules. VPCs, Networking, Firewall Rules, and Elastigroup by Spot Clusters are all configured as separate modules and can easily be individually modified as needed. This granularity and control over each aspect of their environments are what makes Terraform so compelling. New versions of the infrastructure can be easily deployed when a new version of Kubernetes or CoreOS is available and makes updating systems painless.

“The Spot Terraform Provider makes it simple to provision EC2 clusters on spot instances. We simply define the configuration and Spot takes care of the rest. We can easily save 80% or more on our EC2 costs”

Cost Savings and Spot

We currently run about one-third of our Kubernetes nodes on on-demand instances. These are typically instances that have large data volumes attached and typically need to run for longer periods of time (like MySQL for example). The other two-thirds of our instances are all run as spot via Spot Elastigroups. These Elastigroup clusters are all provisioned with Terraform as well and when we need to make a change like an AMI or UserData script, for example, we can easily make this change with Terraform. If we need to launch MySQL pods these will be provisioned on instances labeled on-demand. Most of our services are spikey, this is why it makes more sense for us to scale our Kubernetes nodes running on spot via memory rather than CPU. We also have a separate GPU Kubernetes nodes specifically for Data Science. These are less time critical and typically run as batches at night. We also built an Erlang program called PodMon that just collects all our Kubernetes reservations into CloudWatch. We can then scale the number of Kubernetes nodes with Spot via the custom CloudWatch metric created by PodMon.

Custom workflow to scale based on the number of Kubernetes reservations.

“With Spot we managed to reduce our Kubernetes infrastructure costs by at least 80%. We also have peace of mind that our live production environment is protected from spot interruptions or lack of capacity concerns. The Spot API integration with Kubernetes also ensures that there is a seamless transition from an expensive spot to a cheaper spot that is all handled Seamlessly. We’re experiencing an interruption rate that is basically 0% annually”

Learn more about [24]7 and how they are able to help their clients maximize their ad dollars at https://www.247-inc.com