SimilarWeb uses Spot to cut their AWS migration costs

AWS Migration was time-consuming and potentially expensive

In early 2017, SimilarWeb was faced with the task of migrating a massive on-prem system onto the cloud. Migrations are never going to be easy, but with many data-heavy workloads and various product needs to account for, the complexity of this task was overwhelming. It also can be expensive at first, even if it lowers costs in the long run.

The goal for the SimilarWeb team was clear; migrate onto the cloud without raising their IT costs. Oz Katz, Head of Production Engineering at SimilarWeb, led the move by starting with a few workloads to test their assumptions. Quickly they discovered three essential best practices for their migration:

1. Utilize AWS – and the assistance they offer
2. Be very cognizant of costs– use RIs and spot instances from Day One to cut costs
3. Start slow –  SimilarWeb first migrated a few simple workloads and tested solutions like Spot on those workloads before migrating

How SimilarWeb managed to migrate in one year without going over budget

SimilarWeb’s Best practices for cutting migration costs

1. Spot for spot instances wherever possible

Katz said that “lowering costs is one of the biggest hurdles during migration to AWS.” Spot was a key player in helping them keep costs down – any workloads that they could run on spot instances using Spot, they did.

“Eventually”, Katz said, “we might have been able to tool our own solution for running some workloads on Spot, but Spot’s focus and expertise made it a turn-key solution.” With the weight of an entire migration on their shoulders, tooling and running a spot solution themselves just wasn’t an option, even for their experienced team.

2. Utilizing RIs everywhere else

Wherever running on Spot wasn’t an ideal solution or required a bit of extra tooling, Katz would utilize RIs. By purchasing 1-year RIs, Katz and his team were both able to lower costs instantly and to maintain flexibility if they found they had chosen the wrong instance type or could eventually run this workload on spot to save further.
As the Production Engineering team at SimilarWeb works closely with each product team, lowering costs was not always the main focus. “Sometimes we had other priorities within specific products but eventually, when keeping product costs was a priority, Spot will always be the solution.”

SimilarWeb’s best practices for migration orchestration

Katz found a few platforms to be essential for maintaining a smooth migration and managing workloads thereafter:

Terraform for cloud resource provisioning

Especially when migrating to the cloud, finding a strong orchestration tool for all workloads is essential to save time and effort. Additionally, by leveraging Infrastructure-as-code, their team could create repeatable components and spin up their entire stack on a new AWS region in days.

Nomad for Container Orchestration

Kubernetes is great but far too complicated to manage directly. Nomad was their go-to solution as they were already utilizing HashiCorp’s full range of products, including Consul and Vault. for orchestrating large-scale workloads.

Spot for Nomad autoscaling

While Nomad and Terraform are great orchestration tools, Spot’s autoscaler makes it far easier to manage at scale, leaving SimilarWeb’s team time to focus on their main tasks.

Spot for log visualization

“Visualizing logs at scale,” said Katz, “is very, very expensive.” Ultimately, when you have a ton of logs that demand the ability to see inside, capturing all of this data and storing it costs a lot, “At our scale, most SaaS providers are simply too expensive” Katz added.
Katz quickly realized that he could run these workloads on spot via Spot. Within a week, it was up and running.

Next up for SimilarWeb

After a very successful first year to their migration, SimilarWeb’s main focus in 2018 is to complete their migration and continue implementing the best practices they established in their first year on AWS.