Magnetic

Magnetic’s EC2 Spot Implementation Restricted by Time Consuming Processes and Hadoop Complexities

“Spot isn’t just a vendor to us, they’re a partner we know we can work with.”

– James Marcus, VP of Technical Operations

Posted on Magnetic.com, Magnetic sought to migrate 90% of workloads to EC2 Spot within a year, while 30% of their workloads were already running on EC2 Spot, the more complex systems, like their Hadoop clusters, were more difficult to migrate.

Unlike spot media, using spot servers – at the scale Magnetic does – is a huge technical undertaking. That’s why we enlisted a partner like Spot to help us get up and running ASAP.

– James Marcus, VP of Technical Operations

Accelerating time to market

Spot helped Magnetic achieve its goal of 90% spot instance usage in a matter of months (vs. its original 12-month goal) by:

• Simplifying migration
  With Spot, migration to spot instances took mere hours vs. upwards of 6+ months for certain environments
• Providing stateful application support
  Allowing stateful applications to run on spot instances, with the ability to maintain private IP addresses and reattach data volumes across instances.

This drastic reduction in implementation time meant Magnetic could start saving money quicker. The company’s existing spot instance strategy proved very profitable, enabling them to achieve over $1 million in savings over the past 12 months. With Spot’s help, Magnetic expects to save an additional $1 million per year.

With Elastigroup by Spot, instead of managing several groups of different instance types and sizes to achieve reliable replication on Hadoop, Elastigroup can run one group that mixes on-demand, reserved, and stateful spot instances. All simultaneously optimized for cost and performance.

Customer Support Gives Magnetic Complete Confidence in Spot

Customer support and experience is important to Magnetic. “There’s a problem in the industry. Many technical vendors fail to listen to customers and have a proper discussion with them,” said James. This is not true with Spot.

Over the course of Magnetic’s PoC with Spot, the Magnetic team submitted 15 feature requests so they could get the most out of the Spot platform. All feature requests were completed within 10 days, allowing Magnetic to increase savings while strengthening their confidence in the Spot team.

“We’ve worked with a handful of vendors who have actually listened and truly responded to customer needs. Spot is definitely one of those vendors,” James added.  “Spot has helped Magnetic remove financial risk by enabling increased spot instance utilization. We can reduce our long-term commitments, which releases cash-flow and generates significant cost savings”