Fyber deploys dynamic Kubernetes workloads with Spot Ocean for infrastructure management

About Fyber

• Founded in 2009
• HQ: Berlin
• Industry: Software

Fyber is an app monetization company with a platform that supports more than 10,000 mobile apps and games.

Spot by NetApp Benefits

• Automated, flexible infrastructure to support dynamic Kubernetes applications
• Eliminate manual infrastructure management
• Integration with Terraform for fully automated environments
• Efficient resource allocation to reduce overprovisioning and under-utilization
• Reliably run clusters on spot instances for significant cost savings
• 24/7 expert support

The Challenge

Building a scalable, flexible cloud environment for large-scale data enrichment

Fyber’s next generation app monetization platform is processing 10 billion TBs of data every day, and unlocking this information is critical to support their customer base of app developers and publishers. To extract value and insight from these datasets, Fyber was running a Hadoop cluster with AWS EMR and executing Spark jobs to enrich the data. Initially a static cluster with underutilized resources, Fyber migrated these workloads to AWS Elastic Kubernetes Services (EKS), but they needed a dynamic, flexible cluster to be able respond to the needs of large Spark data enrichment jobs.

The Solution

Ocean to run infrastructure for Kubernetes out-of-the-box

As a long-time customer using Elastigroup, Fyber was already experiencing the cost-saving benefits of working with Spot by NetApp. When it came time to migrate to Kubernetes, Fyber turned to Spot’s serverless container engine, Ocean, to help build the scalable infrastructure their applications needed.

“Using Ocean really helped speed up our migration to Kubernetes and EKS,” said Lioz Nudel, DevOps Senior Team Lead, Fyber. “It saved us from having to configure the whole environment, not to mention how much we save by reliably running on spot instances.”

Results and Benefits

Dynamic autoscaling gives applications flexibility and frees teams from manual infrastructure management

With Spot Ocean integrated with their EKS cluster, Fyber is able to run resource-intensive data enrichment tasks without worrying about infrastructure. The Ocean controller, installed in the data plane of the cluster, continuously gathers events and resource allocation requests from the EKS cluster and adds Kubernetes instance workers in the form of spot instances. When executing a job, Ocean’s intelligent, automated mechanisms know when resources are needed, and scales up the cluster accordingly. Unlike their previous static cluster which always had to be at maximum size, with Ocean the cluster is running more efficiently with automatic scale down when cluster resources are underutilized.

“Autoscaling infrastructure was a black box for us but using Ocean we don’t worry about it anymore. With Ocean, we didn’t need to develop something of our own, the cluster just scales up and down when it needs to,”  said Ori Bracha, DevOps Tech Lead at Fyber.  

Provisioning Ocean with Infrastructure as Code

Fyber uses Hashicorp Terraform to generate architecture for their EKS environments in one click, and Spot’s integration with the infrastructure-as-code tool has made it easy for Fyber to manage resources that interact with both Ocean and Terraform. In fact, the Spot team worked directly with Fyber to ensure that they could provision Ocean with Terraform for a fully automated environment.

Expanding Ocean’s application-driven scaling to more applications and services

Seeing the benefits Ocean afforded their data environment, when Fyber began to move other services into containers, including their main product application, using it was a no-brainer. “It just made sense to move more of our applications into containers and put them into Ocean” said Bracha. “We know it gets great results, and we don’t have to spend the time integrating it to see if it works well.” A new framework was developed for Fyber’s CI/CD deployment pipelines using the EKS data cluster as a blueprint, and these applications were moved into Docker containers running with Ocean. With each pod demanding a certain amount of resources, Ocean’s application-aware automation spins up new nodes according to requirements, always considering the application’s specific taints, tolerances and affinities. This autonomous system also made resource utilization much more efficient, with right-sizing capabilities that helped Fyber allocate the right type, size and amount of resources.

In addition to the day-to-day value Spot Ocean was bringing to the DevOps and data teams at Fyber, Spot by NetApp provides direct customer support for any question or escalation they have. As an early adopter of Spot, many Fyber’s feature requests are now major capabilities that support a wide variety of customers and use cases. “Spot’s customer support is the best we’ve seen from any third party service,” said Nudel. “Every question, every escalation is answered quickly and with a lot of attention.”