How Lacework Innovates Cloud Security Solutions on Kubernetes Efficiently and Affordably with Spot

About Lacework

Founded in 2014, San-Jose based Lacework provides security and compliance for the cloud, leveraging machine learning to automate vulnerability detection and threat identification for ongoing cloud security. Lacework’s ingestion and analysis of vast amounts of security log and activity data runs on Kubernetes clusters in AWS. Without meticulous and time-consuming DevOps management these clusters can be highly inefficient and prohibitively expensive.

Spot by NetApp Benefits

• Freed DevOps teams from tedious management of underlying compute infrastructure
• Ensured nodes always matched pod resource requirements with container-driven autoscaling
• Increased cluster utilization with advanced pod bin-packing onto heterogeneous pool of instance types
• Simplified cost analysis with Kubernetes-native showback
• Streamlined on-boarding and integration with 24/7 expert support

The Challenge – Cluster Inefficiency and Drained DevOps

When Matthew Zeier, Senior Director of Production Engineering & Operations at Lacework joined the company in 2020, their Kubernetes clusters were being manually managed. This resulted in significant cluster inefficiencies with pods either running on nodes that were far too large for their resource requirements, or worse, pods not getting scheduled in a timely fashion where nodes were too small for their needs.

Additionally, Matthew was running the clusters on EC2 spot instances to keep costs low. Coupled with the manual scaling of nodes to match pod requirements, ensuring that the spot instance interruptions didn’t disrupt production environments was a major drain on the team’s time and a distraction from their core tasks.

The Solution – Automated Optimization for Kubernetes Clusters

After seeing that the do-it-yourself approach for Kubernetes was unsustainable for Laceworks’s rapidly expanding business, Matthew chose Spot Ocean from Spot by NetApp as a turn-key solution for automating and optimizing their over 20 Kubernetes clusters running on some 300 EC2 instances.

Spot Ocean is a serverless engine for Kubernetes in the public cloud, empowering businesses to rapidly and reliably deploy mission-critical workloads on an optimal blend of spot, reserved and on-demand instances, without needing to worry about compute infrastructure management.

“For me the entire Spot Ocean narrative is about freeing DevOps from manual work. Any task is more important than managing Kubernetes and its underlying compute,” Matthew noted. He further explained, “Spot Ocean automates cluster optimization for us, rapidly scaling nodes up and down as needed and bin-packing the remaining pods. This increased our compute utilization and cost efficiency by 20%.”

Results and Benefits – Freedom to Focus on World-Class Cloud Security Solutions

With Spot Ocean the Lacework team no longer needs to deal with manual, time-consuming compute layer management. Spot Ocean’s Tetris-like autoscaling allows Lacework to run applications with very different resource requirements, on a heterogeneous blend of all available EC2 instances within a single cluster. Whenever there are underutilized nodes in any of Lacework’s clusters, Spot Ocean will proactively scale down the excess resources, when appropriate, and bin-pack the remaining pods onto other nodes delivering on average a 20% increase in cluster utilization and cost efficiency. Furthermore, Spot Ocean’s predictive algorithms help ensure that Lacework can run their applications on highly affordable spot instances without worrying about workload interruptions.

When it comes to visibility into cluster spend Matthew enthused “I love Spot Ocean’s cost analysis for Kubernetes. Finally, I can have meaningful conversations with our engineers about what each and every pod actually costs in terms of compute and storage. This granularity which goes down the level of namespace, resource, annotations and label, allows us not only to do proper cost allocation, but dig into and remedy any surprise costs.”

In addition to the core benefits of Spot Ocean, from day one, the Spot team was adding value with expert support for integrating Lacework’s older version of Kops (1.12) with Spot Ocean. When Lacework began using EKS, the Spot team was there again to ensure a quick and smooth migration, ensuring throughout that Lacework’s DevOps team were free to focus on higher-value tasks.