The optimization imperative: Sustainably scale K8s in your IDP

It’s considered common practice for platform engineering teams to build their IDP to be a self-service product for developers. This means, among other things, building a dedicated team around it for upkeep and support, creating a roadmap for development, and coming up with metrics they can use to determine its success. But with the freedom given to developers to provision their own resources, a consequence of this approach can be cost inefficiency: in some places, cloud resources like compute, throughput, or storage may be over-provisioned, while in others they may be idle and underutilized. The result is not only expensive but can also make it impossible to manage power consumption, creating difficulties for organizations as they try to meet their sustainability agendas.

Many IDPs are built on top of managed Kubernetes services, so container optimization can be an effective way to deliver savings. To support the IDP’s scalability, platform teams need to leverage automation to continuously manage the scale and sizing of their clusters based on container resource needs.

Let’s consider the four most prominent ways to use automation for continuous Kubernetes resource management.

1. Same load, fewer nodes: Bin packing

In order to support tasks like traffic spikes and batch processing jobs, it may make sense for developers to provision bigger nodes than necessary. However, on a daily basis, this can lead to lots of underutilized nodes — and unnecessary costs. This is where bin packing can help.

Also known as scaling down, bin packing automatically allocates pods onto nodes. It also provides the ability to pack multiple pods into every node, rather than having them each assigned their own node. This reduces the overall number of nodes, helping to optimize resources.

Spot Ocean has bin packing enabled by default, and even lets you specify the exact percentage of scale down required. This makes it possible to prioritize specific workloads, while scaling down others more gradually.

Automatically place your containers in a way that optimizes resource utilization and minimizes wasted capacity.

2. Same load, smaller nodes: Rightsizing

Rightsizing your applications refers to the process of ensuring nodes are optimally sized for your workloads. This is a critical part of building resource efficiency into an IDP’s logic: oversized nodes will lead to unnecessary expenses, while undersized ones can create outages or bottlenecks. But rightsizing effectively requires a great deal of precision, as well as quality automation.

By visualizing the CPU and memory utilization data, Spot Ocean can automatically detect when resource request and utilization consistently varies. Ocean then generates intelligent rightsizing recommendations to solve these issues. Users can either decide to implement these recommendations manually, or create a set of rules that allow Ocean to rightsize the resources for them and match their workloads’ exact needs. By automating the rightsizing process, users can continuously optimize and maximize performance, making it possible to relieve platform engineering teams from having to manually adjust resources to avoid over- or under provisioning.

Maximize utilization and efficiency with tailored recommendations and fully automated rightsizing of compute resources.

3. Shutdown hours

We conserve power by turning off lights when we’re not in a room, so why do many engineering organizations still run Dev & testing resources continuously, even during periods of no activity, such as nights, weekends, and holidays? Instead, they can save energy simply by shutting down resources when they are not used. 

This is exactly what shutdown scheduling does. By setting shutdown hours for non-critical services, the IDP can automatically reduce idle resources during off hours (or any other period when usage is not high). Spot Ocean makes this easy through a simple configuration of the times when specific resources can be shut down. It’s a straightforward, but effective strategy to ensure your developers don’t leave the lights on when they go home!

4. Idle storage resources

The Dev & testing environment has much more to it than just compute. Unused storage resources can play just as big a part in making it inefficient and wasteful. That’s why it’s important for an IDP to regularly audit storage systems used by developers. This makes it possible to remove unused volumes and resize as needed, helping to improve overall efficiency.

Spot Ocean makes this process easy. Just set “dynamic volume size” and “dynamic IOPS,” and Ocean automatically starts removing any underutilized storage resources. This will also align attached volume and IOPS with node size, avoiding overprovisioning in smaller nodes.

How to tune your IDP for continuous K8s resource optimization

Any organization interested in reducing wasted energy and shrinking their carbon footprint needs to be consistently looking for new ways to make their cloud estate more resource-efficient. Thus, when operating an IDP in a K8s environment, continuous optimization is imperative. That said, infrastructure automation is complex and lengthy to develop at-home.

By integrating an enterprise-grade solution like Spot Ocean into the IDP’s resource plan, large-scale organizations can take advantage of a multitude of options for getting more performance from their Kubernetes infrastructure at less cost.

We can help you discover how your internal developer platform can benefit your entire organization. Try Spot Ocean or schedule a demo to optimize your cloud resources and become more efficient.