Instant scale up for even the most dynamic ECS clusters

One of the key features of Ocean by Spot is a “headroom” feature, the ability to maintain a dynamic buffer of spare capacity for immediate scale-up. Ocean continuously predicts which workloads are most likely to require scale-up and adjusts headroom in line with this prediction to enable immediate scheduling of new tasks, without waiting for infrastructure provisioning. This shortens the time to execution for these workloads and dramatically speeds up the scale-up process.

Previously, the automatic headroom algorithm maintained a fixed 5% of available cluster resources (sum of requested CPU and Memory). While this would address a lot of use cases, it was not granular enough to meet the needs of all users.

We are pleased to share that we have now enhanced our headroom mechanism to support the most dynamic ECS clusters by allowing for custom configuration of spare capacity. 

This new feature allows users to customize the percentage of the total cluster capacity that the headroom will take up. The result is a configurable buffer which both scales with the cluster dynamically, and leverages the autoscaler’s ability to learn the cluster’s workload, ensuring every task can be scheduled without delay.

For mission critical clusters, for example, users can increase the headroom percentage to support high availability. A larger amount of spare capacity will provide a larger safety net that will ensure incoming tasks don’t end up in a “pending” state.

On the other hand, for dev/test clusters where task schedule time is less of a priority, customers can decrease the buffer to promote higher savings and resource allocation efficiency.

Furthermore, environments that are characterized with well known peak times can benefit from scheduling the adjustment of the headroom percentage a few minutes prior to the peak time, resulting in a smooth scale of services. Such scheduling can be applied on the user end, utilizing the Ocean API.

For more details about the API to activate this feature, please see the relevant Spot documentation.