What happens when you free DevOps pros from cost management?

This article was originally published on The Register.

If you’re under pressure to improve your users’ digital experiences, from ecommerce to delivering public services, you’re probably painfully aware that legacy approaches to delivering software applications and features just don’t cut it.

One of the key solutions to both legacy organizations and startups that addresses that shortfall is DevOps. Why? The clue is in the name. As a set of practices encompassing software development and operations, DevOps aims to not just “improve” software but to make the development and delivery of applications and features faster and more resilient.

By breaking down traditional silos, developers are not forced to wait for infrastructure to be provisioned, whether for dev, testing, or deployment. Meanwhile, operations teams are not left having to fix problems with newly deployed applications or features for which they have little understanding and minimal input.

From a technology point of view, DevOps usually — though not always — presupposes the extensive use of the cloud, as well as containers and microservices. Automation, feedback and monitoring are key elements in tying this all together.

While all of this was in motion by the end of the last decade, the COVID-19 pandemic accelerated the shift to the cloud and raised the importance of ecommerce and digital experiences even higher. This, in turn, elevated the profile and expectations of DevOps, as organizations attempt to speed up software delivery and get a tighter grip on their infrastructure and costs.

Indeed, the principles of DevOps have spilled beyond the “traditional” sphere of development, giving rise to neologisms such as GitOps, FinOps and AIOps. The DevOps market will be worth an estimated $14.9bn by 2026, according to research by Fortune Business Insights, cited in Spot by NetApp’s DevOps Revealed report.

But while it’s easy for organizations to see the value of both DevOps practices and DevOps-savvy personnel, getting those skills in-house and putting them to work is far more complex. In fact, according to research by CodinGame, demand for DevOps skills outstrips that for AI/Machine Learning/Deep Learning specialists, cloud computing, database software, and mobile development.

A Bain and Company report also showed that 90 percent of business leaders cited DevOps as a “top strategic priority” for their business. So it’s no surprise that even with recent economic uncertainty, competition for personnel with these DevOps skills is still fierce, with salaries heading steadily upwards. Puppet’s 2021 State of DevOps survey found that “more DevOps professionals moved to salaries between $150,000 and $250,000 than in any year since 2019.”

DevOps: A question of value

Yet while businesses understand the importance of DevOps and are willing to make significant investments to recruit talent, it can still be harder to nurture and make the best of it over the longer term. That certainly appears to be the opinion of DevOps pros themselves. Spot by NetApp’s research shows that nearly two-thirds (64 percent) of DevOps leaders “don’t always feel they are valued by the exec team.”

The reasons behind this disconnect are made clear in Spot’s DevOps Revealed research, which highlighted the key motivations of DevOps leaders. Designing how new features work is what really gets their juices flowing, with 84 percent of respondents saying this is what they find most enjoyable. Nearly two-thirds of respondents find writing code for those features most enjoyable, while 31 percent identify learning new tooling and practices as a key motivation. This is all good, forward-looking stuff.

But it’s what DevOps pros don’t enjoy that explains why many organizations are not getting all the benefits they hoped for. The figures are stark, with 85 percent of DevOps leaders saying that “managing infrastructure gets in the way of being able to deliver value to the business.”

This includes the burden of managing costs and of managing containers, with just one in ten DevOps teams finding these tasks “enjoyable.” The problem is that 88 percent of IT decision makers believe that it is important for DevOps to manage costs, while 89 percent of DevOps leaders state this aspect of the job is becoming more important.

That’s a dichotomy that has immediate implications for infrastructure provisioning and management, and DevOps teams are going to be less than enthusiastic about it.

As Rajh Das, Spot’s director of international sales, explains, “DevOps teams are generally not set up for success when it comes to cost management, due to both their time constraints and a lack of suitable tooling made available to them.” Things are further complicated because infrastructure management can be extremely variable.

Therefore, it’s easy to see why directing highly skilled professionals to cut costs rather than deliver value is a misuse of their skills and motivations. And with other organizations willing to pay a premium for their talents, there’s little incentive for disgruntled DevOps professionals to stick around.

Looked at another way, it’s clear that organizations are comparatively rigid in their use of cloud infrastructure. Just within the AWS world, there are hundreds of types of instances available, yet Spot by NetApp’s research shows that over 70 percent of DevOps teams use less than 50 of them.

Moreover, it also found that just 11 percent of DevOps leaders use spot instances, the “spare” capacity that cloud providers typically sell at a massive discount. If customers can find a way to live with the possibility that such instances can be pulled at just minutes’ notice, spot instances can offer cost savings of 90 percent over conventional instances.

Freeing DevOps pros from cost management

But many DevOps pros might perceive spot instances as too risky. The nature of spot instances gives rise to concerns about availability.

As we’ve seen, managing risk and infrastructure admin is not top of the list of DevOps pros’ favorite pastimes. It’s clear that many of the systems and processes DevOps teams have developed to provision infrastructure still include a large degree of time-consuming, error-prone, manual work.

Likewise, when it comes to the use of automation in the provisioning of containers, seen as central to the efficient and reliable delivery of apps across all clouds. Just one in 20 DevOps leaders said they had a container optimization strategy in place. Again, this can lead to unexpected complexity, undermining the efficiency and automation that should be at the heart of a modern, DevOps-powered application strategy.

Spot’s response to this is its Ocean service, which automates container management by providing a serverless infrastructure engine for containers. Ocean continuously analyzes how containers are using infrastructure and automatically scales compute resources to maximize utilization and availability through an optimal blend of spot, reserved and on-demand compute instances. This delivers “hands-free” infrastructure for cloud-native apps, while lowering costs by on average 70 percent.

The approach addresses one of the biggest problems DevOps pros face in automating their infrastructure — which is that it’s really hard to set up.

As Das explains, “Our study found 95 percent felt they do not have efficient container optimization in place. This is because setting up automation takes resources and skills, and it requires active release management with co-dependencies.”

This despite the fact that virtually all respondents to Spot’s research (99 percent) agreed that their organizations should be looking to use more automation to reduce or eliminate unnecessary work within DevOps.

That’s a large part of why companies are not gaining the full benefits from their expensive DevOps teams, while also overpaying for their infrastructure because it’s not being effectively managed.

The use of automation combined with machine learning could allow DevOps leaders to take advantage of the full range of instances and dramatically cut costs. But perhaps, more importantly, it will relieve DevOps pros of the burden of managing infrastructure and focusing on cost control, freeing them up to focus on delivering new applications, features and services.

And that is the point at which companies’ belief — and investment — in DevOps will truly start to deliver value.

Read the full DevOps Revealed Report here.