Continuous verification: Ocean CD metric and custom verifications

Continuous verification key concepts: Ocean CD metric and custom verifications

Reading Time: 4 minutes

Ocean CD is an advanced deployment controller for Kubernetes applications. It provides gradual deployment capabilities, continuous verification, and auto-rollbacks that prevent incidents following code changes. Ocean CD can be installed in minutes to help DevOps teams ensure safer code releases and allow developers to deliver applications faster and with greater confidence.

Ocean CD enables these objectives through three key pillars:

  1. Progressive deployment: The ability to gradually roll out a new code to production to prevent incidents and limit negative impacts, such as downtime
  2. Continuous verification: The ability to process data and trigger queries alongside the rollout in real time
  3. Automatic rollbacks: The ability to automatically reverse the rollout and revert to the previous version if your verifications do not appear exactly as you desire

These three pillars were carefully chosen and implemented upon field requests. Having all three of them wrapped up into one solution is significant and yet rarely found in the CI/CD landscape as a complete solution.

In this blog we will review the continuous deployment process Ocean CD uses with a focus on the second pillar, continuous verification, and how it can be easily configured with Ocean CD.

 

Metric verification

Ocean CD offers quick configuration of progressive deployment strategies that can be utilized by multiple Kubernetes workloads, across clusters. It provides a set of phases that define how a change is being rolled out. Each phase can include multiple metrics that will test the application performance after the code has been changed and affect the progress of the rollout.

As part of the deployment strategy, Ocean CD offers two ways to verify how new code behaves in the environment where it was deployed. The first is Metric Verification, which enables configuration of countless metrics with a set of queries and rules, using data providers like NewRelic, Prometheus, DataDog, and CloudWatch. The uniqueness of Ocean CD is that it processes the query requested in real time along with the rules configured to determine if the metric has failed or succeeded. With Ocean CD there is no need to use a monitoring tool in parallel since all the information is automatically presented in the Ocean CD console.

Canary inventory app CPU usage
Prometheus phase verification occurring during the second phase of a canary deployment

To integrate the above monitoring tools with Ocean CD, the user needs to create (through API, Ocean CD, CLI, or UI) both a Verification Provider Entity, which will grant Ocean CD access to all the monitoring tools using the required credentials, and a Verification Template, to configure the rules Ocean CD will act upon.

 

Custom verification

In case the user is not using any of the above-mentioned monitoring tools, Ocean CD offers another type of verification, which is called Custom Verification.

In Ocean CD, a Custom Verification can be one out of the following:

  1. Job analysis: Will trigger a Kubernetes Job at any given time during the rollout and configure a policy based on the received Job outcome (success or failure).
  2. Web analysis: Will perform an HTTP request against some external services to obtain a measurement. It may be used on various occasions, such as a trigger for external automation. Ocean CD will not only notify the external service but also get its measurements to display it and take automatic actions based on pre-configured policies.

This is an example of how a web analysis can be used:

kind: "VerificationTemplate"

name: "mywebanalysis"

metrics:

  - name: "webanalysis"

    interval: "10s"

    count: 10

    successCondition: "result.ok && result.successPercent >= 0.90"

    provider:

       web:

       url: https://myurl.com

       headers:

       - key: Authorization

         value: BearerToken

       jsonPath: "{$.data}"

The template above performs a GET request to the mywebhook URL, which in turn returns a full JSON. With the jsonPath parameter, Ocean CD displays only the request data from the JSON in question.

This verification will be implemented as part of a deployment strategy that can be used cross-cluster and over multiple workloads.

The UI would then return the following:

Canary Inventory app memory usage
A Web Analysis custom phase verification occurring during the second phase of a Canary deployment

 

How does the received data affect progressive deployments?

Upon success or failure of the metric verification or custom verification, Ocean CD follows the failure policy set prior to the deployment’s trigger. There are three failure options available to users:

  1. Rollback: Ocean CD rolls back the workload version upon verification metric failure. It returns to the previous stable version.
  2. Pause: Ocean CD pauses the rollout upon verification metric failure and allows the user to choose in real time how to pursue the rollout.
  3. Promote: Ocean CD promotes your rollout, regardless of the result of the verification.

For additional flexibility and granularity, the user can choose to deploy the code without any policy related to it. This can be done by enabling the Boolean parameter called Dry Run, found as part of the Verification Template entity.

 

Try Ocean CD for continuous verification

Through this blog, we have explained on the three pillars Ocean CD provides for users when deploying code for Kubernetes applications. We have mostly focused on the Continuous Verifications a user may configure and perform during the rollout of its deployment. Additionally, we showed how Ocean CD allows the setting of built-in verifications using well known monitoring tools, as well as described the way custom verifications may be used.

Ocean CD is now available with free trial to all Spot console users. We have templates and examples to quickly initiate your first deployment strategy. New to Spot? Schedule a demo to get started.