Understanding GitOps: Principles, Workflow, and Deployment Types
What Is GitOps?
GitOps is a set of practices for managing infrastructure and application configuration. It leverages Git, an open source version control system that is commonly used in software development projects.
GitOps builds on infrastructure as code (IaC) technologies, which can take a configuration file specifying a desired state of the infrastructure, and automatically bring the infrastructure to that state. This is called declarative configuration. With GitOps, the Git repository becomes a source of truth for all declarative configurations.
GitOps automatically manages infrastructure provisioning and deployment using Git pull requests. Git repositories contain the entire system state, so you can view and review the tracking of system state changes.
GitOps builds on the developer's experience and empowers teams to manage their infrastructure using the same tools and processes as software development. It is not only based on Git, but also integrates with other technologies in the continuous integration / continuous delivery (CI/CD) and cloud native toolset.
In this article:
- 3 Principles of GitOps
- GitOps Workflow
- Why Use GitOps?
- GitOps vs DevOps
- GitOps Deployment Strategies
- GitOps Best Practices
- GitOps with Spot.io
3 Principles of GitOps
Here are three main principles of the GitOps pattern.
Declarative System
In the GitOps model, the entire system is configured declaratively. This declarative approach focuses on the outcome (the desired state) rather than the steps required to achieve the desired outcome.
This declarative approach allows the user to specify the ultimate goal without having to worry about all the necessary steps—known as an imperative approach. GitOps creates state-aware declarative configurations, allowing users to store these states in Git for easy deployment and rollback.
System State Captured in a Git Repository
All declarative states are stored in a Git repository as the only source of truth. This versioning method makes all system infrastructure changes available in chronological order, allowing users to easily identify infrastructure changes over time. This makes it much easier to perform rollbacks, troubleshooting of failed deployments, and auditing for compliance and security.
Automatic Deployment
When a pull or merge request is made, all changes are stored in Git, so they are verified and approved. In a GitOps model, it is best to immediately and automatically deploy any approved changes. This allows the system to achieve the desired state quickly. It also ensures that the current version of Git configuration matches the actual infrastructure at all times.
GitOps Workflow
GitOps is based on pull requests. When a pull request is initialized, users can see an overview of changes in each repository-based branch. Users can then add a summary of proposed changes, view proposed changes, add tags, and mention other contributors.
After developers create a pull request, they can add a commit in the topic branch. This allows all contributors to see the actual proposed changes. Once everyone approves the changes, they can be merged with the pull request.
Which configuration changes can be included in a pull request?
A GotOps pull request can include configuration changes such as:
- Changes to application configuration
- Changes to container images
- Changes to Kubernetes cluster configuration
- Fixes to errors in an environment
- Defining new infrastructure via declarative configuration
- Updating an environment to new requirements
Troubleshooting
After a pull request is approved and the changes are deployed, if there are any problems, troubleshooting is simple. With GitOps, any problem in an environment can be tracked to specific pull requests, and then troubleshooting can focus on the changes introduced in those pull requests.
Integration with other systems
GitOps can also use other tools for Git push, development, and continuous integration:
- GitOps can work with any CI and Git server.
- GitOps can work with CI/CD tools built for a Kubernetes environment, such as Ocean CD and Jenkins X.
- GitOps can use any Git repository, such as GitHub, Bitbucket, Azure Pipelines, or AWS CodeDeploy.
Why Use GitOps?
Organizations adopting GitOps are looking to be on the cutting edge of agile development practices. GitOps provides the following benefits over existing development processes:
- Productivity—GitOps is a practical way to implement continuous deployment, which improves development productivity. GitOps teams can deploy every change to production instead of only pushing selected releases several times a day. This faster feedback loop can dramatically improve development velocity.
- Developer experience—modern CI/CD pipelines are complex, with a variety of tools that not all developers are familiar with. With the advent of Kubernetes, environments have gotten much more complex. However, all developers are familiar with Git source control. So a GitOps model enables much faster ramp up and a smoother experience, based on concepts all developers are familiar with.
- Reliability—Git forks and reverts allow teams to perform easy, reproducible rollbacks if a problem is found in an environment. The Git repository also provides a single source of truth that allows recovery from a major problem to a stable state within minutes.
- Compliance and security—GitOps makes it easier to meet compliance requirements, because it automatically creates an audit log of all Kubernetes cluster changes. This makes it possible to easily investigate issues, and prove to auditors that security measures are in place. In addition, GitOps provides strong security guarantees, and the ability to sign all changes to prove authorship and origin.
- Consistency—GitOps makes Kubernetes consistent with development occurring in other environments. It provides one model for change management across traditional and cloud-native environments.
GitOps vs DevOps
GitOps and DevOps approach the development process from two different perspectives:
- DevOps is a pipeline process, which mainly focuses on the operational aspects of software development. It is used both by developers and operations professionals.
- GitOps is a development mechanism, which mainly focuses on automating and tracking environment changes in a declarative manner. It is developer-centric.
Here are a few more aspects that set DevOps and GitOps apart.
Imperative and declarative configurations
- DevOps can be both imperative and declarative. Many DevOps tools perform explicit scripting of deployment operations, taking an imperative approach. At the same time, DevOps can equally be applied to containerized and declarative configuration environments.
- GitOps only allows declarative configuration.
Containerized environments
- DevOps is most commonly used with traditional IT infrastructure such as bare metal servers and virtual machines (VMs). It can work with containers and Kubernetes, but without GitOps, can be challenging to manage.
- GitOps is almost always used in cloud native and containerized environments.
Monolithic vs. microservices
- DevOps is more suited to monolithic applications, or applications with a limited number of components.
- GitOps is best for large microservices applications with dozens or hundreds of components.
Can they work together?
- DevOps can accommodate and benefit from GitOps development workflows
- For organizations solely focused on Kubernetes as their infrastructure, GitOps may be enough and may not require additional DevOps tooling and processes.
GitOps Deployment Strategies
Deployment strategies are techniques developers can use to modify or update applications with minimal downtime. GitOps makes it much easier to implement deployment strategies in a consistent manner. Here are some of the most common deployment strategies.
Rolling Strategy
A rolling deployment replaces instances of an application with a new version gradually. It updates some pods, performs a readiness check, and when they are confirmed to be working, shuts down older instances. This means that in case of major issues, a rolling update can be aborted, new instances are terminated and replaced by the current version.
Canary Deployments
A canary deployment is a type of rolling strategy. It involves introducing a new version for a small subset of application users and observing their behavior, for a period of time that can span from minutes to several weeks. If users respond well to the change, it is rolled out to additional users, until the new version replaces the old.
Related content: Read our guide to canary deployments (coming soon)
Blue-Green Deployment
In a blue-green deployment, developers run two versions of an application in parallel. Using a load balancer or service routing, they gradually transition traffic from the current production version (green) to the new version (blue). At this point it is still possible to rollback to the old version by rerouting traffic from blue to green.
When the blue version is confirmed to be working well, the production route is reconfigured to point to it, and it becomes the new production version. The old green deployment is then shut down.
A/B Deployment
An A/B deployment is similar to a blue-green deployment, in that it also runs two versions of the application in parallel. However, unlike blue-green, it defines a percentage of traffic that should go to each version—for example, 50/50 or 20/80—and diverts traffic based on this rule. Over time, when there is more confidence in the new version, the percentage of traffic diverted to it can be increased.
This method is often used to experiment with different versions of the user experience. In other words, both versions run the same back end but with a small variation in the user interface, and development teams can observe differences in metrics like user engagement and conversion.
GitOps Best Practices
Here are a few best practices you can use to improve your GitOps implementation.
- Plan branching strategies—remember that the branching strategy you use in your source control repository has a direct effect on your environments. You must carefully plan source control branches according to the taxonomy of environments you need to set up.
- Avoid mixed environments—GitOps is “all or nothing”. For it to be effective, all parts of an environment must be controlled by infrastructure as code (IaC) tools and all the configuration must be checked into the repository. Running environments where some components are managed via GitOps, and other components manually or via other automated techniques, will have unpredictable results.
- Leverage the merge request discussion—when merge requests are submitted, there is a great opportunity for developers to discuss the implication of changes. Establish an effective merge request procedure, with appropriate automation patterns, and established protocols of communication, specifying what developers should ask during a pull request and what types of exchanges are appropriate for earlier planning stages.
- Respond when something breaks upstream—as soon as something breaks in an upstream environment deployed by GitOps, this indicates something was wrong with the configuration used to build that environment. Set up monitoring and triggers to obtain these signals from upstream environments. When something breaks, this should be treated as an Andon cord that stops work on the GitOps configuration until the problem is resolved.
- Policy as Code—GitOps can and should be complemented by automated policies that align with organizational policies. Open Policy Agent (OPA) is one framework you can use to achieve this. Policy automation can be especially important to ensure deployed environments are aligned with security and compliance requirements.
- Idempotency—no matter what you started with or how many times you ran a declarative configuration, the result should always be the same. Ensure you work with template engines and deployment systems that guarantee idempotency.
