GitOps

Three pillars of GitOps

  • Pipelines
  • Observability
  • Control

##Joined Up Pipelines

  • CI
  • CD
  • Release Automation Git as source of truth

Service Deployments are controlled using operator pattern in jubernetes

Kubernetes pattern - Git config -> Deploy Operator > service/deployments

Similarly now in gitops

  • config is code and everything is config
  • code and config must be version controlled

kubectl get <object> -o yaml --export to extract existing configuration to bootstrap config repo in git Sometimes export is not perfect. So review it properly

Keeping secrets in git?

  • Sealed secrets - Bitnami . Securely automate and share secrets publicly. Asymmetric (public key) cryptography
  • Encrypt data up to and inside k8s cluster
  • uses same operator pattern to decrypt the key

Structure of a GitOps repository

  • 1 repository per application/service
  • Use a separate branch per environment (maps to a kubernetes namespace, or cluster)
  • Push changes such as image nmae, healthchecks, etc to staging(or feature) branches first
  • Roll out to production involves a merge change from staging to production branch. (use git merge -s ours branchname to skip a set of staging-onl changes). merge the change but not applied. So if next person merges his changes to branchname then he doesnot get our changes when he does a pull
  • User protected branches to enfore code reviews

Example of GitOps Pipeline

  1. code change-> vcs -> Build Container CI -> Container Registry -> Configu Updater
  2. Merge staging to prod -> Git config -> Deploy Operator in kubernetes cluster See if weaveflux is interesting as kubernetes operator

Observability

  • Monitoring
  • Logging
  • Tracing & Visualisations

Observability is like TDD for production. Never accept a PR if you dont know how to identify if it is working.

In Gitops, every service should have a unified interactive dashboard. Eg. error rates along with deployment info if a new deployment occurred. Holistic view on observability

Control

counter part of observability

  1. Everything via Git (updates, policy, security)
  2. Orchestration
  3. Diff & Sync to alert dev from desired vs observed state

eg. get a prod-alert kubediff(Firing) kubernetes config in Git differs from Reality on cluster.

GitOps service Lifecycle release driven

ROODA -> Release -> Observe -> Orient -> Decide -> Act kubernetes -> Monitoring/Tracing/Logging -> Control & Operations -> Git -> Kubernetes

“Only What can be described and observed can be automated and controlled and accelerated”

3 Core principles of GitOps

  • Use declarative configuration to define your application and services
  • All changes need to go through git review process
  • User an operator in the cluster to drive the observed cluster state to the desired state as declared in configuration in git

GitOps Basic

  • Entire system is described declaratively
  • Canonical desired system state is versioned (DRY)
  • Approved changes to the desired state are automatically applied to the system
  • Software agents ensure correctness and alert on divergence

What should be gitsopsed

  • Kubernetes manifests
  • Application Configuration
  • Provisioning Scripts
  • Dashboards
  • Alerts
  • Playbook
  • Application Checklists
  • Recording Rules
  • Sealed Secrets