Deployments

For a complete list of Sourcegraph instances we manage, see our instances documentation.

Additional resources:

Deployment basics

Changes to the main sourcegraph/sourcegraph repository are automatically built as images.

Images

Each Sourcegraph service is provided as a Docker image. Every commit to main in sourcegraph/sourcegraph pushes updated Docker images for all of our services to Docker Hub as part of our CI pipeline (i.e. if CI is green, then Docker images have been pushed).

For pushing custom images, refer to building Docker images for specific branches.

Renovate

Renovate is a tool for updating dependencies. deploy-sourcegraph-* repositories with Renovate configured check for updated Docker images about every hour. If it finds new Docker images then it opens and merges a PR (Sourcegraph.com example) to update the image tags in the deployment configuration. This is usually accompanied by a CI job that deploys the updated images to the appropriate deployment.

ArgoCD

ArgoCD is a continuous delivery tool for Kubernetes applications. Sourcegraph’s ArgoCD instance is available at argocd.sgdev.org.

ArgoCD currently handles deployments for k8s.sgdev.org.

Infrastructure

The cloud resources (including clusters, DNS configuration, etc.) on which are deployments run should be configured in the infrastructure repository, even though Kubernetes deployments are managed by various deploy-sourcegraph-* repositories. For information about how our infrastructure is organized, refer to Infrastructure.

deploy-sourcegraph

Renovate downstream master build status Dispatch update

Sourcegraph Kubernetes deployments typically start off as deploy-sourcegraph forks. Learn more about how we advise customers to deploy Sourcegraph in Kubernetes in our admin installation documentation.

There is automation in place to drive automatic updates for certain deployments from deploy-sourcegraph:

For documentation about developing deploy-sourcegraph and cutting releases, refer to the repository’s README.dev.md.

Merging changes from deploy-sourcegraph

We have two Sourcegraph Kubernetes cluster installations that we manage ourselves:

This section describes how to merge changes from deploy-sourcegraph (referred to as upstream) into deploy-sourcegraph-dot-com. The deploy-sourcegraph-dogfood-k8s-2 configuration is automatically updated with the latest deploy-sourcegraph changes.

The process is similar to the process we recommend our customers use to merge changes from upstream. The differences in process originate from the dual purpose of these two installations: they are genuine installations used by us and outside users (in the case of dot-com) and in addition to that they are test installations for new changes in code and in deployment. For that reason they are not pinned down to a version and the docker images are automatically updated to the latest builds.

Note: What is said about deploy-sourcegraph-dot-com also applies to deploy-sourcegraph-dogfood-k8s unless otherwise specified.

Relationship between deploy-sourcegraph repositories

  1. deploy-sourcegraph-dot-com@master strictly tracks the upstream https://github.com/sourcegraph/deploy-sourcegraph/tree/master.
  2. deploy-sourcegraph-dot-com@release only contains the customizations required to deploy/document sourcegraph.com from the base deployment of Sourcegraph.
    • This is the default branch for this repository, since all customizations to sourcegraph.com should be merged into this branch (like we tell our customers to).

These steps ensure that the diff between deploy-sourcegraph-dot-com and deploy-sourcegraph is as small as possible so that the changes are easy to review.

In order to mimic the same workflow that we tell our customers to follow:

Merging upstream deploy-sourcegraph into deploy-sourcegraph-dot-com

  1. Clone this repository and cd into it.
  2. If you have not already, git remote add upstream https://github.com/sourcegraph/deploy-sourcegraph
  3. git checkout master && git pull upstream/master
    • master of this repository strictly tracks master of deploy-sourcegraph, so there should be no merge conflicts.
    • If there are any merge conflicts in this step, you may git checkout master && git rev-parse HEAD && git reset --hard upstream/master && git push -f origin master which should always be 100% safe to do.
  4. git checkout release && git checkout -B merge_upstream to create a branch where you will perform the merge.
  5. git merge upstream/master to merge deploy-sourcegraph@master into merge_upstream
    • This will give you conflicts which you should address manually:
      • On docker image tags conflicts (image:), choose the insiders tag to allow renovate to deploy new builds.
      • On script conflicts (create-new-cluster.sh, kubectl-apply-all.sh, etc.), look for comments like This is a custom script for dot-com that indicate you should choose the current state over incoming changes.
    • If new services have been added (these generally show up as created files in base/), make sure that:
      • namespace: prod is applied to all new resource metadata.
    • Use kubectl apply --dry-run --validate --recursive -f base/ to validate your work.
    • Before you commit, ensure the commit message indicates which files had conflicts for reviewers to look at.
      • Using the default merge commit message, you can copy or uncomment the lines following Conflicts.
  6. Send a PR to this repository for merging merge_upstream into release.
  7. Reviewers should review:
    • The conflicting files.
    • If there are any risks associated with merging that should be watched out for / addressed, such as documented manual migrations that will need to be performed as part of merging the PR.
  8. If there are any manual migrations needed, coordinate with the distribution team and apply those first.
    • For example, new services that require elevated permissions might not be deployed by Buildkite - this must be done manually.
  9. Once approved, squash merge your PR so it can be easily reverted if needed.
    • In general, it might be a good idea to avoid doing this at the end of a PDT workday - if something goes wrong, it is easier to get help if other people are around.
  10. Watch CI, which will deploy the change automatically.
  11. Check the deployment is healthy afterwards (kubectl get pods should show all healthy, searching should work).