How repo-updater works


Sourcegraph mirrors repositories from code hosts. Code hosts may be SaaS products, such as GitHub or AWS CodeCommit, or local installations that are private to a customer’s environment. The repo-updater service schedules repository synchronization activities using gitserver and any configured code hosts.


A repo-updater instance exposes an HTTP server as its primary interface. This interface allows clients to schedule synchronization requests for the following:

  • Code host
  • Repository
  • Repository permission

Although the majority of Git operations are issued directly to gitserver, clones and fetches are routed through repo-updater to ensure that code host limits and other concerns are respected.

As noted earlier, there are a variety of code hosts that Sourcegraph can integrate with. The Source interface abstracts these code host communication details. For example, listing GitHub repositories is handled differently than listing GitLab repositories.

The service’s key data structure is a priority queue of repository updates. It implements the heap.Interface and the sort.Interface and functions in the following ways:


Production instances

There is exactly one instance of repo-updater running, by design. This allows us to:

  • Avoid expensive coordination issues
  • Respecting the aforementioned code host limits

General dependencies

Before repo-updater can begin accepting work, it needs to check that the following services are running and responsive to pings:

  1. frontend - implemented by the internal API client
  2. gitserver instances - implemented by the gitserver client

See “How gitserver works: Production instances” for more information.

Cloud-specific dependencies

If repo-updater is running in mode, it will verify that certain code hosts (specifically GitHub and GitLab) are properly configured. This is a requirement for us to be able to automatically add repositories from those code hosts when users browse to them.

Useful metrics

We track a variety of metrics in repo-updater that you’ll want to familiarize yourself with. For example: