Using Cody Gateway

Provisioning access

Access to the production Cody Gateway instance can be provisioned with the following steps:

1. On sourcegraph.com, go to Site admin > Enterprise subscriptions
2. Find and open a subscription of interest
   1. If you have the license key on hand, you can also use the license key lookup tool.
3. Under “Cody services”:
   1. Enable access to Cody Gateway
   2. If desired, configure custom rate limits to allow more/less usage
   3. Customers can only use the models listed under “Cody services”. To configure other models:
      1. Adding the gpt tag to the latest license will enable access to all OpenAI GPT models. It is not required, this is only a convenience mechanism - overrides will always take precedence.
      2. Adding a custom rate limit (override) allows you to configure other models to allow the customer to use

Once access is provisioned (i.e. enabled via Cody Gateway), the Sourcegraph instance must be configured as well.

Configuration

First, provision access for the customer. Once access has been provisioned, for self-hosted customers please point them to the customer-facing Sourcegraph Cody Gateway docs to enable Cody and self-serve configuration for completions and embeddings.

Analyzing usage

A brief summary of Cody Gateway usage is available in each Enterprise subscriptions’s “Cody services” section:

1. On sourcegraph.com, go to Site admin > Enterprise subscriptions
2. Find and open a subscription of interest
   1. If you have the license key on hand, you can also use the license key lookup tool.
3. View data in “Cody services” section

Usage data is collected on a variety of events going through Cody Gateway, which is then sent to BigQuery. For more advanced use cases, see Usage events for more details.

Migrating from Anthropic/OpenAI completions

Customers using another provider for completions can be migrated to Cody Gateway by first provisioning access, then configuring sourcegraph as the completions provider and replacing accessToken with the provisioned Enterprise subscription access token.

Models (completions.chatModel, completions.completionModel, embeddings.model, and so on) should be updated such that they are prefixed with the previous provider, either anthropic/ or openai/. For example, "chatModel": "anthropic/claude-v1" should be updated to "chatModel": "anthropic/claude-v1".

A full migration example:

{
  "cody.enabled": true,
  "completions": {
    "provider": "anthropic",
    "accessToken": "REDACTED",
    "chatModel": "claude-v1",
    "completionModel": "claude-instant-v1"
  },
  "embeddings": {
    "provider": "openai",
    "accessToken": "REDACTED",
    "model": "text-embedding-ada-002",
    "dimensions": 1536
  }
}

{
  "cody.enabled": true,
  "completions": {
    "provider": "sourcegraph",
    "chatModel": "anthropic/claude-v1",
    "completionModel": "anthropic/claude-instant-v1"
  },
  "embeddings": {
    "provider": "sourcegraph",
    "model": "openai/text-embedding-ada-002",
    "dimensions": 1536
  }
}

Adjusting sourcegraph.com user quota

If you want to adjust quota for yourself / someone else, and you have Site Admin priviledges on sourcegraph.com, do the following:

1. Got to Quotas page for the user that you want to modify
   1. Located in Settings -> Quotas (for yourself)
   2. User list -> find the user you want to modify -> click on their username -> click Settings tab -> click Quotas
2. Adjust the right quota - Custom completions quota for Chat, Custom code completions quota for code completions and Click save.

Changes should be effective immediately (next Cody request).