Managed SMTP

Managed SMTP allows Sourcegraph Cloud instances to have email delivery services configured out-of-the-box, and is enabled on all Sourcegraph Cloud instances by default. SMTP services are currently backed by SparkPost EU.

Customer-facing documentation is available here.

In this document:

Enabling managed SMTP for a Cloud instance - links to guides
Vendor management - management details for the SMTP vendor
Monitoring - monitoring SMTP capabilities
Vendor integration - high-level overview of how the integration with the vendor works

Enabling managed SMTP for a Cloud instance

Generally, managed SMTP will be enabled by default.

To manually enable SMTP, enforce the smtp invariant with:

mi2 instance check -enforce smtp

Disabling managed SMTP for a Cloud instance

A customer should reach out to the account representative to have managed SMTP disabled.

Sourcegraph engineers can disable SMTP by setting the .spec.managedSMTP.disabled to true in the instance’s config.yaml.

Vendor management

Account
- Okta and Entitle: Accounts are provisioned via Okta SSO using Entitle. Accounts should generally only be granted only Reporting access by default. Developer access accounts can be granted through Entitle for development purposes.
Billing: Airbase Virtual Corporate Card
- We are billed based on emails delivered according to our usage plan, included currently up to 250,000 (as of Dec 2022), after which we are billed for overages.
API Keys: list - see vendor integration for more details

Monitoring

Application-side

Alerting: frontend: email_delivery_failures
Dashboards: Frontend: Email delivery
Multi-instance dashboard: Frontend: Total emails successfully delivered every 5 minutes

Vendor-side

Dashboards are available in the SparkPost web application. Relevant metrics to look at:

Accepted count: we are billed for this based on our usage plans, up to 250,000 (as of Dec 2022).
Bounce rates: this indicates emails that are being rejected by recipients due to misconfiguration (invalid recipients) or spam filters.
- Note that the Sourcegraph application and features should be responsible for ensuring recipients are valid, for example via confirmation emails.
- High bounce rates at high email volumes may impact our sender reputation. We may be able to reduce the impact of an customer’s email delivery patterns by:
  - Disabling managed SMTP for the account.
  - Provisioning a custom sending domain or dedicated IP pool.

No alerting is available yet - see future work: vendor-side alerting.

Vendor integration

All integrations will send emails from the same sending domain, @cloud.sourcegraph.com, and default IP pool provided by SparkPost.

Operations are managed with the API key named managed-smtp-operator. This is available in sourcegraph-secrets GSM under SPARKPOST_API_KEY, and the account’s 1Password entry. It only has the following permissions:

Metrics: Read-only
Subaccounts: Read/Write
Sending Domains: Read/Write

For each Cloud instance, we create the following in Sparkpost:

A subaccount corresponding to the instance
An API key named send_key associated with the subaccount, with only 1 permission: Send via SMTP

The Cloud instance is then configured with the following:

A default sender address, noreply+$CUSTOMER@cloud.sourcegraph.com, and SMTP configuration
A GSM entry in the instance project to store the send_key

Security

SparkPost is a SOC2-certified provider, and leverages a variety of security practices that our security team has deemed satisfactory - for more details, refer to the SparkPost DPA.

Additionally, SparkPost has limited retention for most data - see its GDPR compliance documentation and events data retention (10 days). Support for data privacy requests is also available.

Email data access

Reporting access (also see vendor management) via the web application and the message events API can be used to access:

Aggregate numbers (deliveries, bounces, etc)
Specific email events, including receipients, subjects, message sizes, and other diagnostic data (but not email contents)

Account isolation

Our vendor integration is designed such that all usage and access can be constrolled on a per-subaccount (i.e. per-customer) basis. API tokens distributed to Cloud instances are scoped to individual subaccounts with very limited permissions, and can be disabled individually.

Employee Access Provisioning

Sourcegraph Employee access to SparkPost is exclusively granted through Entitle on a per-use basis. By default, accounts should be granted Reporting level access. For development purposes, Employees can request Developer level access.

To begin an access request, open Slack and type /access_request. Complete the dialog as follows:

Entitle Request Form

A member of Security or Cloud will then approve the request. Once approved, users can log in at https://app.eu.sparkpost.com/auth or by clicking the SparkPost icon on the Okta applications dashboard.

Future features

These are unimplemented, but are noted here as potential starting points for accomodating additional features and/or requests.

Vendor-side alerting

Vendor-side alerting can be set up by spinning up a service that can receive and process SparkPost webhook events, or a cron job to query deliverability metrics (also see monitoring).

Data privacy requests

We can list recipients and delete all data associated with the SparkPost data privacy API. Note that SparkPost has limited retention and access for most data - see security.

Custom sending domains

Additional sending domains can be provisioned following this guide. These domains can be assigned to specific subaccounts, which can be configured to use them as needed.

Dedicated IP pools

Very high-volume senders can impact the default sender’s deliverability (see monitoring). They can be created following this guide (note caveats such as reputation warm-up), and subaccounts can be configured to use them as needed.