Creating a managed instance

Creating a new managed instance involves following the steps below. For basic operations like accessing an instance for these steps, see managed instances operations what if there is some text here.

  1. CE creates an issue with the managed instance template in the sourcegraph/customer repository.

  2. Clone and cd deploy-sourcegraph-managed/

  3. Prepare your environment:

    • export MG_DEPLOY_SOURCEGRAPH_MANAGED_PATH=$(pwd)
    • export VERSION=v<MAJOR.MINOR.PATCH>
    • export COMPANY=$COMPANY
    • export PROJECT_PREFIX=sourcegraph-managed (should match GCP project prefix)
    • export PROJECT_ID=$PROJECT_PREFIX-$COMPANY
    • export PROJECT_SLUG=<prefix>.sourcegraph.com (the customer issue mentions it)
    • export TF_VAR_opsgenie_webhook=<OpsGenie Webhook value>
  4. Check out a new branch: git checkout -b $COMPANY/create-instance

  5. Create a new GCP project for the instance by:

  • Adding it to the managed_projects tfvar in gcp/projects/terraform.tfvars
  • Apply terraform in gcp/projects folder - due to the amount of service APIs that are defined in this project, run Terraform with increased parallelism to prevent waiting a long time for the plan to form: terraform apply -parallelism=100
  1. Ensure the target version of docker-compose file is in the golden directory, it should be named docker-compose.x.y.z.yaml

  2. ./util/create-managed-instance-new.sh $COMPANY and commit the result. Make sure that the version exists in deploy-sourcegraph-docker.

  3. Replace base docker-compose.yaml to use golden symlink: cd $COMPANY/red/docker-compose && rm docker-compose.yaml && ln -s ../../../golden/docker-compose.<MAJOR.MINOR.PATCH>.yaml docker-compose.yaml && rm ../VERSION && cd ../../

  4. Open and edit terraform.tfvars according to the TODO comments within and commit the result.

    NOTE: ⚠️ Do not set enable_alerting to true yet as this will cause false alerts to fire until the MI creation process has been completed!

  5. Open and edit deploy-sourcegraph-managed/$COMPANY/red/docker-compose/docker-compose.override.yaml, increase gitserver-0’s cpus: 8 if the instance size is larger than “n2-standard-8”.

  6. In deploy-sourcegraph-managed/$COMPANY run terraform init && terraform apply

  7. Check if all is running

    mg check
    
  8. In the $COMPANY directory run terraform fmt and commit any changes

  9. In the $COMPANY GCP project, create Google Oauth credentials with the following parameters:

    • type: Web Application
    • name: managed-instance-$COMPANY
    • Authorized redirect URIs:
      • Is the instance public, then add only https://$PROJECT_SLUG/.auth/callback
      • Is the instance private, then add both https://$PROJECT_SLUG/.auth/callback AND http://localhost/.auth/callback
  10. Create a GCS secret to be used by OIDC authentication

    mg create-oidc-secret --client-id=<CLIENT_ID_FROM_OAUTH_CREDENTIALS> --client-secret=<CLIENT_SECRET_FROM_OAUTH_CREDENTIALS>
    
  11. Initialize the instance and copy the password reset link that is generated by running

    NOTE: if requested in the new MI request issue, add the CE email in config.yaml:

    additionalAdmins:
    - CE_NAME@sourcegraph.com
    
    mg init-instance $CUSTOMER_ADMIN_EMAIL
    
  12. Go back to terraform.tfvars and set enable_alerting to true. Run terraform apply and verify that only google_monitoring_alert_policy.primary is created.

  13. Commit all changes

  14. Enable metrics collection and GCP alerts for the new instance:

    • cd monitoring
    • add new map entry to monitored_projects in variables.tf
    • run terraform apply
  15. Enable executors, learn more.

  16. Commit the last changes, create a PR for review, apply and merge

  17. Enable security audit logging via terraform apply in infrastructure repository - this will create required resources dynamically, based on project label.

  18. Add an entry for the customer by adding their accounts link to the checklist in the managed instances upgrade issue template.

Giving the customer access

To provide the customer access to the instance:

  1. If IP restrictions are requested, create and apply the Terraform change that grants their IP/CIDR ranges access to the instance, or makes it public/SSO-only, by following the operations guide.
  2. Copy the generated link and provide it to the CE to provide to the customer. Managed instances usually won’t have email set up, so a link will not be sent automatically. Inform the CE this link will expire after 24 hours. If the link expires before the customer was able to sign up, you can generate a new link with
    mg reset-customer-password --email <customer admin email>
    
  3. Ask the CE to add 10 extra seats to the license, as we currently do not exclude DevOps admin accounts from the license usage.

Configuring License, SSO, and repositories

Delivery usually hands off to CE at this point, they will schedule a call with the customer (including a DevOps team member, if needed) to walk the site admin on the customer’s side through performing initial setup of the product including adding the license key, adding repos, configuring SSO, and inviting users.

Is customers using a private code host?

If the customers’ code host is behind a firewall, we will need to provide them the IP of our Cloud NAT

cd $CUSTOMER
terraform output

Provide the value of cloud_nat_ips to CE or customers, and instruct them to allow incoming traffic from referenced IP addresses.

Troubleshooting

KeyRing already exists

Error: Error creating KeyRing: googleapi: Error 409: KeyRing projects/sourcegraph-managed-$CUSTOMER/locations/global/keyRings/primary-key-ring already exists.

You may be trying to re-create an instance in the same project where KMS is only scheduled for deletion instead of being deleted right away. You will need to manuall import below resources

terraform import 'module.managed_instance.google_kms_key_ring.keyring' projects/sourcegraph-managed-$COMPANY/locations/global/keyRings/primary-key-ring
terraform import 'module.managed_instance.google_kms_crypto_key.key' global/primary-key-ring/primary-key

Error creating Brand: googleapi: Error 409: Requested entity already exists

You may be trying to re-create an instance in an existing project. Simply import the resource.

terraform import module.managed_instance.google_iap_brand.project_brand $(gcloud alpha iap oauth-brands list --project $PROJECT_ID --format json | jq -r '.[0].name')