The instructions in this document will guide you on the steps you need to take to migrate your Sourcegraph deployment from one Cluster to another. This is a process that involves the backing up of the Sourcegraph database by generating dumps and restoring the dumps in the newly deployed Sourcegraph instance on another cluster.
WARNING: Only core data will be backed up.
These instructions will only back up core data including user accounts, configuration, repository-metadata, etc. Other data will be regenerated automatically:
- Repositories will be re-cloned
- Search indexes will be rebuilt from scratch
The above may take a while if you have a lot of repositories. In the meantime, searches may be slow or return incomplete results. This process rarely takes longer than 6 hours and is usually much faster.
CAUTION: Backup and Restore must be done on Sourcegraph instances that are on the same version
- For example, if the Sourcegraph instance you take the database dump from is on v3.34.2 then the new instance must also be on v3.34.2.
NOTE: In some places you will see
-n $NAMESPACEto commands if you are not using the default namespace More kubectl configuration options can be found here: kubectl Cheat Sheet
- Backup Sourcegraph databases in the old cluster by generating dumps
- Restore the Dumps in the freshly deployed instance on another cluster.
In this section of the tutorial, you would learn how to back up the primary
sourcegraph database and the
Verify that the deployment is running.
kubectl get pods -A
Stop all connections to the database by removing the frontend deployment.
kubectl scale --replicas=0 deployment/sourcegraph-frontend
Generate the database dumps.
Dumps for the primary sourcegraph DB
kubectl exec -it $pgsql_POD_NAME -- bash -c 'pg_dump -C --username sg sg' > sourcegraph_db.out
Dumps for the codeintel DB
kubectl exec -it $codeintel-db_POD_NAME -- bash -c 'pg_dump -C --username sg sg' > codeintel_db.out
Ensure you copy/move the
codeintel_db.outfiles to the root of the
deploy-sourcegraphdirectory of the new Sourcegraph deployment.
These files are sensitive and should be stored/transported securely.
In this section of the tutorial, you would learn how to restore the primary
sourcegraph database and the
codeintel database from the generated dumps in the previous section.
Before restoring on the new deployment, It is very important you stop all connections to the Database from the frontend.
Scale down all the pods in the deployment
kubectl scale deployment --all --replicas=0
Scale down all the statefulset
kubectl scale sts --all --replicas=0
Then restart ONLY the
Start the pgsql:
kubectl scale deployment/pgsql --replicas=1
Start the codeintel-db:
kubectl scale deployment/codeintel-db --replicas=1
Copy the database files into the pods by running the following command from the root of the deploy-sourcegraph directory
kubectl cp sourcegraph_db.out $pgsql_POD_NAME:/tmp/sourcegraph_db.out
kubectl cp codeintel_db.out $codeintel-db_POD_NAME:/tmp/codeintel_db.out
Exec into the pgsql pod and then the template1 database:
kubectl exec -it <pgsql-pod-name> -- sh $ psql -U sg template1
Drop the sg database, recreate it, then exit the psql session (but still in the pod exec):
# DROP DATABASE sg; # CREATE DATABASE sg; # \q
Note: This is a process that we have to do to work around how Sourcegraph sets up the databases as at the time of writing this documentation. Our database pods automatically create the
sgdatabase on startup if it isn’t present. Our liveness probes also check for it which means it not being present can cause
k8sto kill the pod.
Restore the database dump:
psql -U sg -f /tmp/sourcegraph_db.out sg // exec into the pgsql-pod and run this command psql -U sg -f /tmp/codeintel_db.out sg // exec into codeintel_db-pod and run this command
Exec into the pgsql and run
SELECT * FROM REPO LIMIT 5;to be sure the restoration was complete.
kubectl apply --recursive -f base/to apply the changes.
kubectl get pods -o wideto make sure your pods are running again.
Start the remaining Sourcegraph services by following the steps in applying manifests.
Since your new deployment was running on a port before the restore, there is a probability that you might lose connection. Run
sudo lsof -i -P | grep LISTENto see the services using the port and kill the
kill -9 <PID_number>.
You can now port forward your k8s deployment to localhost or wherever your deployment is hosted. For example,
kubectl port-forward svc/sourcegraph-frontend 3080:30080(note you may need to add
-n $NAMESPACEto the command).