Admin Experience Vision, Strategy, Execution and Metrics

Note: This page is a WIP and should not yet be treated as final.

Goal: Build on top of the Company-Wide VSEM document with a specific focus on the Admin user persona to define Vision, Strategy, Execution plan and Metrics for the Sourcegraph admin experience.

Admin Persona Overview

Before we define the vision, strategy, execution, and metrics, we should describe more about who we mean when we say ‘admin’. For the purposes of this document, we are describing three distinct types of admins.

Vision

Vision: The long-term (3–5-year) description of what we’re building and why. The vision will not change frequently, but it will be reviewed every 3 months to ensure it remains relevant and aligned with Sourcegraph’s mission and values.

Admin Experience Vision Statement:

We believe Sourcegraph’s admin experience has the ability to provide unique value to admins built on strong intelligence, adaptable configurations, and open systems. Admin systems are open meaning current and past state is clearly observable, debuggable, and create predictable outcomes. They are adaptable to ensure admins within the most complex organizations feel confident Sourcegraph is set up in a secure and reliable way. Sourcegraph makes the administration experience delightful by delivering unique admin-focused insights and intelligently remediating errors.

Strategy

Strategy: The strategy is Sourcegraph’s medium-term plan for how to achieve our vision. It highlights the key strategic directions we must execute against in order to make our vision a reality. The strategy will be revisited annually and will likely evolve over time, but the big picture should largely stay the same.

Draft 3-5 Year Strategy:

(We are here!) Stage 1:

Build scalable, enterprise grade core systems for Sourcegraph that support both our customers and other teams across Sourcegraph. In many ways, we know what’s missing, based on feedback, by looking at other great admin experiences, or from our own intuition, and stage 1 is about solving these. By doing this work, we will feel comfortable enabling platform-style capabilities to further unlock our community of users. A few ways we will do this include…

  • Increase the number of locations code Sourcegraph can talk to. This includes eliminating the experience drop-off between popular code hosts (GitHub) and less popular (such as Perforce).
  • Enterprise level permissioning and user management.
  • Ensure all systems are observable, debuggable, and measured with defined and monitored SLAs
  • The admin experience is frictionless and delightful, providing admins with information exactly where they expect it, and giving them the tools they need to scale their instance with the needs of their company
  • Ensure the above systems can scale to meet the needs of our largest customers.
  • Ensure our system remains easy to change, open for expansion to other code hosts, while meeting strict requirements for performance and scalability that other Sourcegraph teams have our APIs.
  • Build a clear interface between source code and features

Stage 2:

Once we have built the core of our systems, we will bring Sourcegraph code intelligence into the site-admin experience. This will help build admin trust in our intelligence and relieve admins of monotonous tasks. A few ways we will do this include…

  • Automatic identification and remediation of errors
  • Leverage intelligence to help admins fully configure their instance (i.e. recommend ways to deploy more precise code intel)
  • Truly predictable change management (i.e. Admins should reliably know how a change will impact the system before they make the change)
  • [… there are many other ways we can inject intelligence into admin experience]
  • Continue supporting/extending foundational systems as needed by customers or internal stakeholders

Stage 3:

Now that Admins trust our intelligence, we will launch targeted, value-add services to admins. Historically, the admin experience is something required to get core value out of Sourcegraph. The goal here is to turn the admin experience itself into something that customers would pay for. A few ways we will do this include…

  • Intelligent Permissions Monitoring (brainstorm here)
  • [other ideas coming soon…]
  • Continue supporting/extending foundational systems as needed by customers or internal stakeholders

2023 Execution Plan

*Execution: The execution plan is the process for how we will implement Sourcegraph’s strategy and highlight the necessary actions needed to achieve our vision. Each execution plan typically comprises a number of important projects and initiatives, critical decisions that need to be made, the allocation of resources, etc Teammates should use this to connect their day-to-day work to the company’s vision and strategy.

Currently, we are working on Stage 1 of the strategy defined in the previous section. We anticipate all of 2023 to be focused in Stage 1 with only small experiments in Stage 2. All major projects will follow the Job Fair prioritization process starting with a PRFAQ.

Note: Anything above the — line has already been greenlit and staffed. Things below are tentatively scheduled for 2023 but not yet greenlit or staffed.

Increase our support for git-based code hosts:

  • Tier 1 ADO (Job Fair Project)
  • Gerrit Permissions syncing (Job Fair Project) and batch changes (Job Fair Project TBD)
  • — Below this line is not yet approved/in-progress and could change.
  • Introduce support for GitHub’s fine-grain auth tokens (currently Beta, GA date TBD)
  • Notify customers of ending support for non-expiring GitLab PATs (Release 16.0 of GitLab happens on May 22, 2023)
  • Introduce GitHub App as a site-level code host (PRFAQ TBD)

Introduce first class support for non-git code hosts:

  • Launch a first-class Perforce integration (validated by TAM estimate here as well as Top-10 asks from GTM). Key existing product gaps include changelist and streams support.
  • — Below this line is not yet approved/in-progress and could change.
  • Fast follow first-class Perforce with first-class SVN (validated by TAM estimate here as well as Top-10 asks from GTM)

Build more for our ICP (Strategic customers):

  • Permissions should work efficiently at 2XL scale (Job Fair Project)
  • SCIM (Job Fair Project)
  • — Below this line is not yet approved/in-progress and could change.
  • Repository syncing at 2XL scale (Job Fair Project TBD)
  • Reduce the operational cost of running Sourcegraph (PRFAQ TBD) including deduping git objects across forks (GitHub Ticket, Job Fair Project TBD) and adding automated infra cost alerting to Sourcegraph
  • Enterprise level User management (Fast follow of SCIM to solve these gaps with a goal of ‘delightful User Management at 2XL scale’
  • Option to temporarily deactivate code host connection (From product gap to aid debugging purposes, PRFAQ TBD)
  • Permissions in parallel (need to validate this need further, RFC)

Increase the maturity of the admin experience systems:

  • Permissions should be debuggable, observable, etc (Job Fair Project)
  • Permissions syncing should be debuggable, observable, etc (Job Fair Project)
  • — Below this line is not yet approved/in-progress and could change.
  • Display SLAs/SLOs/SLIs to admins in the site-admin UI (PRFAQ TBD)
  • Health checking tooling to give customers confidence the deployment is successful (past SDD)
  • Revamp enterprise license generation and introduce proper upsell-based gating between packaging tiers (PRFAQ TBD)

Improve time-to-value for admins setting up a new instance (supporting growth team):

  • The admin experience onboarding process should build trust with admins while guiding first time admins through the process (Job Fair Project)
  • Improved self-hosting experience for first-time-deployments (Job Fair Project 1, and Job Fair Project 2)
  • Improved executor deployment/hosting (Executors in K8s, improved deployment docs, etc) (Job Fair Project)
  • — Anything not currently prioritized by our team should be owned by growth going forward

More importantly, we will NOT work on the below in 2023:

  • A major redesign of the admin experience. We will aim to slowly improve the experience over the next year as part of specific job fair projects but will not meaningfully redesign.
  • Radical improvements to in-product analytics. The spike in 2022 continues to show value to customers and has yet to generate any significant product gap blockers from customers. Feature teams will own adding new analytics if they desire (i.e. own)
  • We will not meaningfully solve the RBAC tech-debt in 2023 (Described in RFC 617). This is a significant effort that needs to be driven by end-user value. Today, this pain does not reach customers often.
  • Zero downtime deployments. We started moving to Rendezvous hashing in 2022 and have not heard meaningful customer requirements for zero downtime recently. We could revisit this in an effort to support the Cloud-First effort at Sourcegraph.
  • CVS support. This has been requested by several customers. In 2023, we will focus on Perforce and SVN as they have higher overall addressable markets. We will revisit this if mid-year we have a simpler way to integrate non-git code hosts.

2023 Key Metrics

Note: This section is a WIP while we finish the above sections. Below is a brainstorming of the various metrics that could matter to us.

Metrics: These are the measures or indicators that Sourcegraph will use to track our progress towards our goals. They should be closely tied to the vision and strategy and will be used to gauge the effectiveness of the execution of our plan.

New admin setup

  • Time to first search
  • % drop-off in the admin setup UI

Build for our ICP

  • We are confident Sourcegraph can support our first wall-to-wall expansion at a strategic customers
  • SLAs are met X% of the time

Improve our code host support (git and non-git)

  • Number of customers successfully deployed with Perforce >5
  • Number of customers successfully deployed with SVN >1
  • % of customers with >1 code host connected to Sourcegraph (increase by X%)

Operational Maturity

  • The number of P0/P1 issues decreases
  • The number of open product gaps on Repo/IAM decreases