Key Metrics and Definitions

Metric: App downloads

  • Definition: The number of distinct users that click the download button or copy the brew install code snippet on the about or docs site.
  • Why this metric: Because there are multiple ways to install the app, this metric shows a consistent view of the intent to download and install the app.
  • Related metrics:
    • Downloads by type - Same definition with the ability to see if the download was linus, mac, or brew.
  • Source of truth: This data is logged by eventlogger, and accessed via Looker (see: “App downloaders” chart)

Metric: App DAU

  • Definition: The number of active users of the app each day
  • Why this metric: Tracking DAU over time show the engagement users have with the app
  • Source of truth: This data is logged by pings, and accessed via Looker (see: “DAU” chart)

Metric: App total repos

  • Definition: The number of repos synced in apps
  • Why this metric: Tracking repos over time show the engagement users have with the app
  • Source of truth: This data is logged by pings, and accessed via Looker (see: “Total repos” chart)

Metric: % active apps

  • Definition: The percentage of all apps whos users was active that day
  • Why this metric: Tracking % active apps over time show how engaged our app users are as our app base grows.
  • Source of truth: This data is logged by pings, and accessed via Looker (see: “% active apps)

Metric: Visitors

  • Definition: The number of unique hits to our website. Visitor numbers exclude users who used Sourcegraph via an extension only (since these users did not actually visit our website). Unless otherwise specified, visitor numbers include traffic to both the product and marketing sites.
  • Why this metric: Tracking quantitative information about visitors will help us measure how many people are aware of our product. Qualitative info about visitors will help us understand who our traffic is and how they found us, so we can assess what’s working.
  • Related metrics:
    • Referrer: the website that directed the visitor to our website
      • Original referrer: The first website that directed this user to our website
      • Session referrer: The website that directed this user to our website for this specific session
    • UTM parameters (ex source, medium, etc): the UTM parameters associated to the URL the user landed on
    • Other qualitative info such as new vs returning user, cohort date, and whether the user has a cloud account
  • Source of truth: This data is logged by eventlogger, and accessed via Looker (see: “Visitors” chart)

Metric: Leads (aka trial requests)

  • Definition: The number of unique cloud trial form submissions (regardless of whether or not they qualified for a trial)
  • Why this metric: This metric helps us understand if visitors are getting enough value from our product to want to try it on their company’s code
  • Related metrics
    • UTM parameters: the campaign that drove this lead to our website
    • Other qualitative info such as request data and current status (converted, disqulaified, etc)
  • Source of truth: This data is logged via Salesforce, and accessed via Looker (see: “Growth Marketing KPIs”, “All leads”)

Metric: Qualified leads (aka trial starts)

  • Definition: The number of leads who qualify for a trial instance.
  • Why this metric: This helps us understand if we’re driving qualified visitors to try our product
  • Source of truth: This data is logged in Salesforce, but should be accessed via Looker(see:“Growth Marketing KPIs”, “All leads”)

Metric: Visitor conversion rate

  • Definition: The ratio of leads to visitors (leads / visitors)
  • Why this metric: This helps us understand what percentage of our visitors are requesting a trial
  • Source of truth: Looker(see: “Growth Marketing KPIs”, “Visitor Conversion”)

Metric: Signup conversion rate

  • Definition: The ratio of leads to signup page visitors
  • Why this metric: This helps us understand what percentage of people who land on the signup page actually signup, which gives us insight into whether the value of the product/signing up is clear
  • Source of truth: Looker(see: “Signup page conversion”)

Metric: Lead qualification rate

  • Definition: The ratio of qualified leads to all leads (qualified leads / leads)
  • Why this metric: Helps us understand what percentage of leads are actually qualified, to give us an idea of whether or not we’re driving the right kind of visitors to signup
  • Sources of truth Looker(see: “Growth Marketing KPIs”, “Trial requests vs qualified trial starts”)

Key data sources

  • EventLogger: An in-house tracking tool we use to track visitors and events on our websites. EventLogger logs data that is sent to our data warehouse and accessed in places like Looker and BigQuery. Documentation for key events in the signup flow can be found below. If you’re interested in the name of an event not specified at link below, see here.
  • Salesforce: Salesforce is used to track which leads are qualified, the status of those leads, and whether or not an opportunity is associated to those leads. Any salesforce data that would be of interest to the growth marketing team should be available via Looker.
  • Plausible tracking: Plausible is a secondary tool we use to track visitors and referrers. While not considered a source of truth, we often use this for real-time analysis (Looker/Amplitude operate on a ~1-2hr delay). In addition, for more simple analyses, some find Plausible’s UI friendlier than our other BI tools.

Known Gaps/Roadmap

  • EventLogger audit/improvements
    • Problem: In the past we’ve mostly used eventlogger on the product site to track in-product activity, and not so much in other places. EventLogger tracking on the product/marketing site also hasn’t been the main priority of any team for some time. As a result, we’re not 100% confident we have all the required tracking on every sourcegraph webpage. With help from the growth marketing team, Analytics is conducting an audit to ensure we have all the right tracking in place.
    • Impacts:
      • We’re missing some data, or that data isn’t as accessible as we’d like
      • We have questions about how eventlogger is logging certain things, and want to ensure the data is as clean and reliable as possible,
    • Plan to fix:
  • Dispersed sources of truth for key metrics, definitions
    • Problem: We’re reporting in a number of places using a number of different tools. It’s not always easy to find what you’re looking for
    • Impacts:
      • It’s unclear where to go to get questions answered
      • We may not all be on the same page about what metrics mean
    • Plan:
      • [DONE] Document key metrics, how they are defined, sources of truth for each, in the handbook. Link to documentation in all reporting for visibility
      • [IN PROGRESS] Make data products (enterprise dashboards, ad hoc analysis) more intuitive/interoperable
  • Additional visualizations for data we didn’t have/didn’t track closely in the past
    • Problem: In the past, the analytics team has not worked extensively with marketing data/stakeholders. As a result, we don’t have a lot of data products tailored specifically to this audience.
    • Impacts:
      • Some high-level marketing metrics that would be useful to all marketing stakeholders (including growth marketing) are not available “out of the box”
    • Plan to fix:
      • [IN PROGRESS] Generate new data products that contain high-level marketing metrics so this data is easily accessible to all stakeholders. This is an ongoing effort, check out the analytics project board for work in flight for the current sprint. We’ll post major updates publicly in #growth.

Ongoing analytics initiatives for growth marketing

Other Resources