Case study · Data · Freelance platform · 2021–2023
One definition per metric: data contracts and a schema-change gate ended conflicting numbers and fixed every broken dashboard
Head of Freelance Platform · product side of data governance, with the CDO · 2021–2023 · Databricks, Amplitude, Looker
- 1definition per metric, across every team
- days → secondsto answer a metric question
- 8 → 0broken dashboards
Situation
80 people in PPC, Operations and Product made daily decisions from data. Product defined the events, backend built and changed them, and each team built its own reports.
Complication
Two teams reported different numbers for the same metric, and each defended its own definition. Answers took days to assemble, 8 dashboards were broken, and leadership meetings opened with “whose number is right?” instead of “what do we do?”.
Decision
Agreement before tooling, decision metrics first. A data catalogue such as Atlan, Collibra or Unity Catalog would have listed the conflicting definitions without resolving them. We wrote the definitions and the process ourselves, starting with the metrics leadership used to set budgets and priorities, then working outwards.
What I did
- 01
Resolved each conflict on evidence, not seniority. Both teams ran their calculations side by side on the same raw data. We agreed one canonical definition per metric with a single owner, and kept legitimate variants as separately named metrics, so neither team lost its view. The CDO broke ties.
- 02
Wrote the source of truth. A 130-event tracking plan and taxonomy, and 50+ written metric definitions, each with an owner, a formula and the decision it informs.
- 03
Made event specs contracts. Every event’s fields and types were agreed with backend in writing as a data contract, and could not change silently.
- 04
Put every change through a gate. A schema change started as a short spec, was reviewed by the event’s product owner, an analyst and backend, and was traced through lineage to every dashboard that depended on it. Breaking changes shipped only with a migration plan.
- 05
Rebuilt reporting on the agreed layer. 10+ self-serve Looker dashboards, used by 80 people in PPC, Ops and Product.
Exhibit
No schema change reached a dashboard without passing four checks
- Spec: Short written proposal for the change
- Review: Event owner (product), analyst and backend
- Lineage check: Every dependent dashboard and metric traced
- Ship: Non-breaking ships; breaking ships only with a migration plan
Impact
- One number per metric across every team; conflicting definitions eliminated.
- Metric questions answered in seconds instead of days; broken dashboards 8 → 0.
- 80 people in PPC, Operations and Product working from self-serve data.
- Every new event and metric followed the same contract and review, so the fix held as the platform grew.
Learning
Data quality is an agreement problem before it’s a tooling problem.