Measuring engineering velocity without falling for vanity metrics
"How fast is the team?" is a fair question with a lot of bad answers. Lines of code, commits, story points burned down — they're easy to measure and easy to game, and none of them tell you whether the product is actually moving.
Measuring velocity well matters more as teams adopt AI tooling, because the tools inflate exactly the numbers that were already misleading. Here's how to tell activity from progress.
What vanity metrics get wrong
- Lines of code rewards verbosity. The best change is often a deletion.
- Commit or PR count rewards splitting work into noise, not shipping value.
- Story points measure estimation, not outcomes — and drift the moment they're used as a target.
Goodhart's law does the rest: when a measure becomes a target, it stops being a good measure. Anything you put on a dashboard and reward, people will optimise — including in ways that hurt the product.
Signals that correlate with shipping
The DORA research popularised four that hold up across teams:
- Lead time for changes — how long from "code written" to "running in production." Short lead times mean small batches and fast feedback.
- Deployment frequency — how often you ship. Frequent, small releases are safer than rare, big ones.
- Change failure rate — how often a release needs a fix or rollback.
- Time to restore — how quickly you recover when something breaks.
To those, add one that teams feel every day: review latency — how long a PR waits for its first review. It's often the biggest hidden drag on flow.
Pair the numbers with a narrative
Metrics tell you where to look, never why. A spike in lead time might be a flaky pipeline, a gnarly feature, or someone out sick. Ask the team. The most useful review pairs a few trend lines with a short human explanation of what changed.
AI raises the stakes
When an assistant writes the boilerplate, output volume climbs — but volume was never the goal. Keep measuring outcomes (shipped, stable, fast to change) and resist the temptation to reward raw throughput, or you'll get more code and not more product.
Track a small set of outcome metrics, review them monthly against a narrative, and never turn a single number into a target.
A good starting set: lead time, change failure rate, and review latency. Three honest signals beat ten flattering ones.