Choosing Product KPIs: The Metrics That Mislead
Key takeaway
Retention is the metric that validates a product; acquisition only tells you whether marketing is working. Define activation as the specific action that predicts retention for your product, measure it by cohort, and be suspicious of any metric that can only go up.
Dashboards full of growing numbers feel reassuring and frequently describe a product that isn't working. The discipline is choosing a small set of metrics that can get worse, and looking at them by cohort.
The metrics worth tracking
- Activation — the share of new users who reach the action that predicts they'll stay. You must identify that action from your own data, not copy someone else's.
- Retention by cohort — the percentage of each signup cohort still active after 1, 4, and 12 weeks. This is the clearest signal of whether you've built something people need.
- Engagement depth among retained users — frequency and breadth of use, which distinguishes habitual users from occasional ones.
- Time to value — how long from signup to the first useful outcome. Shortening it usually improves everything else.
- For commercial products: net revenue retention, which accounts for expansion as well as churn.
The metrics that mislead
- Cumulative totals — registered users, total downloads. They can only rise, so they can never tell you something is wrong.
- Page views and session duration without context. More time in the product can mean engagement or confusion.
- Averages that hide the distribution — a handful of power users routinely disguise a dead middle.
- Feature usage counted without asking whether the feature was on the default path.
- Net promoter score treated as a product metric rather than a conversational prompt.
Finding your activation event
Compare users who retained with users who churned, and look for the early action that separates them — inviting a teammate, importing data, completing a first project. Validate it by checking that users who take it retain at a meaningfully higher rate. Then make it the goal of onboarding. This one exercise usually reshapes a roadmap more than any feature request.
Keep the set small
Pick one primary metric that the team can influence this quarter, with two or three supporting ones and a guardrail metric that must not degrade. A dashboard with thirty numbers doesn't focus a team; it lets everyone find evidence for what they already wanted to do.