13 · Top-k ranking metrics
Precision@k, recall@k, hit-rate, MAP, and NDCG each reward something different. NDCG cares not just whether the right item is in the list, but how high.
One idea, one try. At your own pace.
LEARN BY DOINGPut your intuition to work3 steps
13 · Top-k ranking metrics
Use the lesson’s examples and explanation to check your idea.
Personal reflection. Nothing is sent to an AI service.Top-k metrics turn a ranked list into a number. Precision@k and recall@k ask how many relevant items made the top k; hit-rate asks did ANY; MAP and NDCG go further and reward ranking the relevant items HIGH — NDCG discounting each hit by the log of its position.
Without this:
Optimize plain accuracy and you can't tell a list that buries the one good item at position 10 from one that puts it at position 1 — even though users only look at the top.
Once you hold out relevant items per user (last lesson), you score a recommender by how well its ranked top-k list recovers them. Each metric encodes a different question:
- Precision@k — of the k items I recommended, what fraction were relevant? Rewards not wasting slots. (k in the denominator.)
- Recall@k — of all the items that were relevant, what fraction did I surface in my top k? Rewards coverage of the truth. (relevant-count in the denominator.)
- Hit-rate@k — binary: did at least one relevant item land in the top k? The lenient leave-one-out favorite (with one held-out item, hit-rate@k = recall@k).
- MAP (Mean Average Precision) — averages precision each time you hit a relevant item as you walk down the list, then averages over users. Rewards putting relevant items early and clustering them.
- NDCG (Normalized Discounted Cumulative Gain) — sums a gain for each relevant item, discounted by
1 / log2(position + 1)so a hit at rank 1 is worth far more than at rank 10, then normalizes by the best-possible (ideal) ordering so the score lands in [0, 1].
The crucial divide: precision/recall/hit-rate are set metrics — they only care whether a relevant item is in the top k, not where. MAP and NDCG are rank-aware — moving a relevant item up the list, even within the top k, improves them. Since users overwhelmingly click near the top, rank-aware metrics correlate best with real engagement.
Below we implement all four from scratch and score two ranked lists against the same held-out relevant set. Then we move one relevant item from rank 4 up to rank 1 and watch NDCG (and MAP) rise while precision@k stays flat — the whole point of a position-discounted metric.
Python (in browser)
precision@k, recall@k, hit-rate, MAP and NDCG from scratch. List B ranks the same relevant items higher — set metrics stay flat, but MAP and NDCG climb.
Python runs entirely in your browser via Pyodide (~6 MB on first Run, cached after).
scikit-learn ships ndcg_score; full recsys frameworks (RecBole, Surprise) report precision/recall/NDCG/hit-rate over all held-out users for you.
You move a held-out relevant item from rank 4 up to rank 1 within your top-5 list. Which metric is GUARANTEED to improve?
- Precision@k, recall@k and hit-rate are SET metrics — they only care whether a relevant item is in the top k, not where.
- MAP and NDCG are RANK-AWARE: NDCG discounts each hit by 1/log2(rank+1), so higher positions score more.
- Report NDCG@k as the headline metric — it tracks real top-of-list engagement better than set metrics.
NDCG is the standard ranking metric in search and recsys leaderboards (MovieLens, Amazon datasets, the RecSys challenge); precision@k/recall@k headline most papers' results tables.
If you remove it: Without rank-aware metrics you optimize set membership and ship lists that technically contain the good item but bury it where nobody scrolls.