Recommendation Engines: Build, Buy, or Skip
Key takeaway
Below a few thousand active users or a sparse catalogue, hand-built rules — popular in category, frequently bought together, recently viewed — usually outperform a recommendation model and cost almost nothing. Build or buy a real engine when catalogue size and interaction volume make manual curation impossible.
Recommendations are one of the few machine learning features with a direct, measurable revenue link. They're also routinely built before there's enough interaction data for them to learn anything.
The main approaches
- Popularity and rules — best sellers, category bestsellers, frequently bought together. No model, surprisingly strong, and the baseline anything else must beat.
- Content-based — recommend items similar to what the user engaged with, using item attributes. Works on day one and handles new items well.
- Collaborative filtering — users who liked this also liked that. Strong once you have dense interaction data, and helpless with new users or new items.
- Hybrid — content-based for cold start, collaborative once there's history. What most production systems actually are.
Do you have enough data?
Collaborative approaches need many users interacting with many of the same items. A catalogue of 200 products and 500 monthly users will produce a model that recommends noise. Check the density of your interaction matrix before committing: if most items have a handful of interactions, content-based methods and rules are the honest answer for now.
The problems everyone meets
- Cold start for new users and new items, which is a permanent condition rather than a launch phase.
- Popularity bias — the model recommends what's already popular and suppresses the long tail it was meant to surface.
- Obvious recommendations: suggesting the item someone just bought, or an accessory for a product they returned.
- Business rules the model doesn't know — out of stock, not available in region, low margin, age-restricted. These must be applied as a filter after ranking.
Measure it against the baseline
- Run it as an A/B test against your current rules, not against nothing.
- Measure revenue per session and conversion, not click-through on the widget — clicks are easy to increase without selling more.
- Watch for cannibalisation: recommendations that merely redirect purchases the customer would have made anyway.
- Track coverage too — what proportion of the catalogue ever gets recommended tells you whether it's helping discovery.