Scaling a Web Application: What Breaks First
Key takeaway
The database breaks first, almost every time — usually through missing indexes and N+1 query patterns rather than raw capacity. Fix queries, add caching, move slow work to background jobs, and only then consider horizontal scaling or splitting the application.
Scaling problems arrive in a reliable order, which is useful: it means you can fix them in that order rather than pre-building for a scale you may never reach.
1. The database, and it isn't close
Turn on slow query logging before you do anything else. In most applications, fewer than ten queries account for the majority of database time.
- Missing indexes on columns used for filtering and joining — the single most common cause of a slow application.
- N+1 queries from ORMs — one query becomes 500 because a loop lazy-loads a relation.
- Unbounded result sets: queries with no pagination that were fine at 1,000 rows and fatal at 1,000,000.
- Connection pool exhaustion, which presents as a mysterious total outage rather than gradual slowness.
2. Work done in the request that shouldn't be
Sending email, generating PDFs, calling third-party APIs, and processing uploads inside a web request ties up a worker and couples your availability to someone else's. Move them to a background queue with retries. This is usually the second-largest win and rarely requires architectural change.
3. Missing cache layers
- A CDN for static assets and cacheable pages — often the cheapest performance improvement available.
- Application-level caching for expensive computed results, with a deliberate invalidation strategy.
- Query result caching for hot, rarely-changing reference data.
- HTTP caching headers, which many APIs simply never set.
4. Only now, horizontal scaling
- Make the application stateless — sessions in a shared store, uploads in object storage, nothing important on local disk.
- Add read replicas for read-heavy workloads before considering anything more exotic.
- Autoscale on a metric that reflects user experience, not just CPU.
What to do before any of this
Instrument first. Without request tracing and percentile latency (p95 and p99, not averages), scaling work is guesswork — and teams routinely spend a quarter optimising something that was never the bottleneck. Load-test against a realistic data volume, because a query plan on 10,000 rows tells you nothing about its behaviour on ten million.