System Design Basics: How Big Applications Stay Up

Introduction

System design is the discipline of arranging servers, databases, and networks so an application stays fast and available as it grows. It's the core of senior engineering interviews, but more importantly it's the difference between a site that works in the demo and one that works on launch day.

The subject looks vast, but it compresses well: a handful of building blocks, combined and recombined, driven by one repeated question — what breaks first, and what does fixing it cost? This tutorial teaches the blocks by growing one application from a single server to something that could serve millions, one bottleneck at a time.

What you will build or practice

You will design — on paper — the architecture of a notes-sharing web app at three sizes: a hundred users, a hundred thousand, and tens of millions. Each jump breaks something specific, and each fix introduces exactly one new building block.

Before you begin

No accounts, no cloud console — just a pen. Vocabulary you'll want loaded:

• Server: a computer running your application code.
• Database: where the data lives; the hardest thing to scale.
• Request: one user asking your system to do one thing.
• Latency: how long a request takes. Throughput: how many you can serve per second.

Key concept

Every design is a trade-off, and "it depends" is the correct answer — followed by what it depends on. There is no best architecture, only fits between load, budget, and how much staleness or downtime you can tolerate. Interviewers probe for this; production rewards it. If you remember one sentence, make it this one: state what breaks, fix only that, and say what the fix costs.

Step 1: One server (and why it's underrated)

Stage one: a single machine running your app and your database. DNS points your domain at it.

This is not a toy. A well-provisioned single server handles enormous traffic — plenty of profitable businesses never outgrow one box plus backups. When it slows down, the first move is vertical scaling: buy a bigger machine. More RAM, faster disks, more cores. No code changes, no new concepts.

Vertical scaling fails in two ways: machines have a size ceiling (and prices grow faster than specs near it), and one machine is one point of failure — reboot it and you're offline.

Step 2: Separate the database, add a second server

First structural change: move the database to its own machine, and run two app servers. Now nothing competes with the database for resources, and either app server can die without taking you down.

But two servers create a question that didn't exist before: which one gets the request? Enter the load balancer — a small, fast machine in front that spreads requests across your servers and, crucially, stops sending to a server that fails its health checks. Users hit one address; the balancer does the rest.

This is horizontal scaling — more machines instead of bigger ones. It requires one discipline from your code: app servers must be stateless. Any server must be able to serve any user, which means session data lives in the database (or a shared store), never in one server's memory.

Step 3: Stop asking the database the same question

At a hundred thousand users, your database is answering the same reads thousands of times a minute. The fix is a cache: a fast in-memory store (Redis and Memcached are the standard tools) that sits beside the database and remembers recent answers.

The pattern is almost always cache-aside: check the cache; on a miss, read the database and store the answer with an expiry time. Reads that hit the cache come back in microseconds and never touch the database at all.

The cost is the first genuinely hard trade-off: staleness. Cached data can be seconds or minutes old. For a view counter, nobody cares; for an account balance, everybody does. Deciding what may be stale, and for how long is real system design — that decision, not the tool choice, is the design.

Static files (images, CSS, JavaScript) get the same treatment via a CDN — a network of edge servers that hold copies close to users, so a request from Sydney doesn't cross an ocean for a logo.

Step 4: Scale the reads, then the writes

Traffic keeps growing, and the database is again the bottleneck — caching absorbed repeated reads, but not the long tail. Two moves, in order of pain:

Read replicas (mild): keep one primary database for writes; continuously copy its data to replicas that serve reads. Most apps read far more than they write, so this multiplies capacity where it's needed. New trade-off: replication lag — a replica can be a moment behind, so a user might not instantly see their own write unless you route their next read to the primary.

Sharding (severe): when writes outgrow one primary, split the data itself — users A–M on one database, N–Z on another, or hash-based splits across many. Capacity becomes nearly unlimited; simplicity is the casualty. Queries that cross shards get hard, rebalancing data is delicate, and operational complexity jumps. Sharding is famously a last resort — worth knowing, worth deferring.

Step 5: Stop doing slow work during the request

A user shares a note; your app must save it, notify followers, generate a preview, and update a search index. Doing all that before responding makes the user wait on work they don't see.

The fix is a message queue (RabbitMQ, Kafka, and cloud equivalents like SQS are the standard tools): the app saves the note, drops "note shared" onto the queue, and responds immediately. Separate worker processes consume the queue and do the slow work seconds later.

This buys speed and resilience — if workers die, the queue holds the backlog until they return. The trade-off is a new consistency flavor: the system is eventually consistent — briefly, a follower hasn't been notified yet — and you've accepted that the notification is guaranteed to happen but not instant.

Practice exercise

Design the URL shortener (the classic for good reason):

1. The task: sketch a system where users create short links and clicks redirect. Assume reads outnumber writes 100:1. Where do the building blocks go?
2. Expected output: a one-page diagram: load balancer → stateless app servers → cache → primary DB with read replicas. Somewhere, a decision about how short codes are generated.
3. One hint: with a 100:1 read ratio, the cache is your whole performance story — most-clicked links should almost never touch the database.
4. Stretch goal: click counts are heavy writes on hot links. Could a queue make counting asynchronous? What does the count's accuracy cost if so?

Common mistakes

• Designing for scale you don't have. Sharding a database with a thousand users burns months and buys nothing. Name the current bottleneck; fix that.
• Scaling servers with state inside them. Sessions in one server's memory break the moment a balancer routes the user elsewhere. Stateless first — it's a code discipline, not an infrastructure purchase.
• Caching without an invalidation story. Every cached value needs an answer to "when is this wrong, and how wrong is acceptable?" — an expiry time is the minimum viable answer.
• Treating diagrams as the deliverable. The deliverable is the trade-off reasoning; the diagram just records it.

Check your understanding

1. Your app is slow. What do you measure before choosing between a cache, a replica, and a bigger server?
2. Why must app servers be stateless before horizontal scaling works?
3. A user posts a comment and doesn't see it on refresh. Which two building blocks from this tutorial could each explain that, and how would you tell them apart?

Key takeaways

• Scale vertically until it stops making sense; it's the cheapest complexity you'll ever buy.
• Horizontal scaling = load balancer + stateless servers. State moves to shared storage.
• Caches trade freshness for speed; the design decision is what may be stale and for how long.
• Scale database reads with replicas; shard for writes only when forced.
• Queues move slow work out of the request path, buying speed at the price of "eventually."
• Every block answers "what breaks first?" — that question, asked repeatedly, is system design.

Next steps

Sketch a system you actually use — where must the queue be? where's the cache? Then re-run the practice exercise from memory in a week. For the security dimension of these designs, the OWASP Top 10 in Plain English covers what attackers do to exactly these architectures.

Related resources

• The OWASP Top 10, Explained in Plain English — securing what you just designed
• Linux Command Line Essentials — the servers in these diagrams run Linux
• Project Starters — build something worth scaling

Newsletter or next lesson

This is the first system-design lesson on the site — deeper dives (caching patterns, database choices, real capacity math) are on the roadmap. The tutorials collection has the current set.