2. What is a load balancer

Description

In the introduction it became clear why Enlace needs to spread requests among several servers. This lesson answers the what: what exactly a load balancer is, how it sits in the architecture, and what the system gains by having it. The definition fits in one sentence: a balancer is a component placed in front of a pool of servers that spreads the incoming requests among them, presenting itself to the world as a single address. The client talks to a single IP (or a single domain name); behind it, the balancer decides which of the N servers in the pool it passes each request to. It's a reverse proxy: an intermediary that receives on behalf of many servers and distributes.

You'll see where the balancer lives (between the client and the app servers), the four concrete things it gives you (a single point of entry, load spreading, adding and removing servers without the client noticing, and taking the downed ones out of rotation), the difference between balancing at layer 4 (looking only at IPs and ports) and at layer 7 (looking at the HTTP content), and an uncomfortable question that has to be faced once and for all: if everything passes through the balancer, isn't it the single point of failure? The answer —deploying it redundantly— we sketch here and module 7 formalizes. By the end you'll know how to place a balancer in a diagram and explain what role each arrow plays.

Connection to the module: this lesson installs the physical piece; the following ones fine-tune it. Lesson 3 opens the black box of "spreads among them" and shows you how the balancer chooses the server (the algorithms). Lesson 6 gives the balancer the mechanism by which it knows which server is "down" to take it out of rotation (the health checks). Lesson 7 uses the "add and remove without the client noticing" property to scale horizontally. Everything stated here as a balancer capability is detailed in a later lesson; this is the general blueprint of the machine.

The telephone exchange switchboard

Think of it this way. In the old telephone exchanges, when you called a large company, you didn't dial a specific employee's number: you dialed the company's number, and an operator at a switchboard received your call and passed it to the free extension that corresponded. From outside, you saw a single number. Inside there were fifty phones, but that wasn't your problem: the operator connected you with one that was available.

That switchboard is a load balancer, and the analogy is surprisingly exact. The company's public number is the IP or the single domain the client talks to (enla.ce). The operator is the balancer: it receives each call and decides which phone to pass it to. The fifty phones are the pool of app servers, invisible from outside. And notice the three things the operator lets you do without the caller noticing: it can distribute the calls so no employee saturates; it can add a new phone to the switchboard on a Monday and start passing calls to it without changing the public number; and if a phone is broken, it stops passing calls to it and sends them to the ones that do work. No one calling from outside knows how many phones there are, which were added yesterday, or which is broken. They see a number, and it works.

That opacity is the balancer's gift: it decouples the client from the number and state of the servers. The client talks to a stable address; behind it, you add servers, remove servers, restart servers, and the client never finds out. Without a balancer, the client would have to know the IPs of the fifty servers and decide which to call —and every time you add or remove one, you'd have to notify every client in the world—. With a balancer, the list of servers is an internal secret.

A load balancer is a reverse proxy that presents itself as a single address and spreads the incoming requests among a pool of servers, hiding from the client how many servers there are and which is alive. It decouples the client from the internal topology.

Where the balancer lives

Let's bring the analogy down to the diagram. The balancer sits on the single path between the client and the app servers: everything enters through it.

flowchart TD
    C1[client A] --> LB
    C2[client B] --> LB
    C3[client C] --> LB
    LB[LOAD BALANCER<br/>enla.ce · one public IP]
    LB --> S0[app-0]
    LB --> S1[app-1]
    LB --> S2[app-2]
    S0 --> Shared[(shared cache<br/>+ database)]
    S1 --> Shared
    S2 --> Shared

Read it top to bottom. All the clients talk to a single address (enla.ce, which resolves to the balancer's IP). The balancer spreads each request to one of the servers in the pool (app-0, app-1, app-2). And all the servers share the same cache and the same database below —which, note, already hints at the statelessness of lesson 4: the servers keep nothing of their own, everything shared lives below them, not inside them—.

Note one thing about the addresses. The balancer has a public IP (the one the internet sees); the pool's servers live in a private network with IPs the client never sees or needs. This isn't just tidiness: it's security. The app servers aren't exposed directly to the internet; only the balancer is, and it decides what passes inward. The pool is a closed box with a single door.

The four things a balancer gives you

A balancer isn't just a distributor. Let's state them explicitly, because each one resolves a real problem and the following lessons develop several:

1. A single point of entry. The client knows one address, not N. This is what makes everything else possible: since the list of servers is internal, you can change it freely. Without this decoupling, scaling would be impossible without reconfiguring every client.

2. Load spreading. It distributes the requests among the servers so none saturates while others are idle. How it distributes them —round-robin, least-connections, by hash— is lesson 3. This is the name of the component, but it's not the only thing it does.

3. Adding and removing servers transparently. You can add a new server to the pool and the balancer starts sending it traffic without any client changing anything; you can take one out for maintenance and the balancer stops sending to it without anyone noticing an error. This property is the basis of horizontal scaling (lesson 7) and of downtime-free deployment.

4. Taking the downed servers out of rotation. The balancer monitors each server's health and, if one stops responding, stops sending it requests until it recovers. That way a dead machine doesn't translate into errors for the client: the balancer simply routes to the live ones. The "monitor the health" mechanism is lesson 6 (health checks).

The four rest on the first. Because the client only sees one address, the balancer has total freedom to distribute, rotate, add, and remove below. That indirection —"talk to me, I'll handle the rest"— is the whole idea.

Layer 4 vs. layer 7: what the balancer looks at

Not all balancers look at the same thing before deciding. There are two levels, and the name comes from the layers of the network model (OSI):

• Layer 4 balancing (transport). The balancer looks only at the transport information: source IP, destination IP, ports. It doesn't open the request's content; it doesn't know whether it's a GET /aX9kR2q or a POST /shorten. It only sees "a connection from this IP to this port" and routes it to a server. It's fast and cheap (it parses nothing) but blind to the content: it can't make decisions based on the URL, the cookies, or the headers.

• Layer 7 balancing (application). The balancer opens the HTTP request and looks at its content: the path (/aX9kR2q), the headers, the cookies, the method. With that it can make smart decisions: send /shorten (writes) to one pool and /* (reads) to another; route by session cookie; terminate TLS; rewrite headers. It's slower and more expensive (it parses each request) but much more flexible.

   LAYER 4 (transport)                    LAYER 7 (application)
   sees: IP:port -> IP:port               sees: GET /aX9kR2q  Host: enla.ce
   decides: fast, per connection                Cookie: session=...
   blind to HTTP content                  decides: by path, header, cookie
                                          can terminate TLS, route by path

Which one does Enlace get? For the hot path —the resolve, a GET /short_code that just has to be spread evenly— a layer 4 balancer is more than enough: it's simple, fast, and you don't need to look at the content to spread. If later Enlace wants to route the writes (POST /shorten) to a different pool from the reads, or terminate TLS at the balancer, or route by cookie, then it moves up to layer 7. The practical rule: layer 4 when you only spread; layer 7 when you need to decide according to the content. Many real systems use layer 7 by default (nginx, HAProxy, AWS ALB operate there) because the flexibility almost always is worth its cost; layer 4 balancers (AWS NLB) are reserved for when extreme performance matters more than intelligence.

The uncomfortable question: isn't the balancer a single point of failure?

Here we have to be honest. We just made all the traffic pass through a single machine, the balancer. The question jumps out: if the balancer goes down, does all of Enlace go down? Because it doesn't matter how many healthy app servers you have behind it —if the only door is walled up, no one gets in. We traded "one app server is the bottleneck" for "the balancer is the single point of failure". Did we gain anything?

Yes, but that hole has to be closed, and the answer is the same idea applied to the balancer: don't have just one. You deploy two or more balancers in redundancy, typically in active-passive mode: one serves the traffic and the other waits ready; if the active one dies, the passive takes its place (often inheriting a floating IP that "jumps" to the healthy balancer). Another option is to have several active balancers at once, and spread among them with DNS (several IPs for enla.ce). The point is that the balancer, the piece everything passes through, is never a single machine in production.

How it's detected that a balancer died, how the passive takes over without both believing they're the active at once (the "split brain" problem), how the floating IP jumps —all of that is reliability and failover, a topic of module 7 and the resilience guide. Here it's enough to know that the objection is real and has an answer: the balancer is made redundant, just as you made the app servers redundant. The module's underlying rule applies also to the piece that spreads everything: nothing critical lives on a single machine.

Common mistakes

Exposing the app servers directly to the internet, "in addition" to the balancer (architecture mistake). What happens: someone puts the balancer but leaves the app servers' IPs accessible from outside "just in case". Now there are two ways into the system —through the balancer and bypassing it— and the second evades the spreading, the health checks, and any of the balancer's rules. Why it happens: the balancer is seen as an extra, not as the only door. How to detect it: if you can make a direct request to app-1 from your home, the back door is open. How to fix it: the app servers live in the private network, with no public IP; the only entrance is the balancer. The pool is a closed box with a single door, and that door is the balancer.

Believing a layer 4 balancer can route by URL or cookie (concept mistake). What happens: someone configures a layer 4 balancer and expects it to send /shorten to one pool and /* to another, and it doesn't work —the layer 4 balancer doesn't see the URL—. Why it happens: it's assumed that every balancer "understands HTTP", when the layer 4 one only sees IPs and ports. How to detect it: if you need a decision based on the request's content (path, header, cookie) and you use layer 4, the decision is impossible by design. How to fix it: to route by HTTP content you need layer 7. Choose the layer according to what you need to decide: layer 4 if you only spread connections, layer 7 if you decide according to what the request says.

Forgetting that the balancer is a single point of failure (reliability mistake). What happens: someone puts a single balancer in front of a pool of ten healthy servers, feels safe for having ten, and the day the balancer restarts, all of Enlace is inaccessible despite the ten live servers. Why it happens: attention is concentrated on the servers' redundancy and the redundancy of the piece that spreads them is forgotten. How to detect it: ask yourself "which machine, if it dies alone, brings down the whole system?". If the answer is "the balancer", you have a single point of failure. How to fix it: deploy the balancer in redundancy (active-passive with a floating IP, or several active by DNS). The failover detail is module 7, but the decision to not have just one is made in the design.

Exercises

Exercise 1 — Place the pieces. Draw (or describe in text) Enlace's architecture with: three app servers, a balancer, a shared cache, and a shared database. Mark which has a public IP, which live in the private network, and where a resolve request enters. Then answer: how many addresses does the client know?

client  --(1)-->  BALANCER (public IP)  --(2)-->  app-0 / app-1 / app-2  (private network)
                                                          |
                                                      (3) v
                                                cache + database (private network, shared)

Exercise 2 — Layer 4 or layer 7. For each requirement of a balancer, say whether it demands layer 7 (looking at the HTTP content) or whether layer 4 (IP and port) is enough, and why. (a) Spread the resolve requests evenly, which are all a GET to a short_code. (b) Send the writes (POST /shorten) to one pool of servers and the reads to another. (c) Terminate the TLS encryption at the balancer so the app servers speak plain HTTP inside.

Exercise 3 — The single point of failure. A design has a balancer in front of five app servers. A colleague says: "With five servers, Enlace tolerates failures: up to four can go down and it stays up". Correct the statement by identifying which machine, if it goes down alone, brings everything down, and propose the solution in one sentence.

Summary and next step

In this lesson you installed the module's physical piece: the load balancer, a reverse proxy that presents itself as a single address and spreads the incoming requests among a pool of servers. With the telephone exchange switchboard you saw that its gift is the opacity: the client talks to a single number and never finds out how many servers there are, which were added yesterday, or which is broken. You enumerated the four things it gives you —a single point of entry, load spreading, adding and removing servers transparently, and taking the downed ones out of rotation—, all resting on that decoupling. You distinguished layer 4 balancing (fast, blind to content, looks at IP and port) from layer 7 (more expensive, smart, looks at the HTTP), and you saw that layer 4 is enough for Enlace's hot path. And you faced the uncomfortable question: the balancer is a single point of failure, which is closed by deploying it redundantly (detail in module 7).

Before moving on you should be able to: define a balancer in one sentence; place it in a diagram with the correct public IP and private network; name the four things it gives you; distinguish layer 4 from layer 7 with an example of each; and explain why the balancer is deployed redundantly.

What comes next is opening the black box of "spreads among them". In lesson 3 you'll see how the balancer chooses which server to send each request to: the three algorithms that matter —round-robin (rotate in order), least-connections (to the one with the least work), and by hash (the same key always to the same server)—, run and measured, so you know which to choose and why.

Resources

• nginx — What Is Load Balancing? — nginx's official introduction to the concept, with the same reverse-proxy-in-front-of-a-pool model. Short and concrete; it anchors this lesson's abstractions in the balancer you'd probably deploy.
• HAProxy — Starter Guide, "Load balancing" — the documentation of HAProxy, a reference balancer, explains the reverse-proxy architecture, the difference between TCP mode (layer 4) and HTTP mode (layer 7), and the pool's role. The source for going from the concept to the real configuration.
• System Design Primer — Load balancer (Layer 4 vs Layer 7) — the summary that contrasts layer 4 and layer 7 with their tradeoffs, and places the balancer in the complete system design. A good second angle on this lesson's central distinction.