Diverting traffic by percentage

Overview

You have the facade in place (lesson 2) and the new service built alongside (lesson 3). The two pieces are in place, waiting. This lesson turns the knob: diverting the traffic by percentage. It's the phase that gives the strangler fig life —the moment the real traffic starts moving from the legacy to the modern, controlled by a single number, the traffic_percent—. You start at 0 (everything to the old), go up to 10 (a canary), to 50 (half and half), to 100 (everything to the new), observing at each level before going up to the next.

The idea of going up in steps —and not jumping from 0 to 100— is the heart of the pattern's safety. This is called a canary release: you send first a small fraction of the traffic to the new one, like miners used to lower a canary into the mine to detect gas before risking the people. If the canary sings, all good, you raise the percentage. If the canary falls, you only lost the canary —a small fraction of the traffic—, not the whole mine. Diverting 10% first means that, if the modern has a problem, only 10% of the users see it, and you find out in time to fix it before exposing the rest.

And there's a second piece that makes the diversion truly safe: the fallback. If the modern throws an error processing a request, the router doesn't return the error to the user —it sends it to the old route, to the legacy, which is still there and still working—. The user receives a correct response (the legacy's), even though the new one failed. The fallback turns "the modern failed" from a visible incident into an invisible event that only you, in your metrics, record. This lesson executes it and discovers something revealing: even at 100% of the traffic "on the new," the cases the modern doesn't yet handle keep falling to the legacy via the fallback —and that tells you, with numbers and without drama, that the modern isn't complete yet—.

Connection with the module. Lessons 2 and 3 left the facade and the modern ready; this one does phase 3, the diversion by percentage with fallback, which is the core of the strangler. Lesson 5 opens the ways of deciding the diversion (by route, by percentage, by feature flag); lesson 6 goes deeper into the fallback and into the observability of the two routes to decide when to go up; and lesson 7 takes the percentage to 100 and retires the legacy. Notice the boundary: here the diversion is by percentage of the traffic (the request's bucket decides). Diverting by endpoint or by user are other strategies that lesson 5 compares; and deciding when it's safe to raise the percentage —with what metrics— is lesson 6. Here we build the mechanism of the diversion by percentage and the fallback that protects it.

An analogy: open the tap little by little, with the old bucket underneath

Imagine you're replacing the water pipe of a house. The old pipe works; the new one you just installed but you don't fully trust it. You don't open the new pipe's tap all the way at once —if it has a leak, you flood the house—. You open the tap little by little: first you let a trickle through, 10% of the water, and you look for leaks. If there are none, you open it halfway. You look again. You open it all the way. Only when all the water passes through the new pipe without a drop out of place do you close the old one.

And just in case, while you do this, you leave the old pipe connected as backup: if the new one starts to drip at a point, a valve diverts the water back through the old one while you fix it. Nobody goes without water for a second. The house always has water running; the only thing that changes is which pipe.

The traffic_percent is how far you open the new pipe's tap: 0, 10, 50, 100. The canary is opening only a trickle first to detect leaks with low risk. And the fallback is the old pipe connected as backup: if the water can't pass through the new one (the modern throws an error), the valve sends it through the old one (the legacy) and the house never runs dry. Opening the tap little by little, with the backup connected, is exactly this phase of the strangler.

Worked example: the 0→10→50→100 ramp with fallback

We're going to execute the complete diversion. We have the legacy_catalog —solid, handles all the cases— and the modern_catalog —new, fast, but with a bug: it doesn't yet handle the "out of stock" case and throws an error when the query is webcam—. The strangler_router routes by percentage using the stable bucket, and —key— wraps the call to the modern in a try/except: if the modern throws any error, it falls to the legacy and counts it as a fallback. We run a fixed batch of 1000 requests, where 1 in every 10 queries webcam (the buggy case), and we raise the traffic_percent through the steps 0, 10, 50, 100, measuring how many went each way.

import zlib

# === Legacy: solid, handles all the cases (including stock 0). ===
def legacy_catalog(request):
    return {"status": 200, "source": "legacy", "product": request["q"]}

# === Modern: new, fast... but STILL has a bug in the "out of stock" case. ===
def modern_catalog(request):
    if request["q"] == "webcam":          # route not implemented yet
        raise ValueError("modern: unhandled out-of-stock path")
    return {"status": 200, "source": "modern", "product": request["q"]}

# === The strangler router: routes by percentage, with FALLBACK to the legacy. ===
def bucket(request_id):
    return zlib.crc32(str(request_id).encode()) % 100

def strangler_router(request, traffic_percent, counters):
    if bucket(request["id"]) < traffic_percent:
        try:
            resp = modern_catalog(request)          # tries the new route
            counters["modern_ok"] += 1
            return resp
        except Exception:
            counters["fallback"] += 1               # the new one failed: falls to the old
            return legacy_catalog(request)           # the old route is the net
    counters["legacy"] += 1
    return legacy_catalog(request)

# --- A fixed batch: 1 in every 10 requests queries "webcam" (the buggy case). ---
N = 1000
requests = [{"id": i, "q": "webcam" if i % 10 == 0 else "ssd"} for i in range(1, N + 1)]

print(f"Raising traffic_percent 0 -> 10 -> 50 -> 100  ({N} requests per level)\n")
print(f"{'traffic':>8}{'modern_ok':>11}{'fallback':>10}{'legacy':>8}"
      f"{'served by legacy':>19}")
print("-" * 56)
for pct in (0, 10, 50, 100):
    counters = {"modern_ok": 0, "fallback": 0, "legacy": 0}
    for req in requests:
        strangler_router(req, pct, counters)
    served_by_legacy = counters["legacy"] + counters["fallback"]
    print(f"{pct:>7}%{counters['modern_ok']:>11}{counters['fallback']:>10}"
          f"{counters['legacy']:>8}{served_by_legacy:>19}")

print("-" * 56)
print("\n  At 100%: modern serves the majority, but the ~100 'webcam' requests")
print("  keep falling to the legacy via the FALLBACK -> the new one is NOT ready yet")
print("  to retire the legacy. The fallback protects you AND warns you.")

What to expect. When you run the file, the output is exactly this:

Raising traffic_percent 0 -> 10 -> 50 -> 100  (1000 requests per level)

 traffic  modern_ok  fallback  legacy  served by legacy
--------------------------------------------------------
      0%          0         0    1000               1000
     10%         99        10     891                901
     50%        471        49     480                529
    100%        900       100       0                100
--------------------------------------------------------

  At 100%: modern serves the majority, but the ~100 'webcam' requests
  keep falling to the legacy via the FALLBACK -> the new one is NOT ready yet
  to retire the legacy. The fallback protects you AND warns you.

Read the table step by step, because each row tells part of the story.

At 0%, the 1000 requests go to the legacy (legacy=1000), zero to the modern. The tap is closed; all the water passes through the old pipe. modern_ok and fallback at zero: the modern isn't even touched.

At 10%, the bucket sends 109 requests to the modern's route. Of those, 99 succeed (modern_ok=99) and 10 fall into fallback —they're the ones that queried webcam, the case the modern doesn't handle, and that the try/except diverted to the legacy—. The other 891 were never selected for the modern and went straight to the legacy. Notice something important: those 10 webcam requests were not an error for the user. The modern failed, yes, but the fallback served them with the legacy, so the user received their correct response. The modern's failure stayed invisible to the client and visible only in your fallback counter.

At 50%, the bucket selects ~520 requests for the modern; 471 succeed and 49 fall into fallback (the webcam ones that fell in the selected half). And at 100%, the bucket selects all the 1000: 900 succeed in the modern and 100 fall into fallback —the 100 webcam requests of the whole batch—. The legacy column (direct, not via fallback) reaches 0: no request was chosen for the legacy.

Here's the lesson's revelation, in the served by legacy column (which sums legacy + fallback). At 100%, that column marks 100, not 0. Even though you diverted "all the traffic to the new," 100 requests kept being served by the legacy —via fallback, because the modern didn't know how to handle them—. This is very valuable information: it tells you, with numbers and without any user having been affected, that the modern isn't complete yet. You can't retire the legacy: if you turned it off now, those 100 webcam requests would have no one to serve them. The fallback did two things at once: it protected you (no user saw an error) and it warned you (the modern has a hole in the webcam case). When you fix that case, the fallback at 100% will drop to 0, and then —and only then— you'll be able to retire the legacy. That's lesson 7.

Deep dive: the fallback as a net and as a detector

The fallback has two functions, and it's easy to see only the first. The obvious one is the safety net: if the modern fails, the user isn't left without a response. The less obvious one, and perhaps more valuable, is that the fallback is a completeness detector: each request that falls into fallback is proof, taken from the real traffic, that the modern has a case it doesn't yet handle. The burn-down of fallbacks toward zero is your measure of how ready the modern is to completely replace the legacy.

             request
                │
        ┌───────┴────────┐
   bucket < percent?      no ──> legacy_catalog  (chosen for the old)
        │ yes
        ▼
   modern_catalog
        │
   ┌────┴─────┐
  ok          error
   │            │
 response    FALLBACK ──> legacy_catalog  (the new one failed; the net catches)
 of modern    (+counts the fallback as a signal of "modern incomplete")

An important design detail: the fallback catches the error after trying the modern, which means the request pays the cost of trying the modern and the cost of the legacy. It's slower than going straight to the legacy. That latency overhead is acceptable because it's temporary (it disappears when you fix the modern) and rare (it only happens in the cases that fail), but it's real, and lesson 6 measures it. The rule is: the fallback is for unexpected errors of the modern, not for cases you know the modern doesn't handle. If you know the modern doesn't handle webcam, the best thing isn't to let it fail and fall into fallback every time —it's not to route webcam to the modern in the first place (a per-route decision, lesson 5)—. The fallback is the net for what you didn't see coming; not a substitute for completing the modern.

There's a subtle decision the example hides: what counts as an "error" that triggers the fallback? In the example, any exception. In production, the line is finer: a 500 error from the modern clearly triggers a fallback; but a timeout? a legitimate 404 response (the product doesn't exist)? A 404 isn't a failure of the modern —it's a correct response— and it shouldn't trigger a fallback, or you'd mask valid responses. Defining what is a failure that deserves a fallback is part of designing the router, and it's tuned with the observability of lesson 6.

Common mistakes

Jumping from 0% to 100% without a canary. What happens: confident that the modern is "already tested," the team puts the traffic_percent at 100 in a single change. Why it happens: the intermediate steps feel slow and bureaucratic —"if it works, it works"—. How to spot it: there's no 10% or 50% period in the history; the traffic went from all-old to all-new in one deploy. How to fix it: the whole point of the canary is to discover the modern's problems with little traffic exposed. At 10%, a modern bug affects 10% of the users and gives you time to react; at 100%, it affects everyone at once, which is exactly the big-bang the strangler exists to avoid. Go up in steps, observe at each one (lesson 6), and only go up to the next when the current one is healthy. The speed of going up isn't a virtue; the safety of going up is.

Diverting traffic without a fallback. What happens: the router sends the percentage to the modern and, if the modern fails, returns the error to the user. Why it happens: the fallback is extra code (the try/except, the second call) and it's omitted "to simplify." How to spot it: when the modern fails in production, 500 errors appear visible to the users in the diverted fraction of traffic. How to fix it: the fallback isn't optional in a strangler —it's what makes diverting traffic safe—. Without a fallback, each request you send to the modern is a bet without a net: if the modern has an unhandled case, the user sees the error. With a fallback, the worst case is a response from the legacy (correct, maybe slower) and a counter that increments. The cost of writing the try/except is minuscule compared to the cost of exposing the modern's bugs to real users.

Confusing "100% of traffic routed" with "the modern is complete." What happens: the team sees the traffic_percent at 100 and declares the migration finished, ready to retire the legacy. Why it happens: "100%" sounds like the end. How to spot it: even though the traffic_percent is 100, the fallback column (or served by legacy) isn't 0 —as in the example, where at 100% 100 requests still fell to the legacy—. How to fix it: the criterion for retiring the legacy isn't "the traffic_percent reached 100," it's "the legacy no longer serves a single request, not even via fallback." As long as the fallback is greater than 0 at 100%, the modern has holes the legacy is covering, and turning off the legacy would leave those requests unserved. The traffic_percent measures what you tried; the fallback measures what the modern couldn't. Only when the second is 0 can you retire the legacy (lesson 7).

Exercises

Exercise 1 — Read the split with fallback. In the output, at traffic_percent=10 there was modern_ok=99, fallback=10, legacy=891. (a) How many requests were routed to the modern in total? (b) Why did 10 of them end up in the legacy? (c) If the modern fixed the webcam case, what would fallback be worth at 10%, and what modern_ok?

See solution

(a) 109 requests were routed to the modern: the ones that succeeded (modern_ok=99) plus the ones that fell into fallback after trying the modern (fallback=10). The bucket selected 109 for the new route; of those, 99 the modern resolved and 10 failed. 99 + 10 = 109, approximately 10% of 1000.

(b) Because those 10 requests queried webcam, the case the modern doesn't handle (it throws ValueError). They were chosen for the modern by their bucket, the modern tried to serve them and failed, and the try/except diverted them to the legacy —the fallback—. The user received the legacy's response; the failure stayed only in the counter.

(c) If the modern fixed webcam, the fallback at 10% would be worth 0 (nothing fails anymore) and modern_ok would be worth 109 (the 109 requests routed to the modern now all successful). All the traffic routed to the modern would be served by the modern. That's the healthy state that allows raising the percentage with confidence, and —at 100%— retiring the legacy.

Exercise 2 — The fallback as a detector. The text says the fallback "protects you AND warns you." (a) Explain what each part means with the 100 requests that fell into fallback at 100%. (b) Why is it better to find out about a hole in the modern via the fallback than via a test you forgot to write? (c) What metric, observed along the ramp, would tell you the modern is ready to completely replace the legacy?

See solution

(a) Protects you: those 100 webcam requests would have been 500 errors for the users if the modern had served them without a net; instead, the fallback sent them to the legacy and the users received correct responses. Nobody was affected. Warns you: the fallback=100 counter at 100% of traffic is the evidence, taken from the real production traffic, that the modern has a case (webcam) it doesn't yet handle. Without that counter, you wouldn't know the hole exists.

(b) Because the fallback warns you with real traffic data, not with your imagination of what cases might be missing. A test you forgot to write is, by definition, a case you didn't think of —and if you didn't think of the test, you didn't think of the code either—. The fallback, in contrast, exercises the modern with the requests the users actually make, and illuminates exactly the holes that matter (the ones people really query). It's a detector fed by reality, not by your list of anticipated cases.

(c) The metric is the fallback count at a high traffic_percent (ideally at 100%). If at 100% the fallback is 0 —the modern successfully served all the requests the users made— the modern is ready to replace the legacy: there's no case the legacy is covering. As long as the fallback at 100% is greater than 0, the modern has holes and the legacy is still needed. It's exactly the retirement criterion of lesson 7.

Exercise 3 — The canary that saves. Imagine the modern has a serious bug that makes all the requests fail (not just webcam). (a) With traffic_percent=10 and fallback, what would the users see? (b) What would you see in the counters? (c) Compare with what would happen if you had jumped to traffic_percent=100 without a fallback.

See solution

(a) With 10% and fallback: the bucket routes ~109 requests to the modern, all fail, and all fall into fallback to the legacy. The users receive correct responses from the legacy —the fallback catches 100% of the modern's failures—. From outside, the system works perfectly. No user sees an error.

(b) You'd see modern_ok=0, fallback≈109, legacy≈891 at 10%. A triggered fallback (all the traffic routed to the modern falling to the legacy) is a very clear alarm: the modern is completely broken. You detect it immediately, with 100% of the users still protected, and you stop the ramp (lower the traffic_percent to 0) while you fix it. The canary sang and only "died" in your metrics, not in anyone's experience.

(c) If you had jumped to traffic_percent=100 without a fallback: the bucket routes all the 1000 requests to the modern, all fail, and without a fallback the 500 error reaches all the users. It's a total outage of GET /products in production, for 100% of the people, all at once. The combination of canary (little traffic exposed) plus fallback (safety net) turns a catastrophic bug into a non-event that only you see; jumping to 100% without a fallback turns it into a major incident. This is, in one image, the reason for being of the pattern.

Summary and next step

In this lesson you did the central phase of the strangler fig: diverting the traffic by percentage. You saw, with the water tap opened little by little and the old bucket underneath, that raising the traffic_percent in steps (canary) and keeping the fallback connected makes the diversion safe: if the new one fails, the old one serves and nobody runs out of water. And you executed it over 1000 requests, going up 0→10→50→100, measuring how many went each way. You discovered the fallback's double function: it protects you (the 100 failures of the webcam case stayed invisible to the users) and it warns you (those 100 fallbacks at 100% tell you, with real data, that the modern still has a hole and that you can't retire the legacy yet).

Before moving on you should be able to: explain why you go up in steps (canary) and don't jump to 100%; describe the fallback's double function (net and detector); read a split with modern_ok, fallback, and legacy and correctly add up how many each service served; and argue why "100% of traffic routed" isn't the same as "the modern is complete."

Lesson 5 opens the ways of diverting. In this lesson you diverted by percentage (the request's bucket decides), but it's not the only way: you can divert by route/endpoint (the whole listing to the new, the detail still to the old), by percentage/canary (what you saw, but stuck to the user so they don't jump route mid-session), or by feature flag/user (only staff and beta see the new one, outside the chance of the percentage). You're going to execute the three over the same queue of requests and measure the per-user stability —why a customer must not land on the new one on one request and on the old one on the next—.

Resources

• Martin Fowler, "CanaryRelease" — martinfowler.com/bliki/CanaryRelease.html. The entry that defines the canary release: expose a new version to a small subset of the traffic before going to everyone. The origin of the image of the canary in the mine. In English.
• Sam Newman, Monolith to Microservices (O'Reilly, 2019), ch. 3 — the incremental diversion of calls from the monolith to the new service, and the role of the proxy that controls what fraction of the traffic goes to each side. In English.
• Chris Richardson, "Pattern: Strangler application" — microservices.io/patterns/refactoring/strangler-application.html. The gradual diversion of functionality from the monolith to the set of new services, with the old implementation available as backup. In English.
• Paul Hammant, "Legacy Application Strangulation: Case Studies" (2013) — paulhammant.com/2013/07/14/legacy-application-strangulation-case-studies. Real cases where the traffic is diverted little by little with the old route as fallback, and how the fraction is decided to go up. In English.