Module 3: The Strangler Fig Pattern

Ways of diverting: by route, by percentage, by flag

Next →

Overview

In the previous lesson you diverted traffic by percentage: each request's bucket decided whether it went to the legacy or the modern, and you raised a single number —the traffic_percent— to move the traffic from the old to the new. The percentage is the best-known way of diverting, but it isn't the only one. This lesson opens the three ways of diverting traffic in a strangler fig, and —more importantly— when each one is best:

  • By route/endpoint. You decide by the request's path: GET /products (the listing) goes to the modern, GET /products/{id} (the detail) stays in the legacy. Coarse-grained, a whole endpoint at a time, all or nothing per path.
  • By percentage/canary. You decide by a fraction of the traffic, as in lesson 4. Gradual, a sample of the traffic, ideal for going up little by little. With a nuance this lesson introduces: the bucket is computed on the user, not on the request, so the same user doesn't jump route mid-session.
  • By feature flag/user. You decide by who makes the request: only the internal users (staff) and the beta users see the modern; the rest stay in the legacy. Targeted, outside the chance of the percentage, perfect for testing with people who know they're testing.

The three are legitimate and are often combined. The lesson executes them over the same queue of requests to see how each one splits the traffic differently, and pauses on a critical property they only have if they're designed well: per-user stability. A customer must not see the new catalog on one request and the old one on the next within the same session —it would be confusing and would make bugs impossible to reproduce—. You're going to see, measured, how the bucket stuck to the user guarantees that each person always falls on the same route.

Connection with the module. Lesson 4 diverted by percentage over the request; this one shows the three strategies and when to use each, with the percentage now stuck to the user. Lesson 6 uses the observability of the two routes to decide when to raise the percentage (independent of which strategy you choose); lesson 7 takes any of the three to its end and retires the legacy. Notice the boundary: here we decide how to choose the route (the diversion criterion). When it's safe to raise that diversion —with what metrics— is lesson 6. And all these strategies are variants of the same phase 3 of the strangler; the router's mechanism (facade + fallback) is that of lesson 4.

An analogy: three ways of distributing the customers between two lines

Imagine a bank with two tellers: the old teller (slow but tested) and the new teller (which you're breaking in). An employee at the entrance distributes the customers between the two lines. They have three possible criteria to distribute, and each makes sense at a different moment.

By type of transaction (by route). "Deposits, to the new teller; withdrawals, to the old one." Distributes by the type of operation. It's coarse-grained: all deposits go to the new, all withdrawals to the old. Useful when the new teller already knows how to do one type of transaction completely and well, but not yet the others.

By count (by percentage). "One in every ten customers, to the new teller; the rest, to the old one." Distributes by fraction, regardless of the transaction or the person. Useful for testing the new teller with few people first and going up little by little. But careful: if María goes, deposits, leaves, and comes back in, should she go to the same teller? Yes —so as not to confuse her—, so the distribution by count is best tied to the customer's identity, not to a die that's rolled on each visit.

By list (by feature flag). "The bank's employees and the customers who signed up for the pilot program, to the new teller; the rest, to the old one." Distributes by who the customer is. Targeted: you give the new one to people who know they're testing and who will report the problems to you. Outside the chance: it doesn't depend on the luck of the count, it depends on a list.

The employee at the entrance is the router. The type of transaction is the route; the count is the percentage; the list is the feature flag. And "that María always goes to the same teller" is the per-user stability this lesson measures.

Worked example: the three strategies over the same queue

We're going to execute the three strategies over the same queue of requests —four users, each hitting the listing (GET /products) or the detail (GET /products/42)— and see how each one decides differently. The by-percentage strategy computes the bucket on the user (not the request), so a user always falls on the same route. At the end, we verify that stability: we send the same user three times and check it doesn't jump route.

import zlib

def bucket(key):
    return zlib.crc32(str(key).encode()) % 100

# --- Three strategies to decide legacy vs modern. All deterministic. ---

# (A) By ROUTE/endpoint: the listing already goes to the new; the detail, still to the old.
def route_by_endpoint(request):
    return "modern" if request["path"] == "GET /products" else "legacy"

# (B) By PERCENTAGE/canary, stuck to the USER: the same user always falls the same
#     (doesn't jump between old and new mid-session).
def route_by_percentage(request, traffic_percent):
    return "modern" if bucket(request["user"]) < traffic_percent else "legacy"

# (C) By FEATURE FLAG/user: only the beta users see the new service.
BETA_USERS = {"u-staff", "u-beta"}
def route_by_feature_flag(request):
    return "modern" if request["user"] in BETA_USERS else "legacy"

# --- A fixed batch: 4 users, each hits listing and detail. ---
requests = [
    {"user": "u-alice", "path": "GET /products"},
    {"user": "u-alice", "path": "GET /products/42"},
    {"user": "u-beta",  "path": "GET /products"},
    {"user": "u-beta",  "path": "GET /products/42"},
    {"user": "u-bob",   "path": "GET /products"},
    {"user": "u-staff", "path": "GET /products/42"},
]

print("Same queue of requests, three diversion strategies (traffic_percent=50)\n")
print(f"{'user':<9}{'path':<20}{'by route':>10}{'by %':>8}{'by flag':>10}")
print("-" * 57)
for req in requests:
    a = route_by_endpoint(req)
    b = route_by_percentage(req, traffic_percent=50)
    c = route_by_feature_flag(req)
    print(f"{req['user']:<9}{req['path']:<20}{a:>10}{b:>8}{c:>10}")

# --- Stability: the same user, 3 requests, must not jump route. ---
print("\nPer-user stability (by %): u-alice hits 3 times in a row")
routes = [route_by_percentage({"user": "u-alice"}, 50) for _ in range(3)]
print(f"  u-alice -> {routes}   (bucket={bucket('u-alice')}, stable: {len(set(routes))==1})")
print("\n  By route: coarse-grained, a whole endpoint at a time.")
print("  By %: gradual canary; stuck to the user, no jumps mid-session.")
print("  By flag: targeted (staff/beta first), outside the chance of the percentage.")

What to expect. When you run the file, the output is exactly this:

Same queue of requests, three diversion strategies (traffic_percent=50)

user     path                  by route    by %   by flag
---------------------------------------------------------
u-alice  GET /products           modern  modern    legacy
u-alice  GET /products/42        legacy  modern    legacy
u-beta   GET /products           modern  modern    modern
u-beta   GET /products/42        legacy  modern    modern
u-bob    GET /products           modern  legacy    legacy
u-staff  GET /products/42        legacy  legacy    modern

Per-user stability (by %): u-alice hits 3 times in a row
  u-alice -> ['modern', 'modern', 'modern']   (bucket=32, stable: True)

  By route: coarse-grained, a whole endpoint at a time.
  By %: gradual canary; stuck to the user, no jumps mid-session.
  By flag: targeted (staff/beta first), outside the chance of the percentage.

Read the table by columns, because each column is a strategy with its own logic.

The by route column decides only by the path: all the requests to GET /products (the listing) go to the modern, all the ones to GET /products/42 (the detail) to the legacy. Notice that it doesn't care who makes the request or about chance: u-alice, u-beta, u-bob —all— go to the modern when they ask for the listing, and to the legacy when they ask for the detail. It's coarse-grained: you migrated the listing endpoint entirely, and the detail one not yet. Ideal when the modern already knows how to do a whole endpoint well but not others.

The by % column decides by the user's bucket with traffic_percent=50. u-alice has bucket 32 (less than 50) → always modern; u-beta has a bucket less than 50 → modern; u-bob has a bucket greater than or equal to 50 → legacy; u-staff likewise → legacy. And here's the crucial thing: look at the two rows of u-alice. In the "by route" column, u-alice goes to modern in one row and to legacy in the other (because the endpoint changed). But in the "by %" column, u-alice goes to modern in both rows —because the bucket is computed on the user, not on the request—. u-alice always falls on the same side, regardless of what they ask for. That's the per-user stability.

The by flag column decides by the list BETA_USERS = {"u-staff", "u-beta"}. Only u-beta and u-staff see the modern; u-alice and u-bob —no matter what buckets they have— go to the legacy, because they're not on the list. Notice that it's orthogonal to the percentage: u-staff goes to the legacy in the "by %" column (its bucket is high) but to the modern in the "by flag" column (it's on the beta list). The feature flag ignores chance and chooses by identity.

And below, the stability test: we send u-alice three times through the percentage strategy and get ['modern', 'modern', 'modern'] —stable: True—. Its bucket (32) doesn't change between requests, so it falls on modern all three times. If the bucket were computed on something that changes on each request (a random request id, the time), u-alice could fall on modern, then legacy, then modern —jumping route mid-session—, which is exactly what per-user stability avoids.

Deep dive: when each strategy, and why stability matters

The three strategies don't compete: they're used at different moments of the migration, and often together.

Strategy        Decides by     Granularity    Best for
──────────────  ─────────────  ─────────────  ───────────────────────────────
by route        the endpoint   coarse         migrate a whole endpoint at
                                              a time; clear boundary
by percentage   fraction of    fine           canary: raise 0->100 gradually,
                traffic (user)                random sample of the traffic
by flag         identity of    targeted       test with staff/beta first;
                user                          dogfooding before the canary

A typical sequence combines the three: first you turn on the modern by feature flag only for the staff (dogfooding: the team uses its own product and finds the obvious bugs). When the staff is happy, you open by percentage a canary of 1% of real users, and you go up 1→10→50→100. And if the modern only covers one endpoint, you use by route to send only that endpoint to the modern while the others stay in the legacy. Each strategy answers a different question: what (route), how much (percentage), who (flag).

Per-user stability deserves its own paragraph because it's where most badly designed stranglers fail. If the by-percentage diversion computes the bucket on something that changes between requests of the same user, the person jumps route:

Bucket over the REQUEST (unstable):         Bucket over the USER (stable):
  u-alice, req#1 -> bucket 12 -> modern       u-alice, req#1 -> bucket 32 -> modern
  u-alice, req#2 -> bucket 88 -> legacy       u-alice, req#2 -> bucket 32 -> modern
  u-alice, req#3 -> bucket 45 -> modern       u-alice, req#3 -> bucket 32 -> modern
  (jumps old/new mid-session)                 (always the same route)

Why does it matter so much? Three reasons. First, experience coherence: if the modern and the legacy have subtle differences (a different order, an extra field), a user who jumps between the two sees the catalog "flicker" between two versions. Second, bug reproducibility: if a user reports a problem, you need to know which route they got; if they jump on each request, the bug is impossible to reproduce. Third, session state: if the modern and the legacy handle session or cache differently, jumping between them can corrupt the user's state. The rule is strict: the bucket of the by-percentage diversion is computed on a stable identifier of the user (their id, their session), never on something that changes per request. One user, one route.

Common mistakes

Computing the percentage's bucket on the request instead of the user. What happens: the router computes the bucket with a random request id, or with the time, so the same user falls on modern sometimes and on legacy others. Why it happens: it's the easiest to write —each request brings its own id— and at 10% it "splits well" in aggregate, so the problem isn't seen in the metrics. How to spot it: a user reports that "sometimes the catalog looks different" or that a bug "appears and disappears"; on investigating, you discover they jump route between requests. How to fix it: compute the bucket on a stable identifier of the user (bucket(user_id)), not on the request. That way, even if the user makes a thousand requests, all fall on the same route as long as the traffic_percent doesn't change. Per-user stability isn't a luxury: it's what makes the experience coherent and the bugs reproducible.

Choosing the wrong strategy for the state of the modern. What happens: the team diverts by percentage (a sample of all the endpoints) when the modern only implements one endpoint well. Result: the requests to the endpoints the modern doesn't cover fall into fallback constantly. Why it happens: the percentage is the "default" strategy and it's applied without thinking about what the modern covers. How to spot it: the fallback is high for certain endpoints and low for others —a sign that the modern covers some and not others—. How to fix it: if the modern covers a whole endpoint but not others, divert by route: send only that endpoint to the modern and leave the others in the legacy until the modern implements them. The strategy must match what the modern knows how to do: by route when the coverage is per endpoint, by percentage when the modern covers everything and you want a gradual canary.

Staying forever in a feature flag "only for beta." What happens: the modern has been on for months only for the beta users, and it's never opened to the general traffic. Why it happens: the feature flag for beta is convenient and safe —nobody complains because the beta users know they're testing— and it takes away the pressure of completing the migration. How to spot it: the BETA_USERS set doesn't grow toward "everyone" over time; the migration is frozen in a "permanent pilot." How to fix it: the feature flag for staff/beta is the first phase (dogfooding), not the last. Its purpose is to find the obvious bugs with tolerant people before opening the by-percentage canary to real users. Put a date on it: "two weeks of beta, and then we open the canary at 1%." A beta flag that's never opened is another form of eternal migration —the legacy keeps serving 99% forever—, which lesson 7 combats.

Exercises

Exercise 1 — Choose the strategy. For each situation, say what diversion strategy you'd use (by route, by percentage, by feature flag) and why: (a) the modern implements the whole GET /products but not yet GET /products/{id}; (b) you want the development team to test the modern in production before anyone else; (c) the modern covers all the endpoints and you want to raise the traffic from 1% to 100% gradually and measured.

See solution
  • (a) By route. The modern covers one endpoint (GET /products) but not another (GET /products/{id}). Divert by route: send GET /products to the modern and GET /products/{id} to the legacy. That way the modern only receives the traffic it knows how to serve, without constant fallbacks in the endpoint it doesn't cover. When the modern implements the detail, you add that route.
  • (b) By feature flag. You want a specific group —the development team— to see the modern before the public. A feature flag with the internal users on the list gives the modern to them and leaves everyone else in the legacy. It's dogfooding: the team finds the obvious bugs through its own use before exposing real users.
  • (c) By percentage (canary), stuck to the user. The modern covers everything and you want a gradual canary: raise 1→10→50→100 measuring at each step. The by-percentage diversion over the user's bucket gives you exactly that: a growing fraction of the real traffic, with each user stable on their route.

In a mature migration, you'd use them in sequence: (b) feature flag for staff → (c) canary by percentage for the endpoint the modern covers, restricted (a) by route to that endpoint until the modern covers the others.

Exercise 2 — Diagnose the route jump. A user reports: "the catalog sometimes shows me the products in one order and sometimes in another, in the same session." The team uses by-percentage diversion at 50%. (a) What's the most likely cause? (b) What is the router almost certainly computing the bucket on? (c) How do you fix it, and why does that resolve the report?

See solution

(a) The most likely cause is that the user is jumping between the legacy and the modern between requests of the same session. The legacy and the modern return the products in different orders (a subtle implementation difference), so when the user falls on modern they see one order and when they fall on legacy they see another. The "sometimes one, sometimes the other" is the signature of the route jump.

(b) Almost certainly the router computes the bucket on something that changes on each request —a random request id, a timestamp— instead of on the user's identity. With traffic_percent=50, each request of the user has ~50% chance of falling on each side, independently, so over a session the user bounces between the two routes.

(c) It's fixed by computing the bucket on a stable identifier of the user (bucket(user_id) or bucket(session_id)), not on the request. That resolves the report because the user's bucket is constant: as long as the traffic_percent is 50, that user falls always on the same side (modern if their bucket < 50, legacy if not), the whole session. They stop seeing the catalog "flicker" because they stop jumping route. One user, one route.

Exercise 3 — Combine the strategies. Design, in prose, a three-phase diversion plan to migrate Mercado's catalog, using the three strategies in the correct order. For each phase, say what strategy you use, who the modern exposes to, and what signal tells you that you can move to the next phase.

See solution

A reasonable three-phase plan:

Phase 1 — Dogfooding (by feature flag). You turn on the modern only for BETA_USERS = {staff}: the development team and some internals. You expose the modern to people who know they're testing and who report the bugs directly. Signal to advance: the staff uses the modern for, say, a week without finding blocking bugs; the fallback count for the beta users is zero or nearly.

Phase 2 — Gradual canary (by percentage, stuck to the user). You open the modern to real users by percentage, starting at 1% and going up 1→10→50→100, with the bucket on the user's id (stability). Each user always falls on the same route; a growing fraction sees the modern. Signal to advance between steps: at each level, the modern's error_rate is below the threshold and the fallbacks don't grow (the metric lesson 6 formalizes with a gate). Only then do you go up to the next step.

Phase 3 — Restriction by route (if needed). If halfway through the canary you discover that the modern covers GET /products well but not yet GET /products/{id}, you combine with by-route diversion: you send GET /products to the by-percentage canary and leave GET /products/{id} 100% in the legacy until the modern implements it. Signal to advance: the modern implements the detail and its fallbacks for that endpoint drop to zero; then you add that route to the canary.

The result: you exposed the modern first to whoever tolerates it (staff), then to a growing sample of real users (canary), always respecting what the modern knows how to do (by route). When the canary reaches 100% and the fallbacks are zero, you're ready for the final cutover of lesson 7.

Summary and next step

In this lesson you opened the three ways of diverting traffic in a strangler fig: by route (by the endpoint, coarse-grained, a whole path at a time), by percentage (by a fraction of the traffic, gradual canary, stuck to the user) and by feature flag (by identity, targeted at staff and beta first). You executed them over the same queue of requests and saw how each one splits differently: the route ignores who makes the request, the percentage respects the user, and the flag ignores chance. And you measured the per-user stability: with the bucket computed on the user, u-alice fell on modern for its three requests, without jumping route —the property that makes the experience coherent and the bugs reproducible—.

Before moving on you should be able to: name the three strategies and what question each one answers (what, how much, who); choose the appropriate strategy according to what the modern covers and who you want to expose; explain why the percentage's bucket is computed on the user and not on the request; and diagnose a route jump from the symptom of "the catalog flickers."

Lesson 6 answers the question these strategies leave open: you chose how to divert, but when is it safe to raise the percentage? You're going to set up the observability of the two routes —counts, error_rate, and latency per route— and build a promotion gate that only raises the traffic_percent when the new route is healthy. You're going to see, measured, how with the buggy modern the gate doesn't promote, and how after fixing it the gate does promote —turning the decision to go up from a hunch into a criterion with numbers—.

Resources

  • Pete Hodgson, "Feature Toggles (aka Feature Flags)" — martinfowler.com/articles/feature-toggles.html. The reference article on feature flags: types of toggle, how they're implemented, and why a "release toggle" (like our by-user diversion) should be short-lived. In English.
  • Martin Fowler, "CanaryRelease" — martinfowler.com/bliki/CanaryRelease.html. The by-percentage diversion as a canary, and why the routing must be stable per user for a coherent experience. In English.
  • Sam Newman, Monolith to Microservices (O'Reilly, 2019), ch. 3 — the different ways of splitting the traffic between the monolith and the new services (by request, by user, by type of operation). In English.
  • Chris Richardson, "Pattern: Strangler application" — microservices.io/patterns/refactoring/strangler-application.html. The routing in the strangler facade and how it's decided which requests go to the new service. In English.
← PreviousDiverting traffic by percentageNext →Fallback and observing both routes