7. Chained timeouts and the time budget

Overview

In lesson 6 you established that your worst-case latency is the sum of your dependencies' timeouts. That word —sum— hides the problem this lesson solves. When orders makes not one call but a chain of calls —to catalog, then to payments, then to shipping—, and its own client imposed a deadline (the mobile app waits for the checkout at most 1.2 s), the sum of each link's local timeouts can exceed that deadline. When that happens, something absurd occurs: orders keeps working after its client already gave up and closed the connection. Every millisecond of that work is pure effort for the trash —a response no one will receive—, and under load, "working for the trash" is itself a source of overload.

The solution is the time budget (time budget or deadline propagation): instead of each link using its local timeout blindly, the client's deadline is propagated through the chain, and each call is bounded by the time that remains, not by its nominal timeout. If orders has 160 ms of budget left when it's about to call shipping, then the effective timeout of that call is 160 ms —even though shipping's local timeout is 300 ms—, because it makes no sense to wait 300 ms for something whose result will arrive 140 ms after the client already left. And if the budget already ran out before reaching a link, that link isn't even called: it fails fast, without wasting the attempt.

Connection with the module: this lesson closes the module's arc. Lesson 5 gave you each dependency's local timeout (its p99); lesson 6 showed you that those local timeouts sum into your worst case; this lesson solves what to do when that sum doesn't fit in the deadline you were given. The time budget is the disciplined way to chain timeouts without blowing the upward promise. It's also the mechanism by which a service tells its dependencies "how long it's worth working" —the "downward" side of lesson 6's contract—. With this, you have the complete timeout: what it is, why it matters, where to put it, what value to give it, what it promises, and how to chain it.

The trip's budget: splitting a total time among stages

Imagine you have an international flight and you leave home with exactly 3 hours before takeoff —that's your deadline, non-negotiable—. The trip has stages: taxi to the airport, check-in line, security control, walking to the gate. Each stage has a "normal" duration (the taxi usually takes 40 minutes, security usually takes 20). If you add up the normal durations and they fit in 3 hours, great. But the key is in how you handle the unexpected: if the taxi gets stuck in traffic and instead of 40 minutes takes 90, you can't afford to grant the check-in line its "normal" time as if nothing happened —you have to look at how much time you have left and adjust—. Maybe you no longer have time to buy coffee; maybe you have to go straight to security. You manage the trip against the time remaining until the deadline, not against the nominal duration of each isolated stage.

And there's a decision the analogy makes crystal clear: if halfway through the taxi you realize it's already 2:55 and the flight leaves at 3:00, it makes no sense to keep heading to the airport —you already missed the flight, better to turn around than spend more on the taxi—. That's the heart of the budget: each stage is bounded by what remains, and if nothing remains, you don't start the stage. orders is the traveler; the mobile app's deadline is the flight time; each call (catalog, payments, shipping) is a stage; and "don't keep heading to the airport if you already missed the flight" is "don't call shipping if the budget already ran out."

The problem: the sum of local timeouts blows the deadline

Let's put it in numbers. The mobile app gives orders a deadline of 1200 ms for the whole checkout. orders makes three sequential calls, each with its local timeout chosen from its p99 (lesson 5): catalog 300 ms, payments 800 ms, shipping 300 ms. The sum of the local timeouts is 1400 ms —already, from the start, 200 ms more than the deadline—. In the happy case (all respond fast) nothing happens. The problem appears in the worst case, and this is exactly the scenario an incident produces.

We model a run where catalog takes 280 ms (almost its timeout), payments takes 760 ms (slow but under its timeout), and shipping is degraded and takes its full timeout. We compare two strategies: naive, each link uses its local timeout without knowing about the deadline; and budgeted, the deadline is propagated and each link is bounded by what remains.

CLIENT_DEADLINE_MS = 1200
HOPS = [("catalog", 300, 280), ("payments", 800, 760), ("shipping", 300, 300)]
#        name       local  latency_of_this_run

def run_budgeted():
    elapsed = 0
    for name, local_to, latency in HOPS:
        remaining = CLIENT_DEADLINE_MS - elapsed
        if remaining <= 0:
            continue                      # budget exhausted: fail fast, without calling
        effective_to = min(local_to, remaining)   # bound by what remains
        spent = min(effective_to, latency)
        elapsed += spent

Worked example: naive vs budgeted

What to expect. The actual output of the two strategies:

Client deadline for checkout = 1200 ms
Sum of local timeouts        = 1400 ms  (worst case if every hop uses its own)

NAIVE: every hop uses its own local timeout, blind to the deadline
  catalog   local_timeout= 300 ms  took  280 ms (ok)   elapsed=280 ms
  payments  local_timeout= 800 ms  took  760 ms (ok)   elapsed=1040 ms
  shipping  local_timeout= 300 ms  took  300 ms (timeout)   elapsed=1340 ms
  total elapsed = 1340 ms vs deadline 1200 ms  -> OVER (client already gave up)

BUDGETED: deadline propagated; each hop timeout = min(local, remaining)
  catalog   effective_timeout= 300 ms  took  280 ms (ok)   elapsed=280 ms
  payments  effective_timeout= 800 ms  took  760 ms (ok)   elapsed=1040 ms
  shipping  effective_timeout= 160 ms  took  160 ms (timeout)   elapsed=1200 ms  <- clipped by budget
  total elapsed = 1200 ms vs deadline 1200 ms  -> WITHIN

Follow the thread. In both strategies, catalog and payments behave the same: catalog takes 280 ms (elapsed 280), payments takes 760 ms (elapsed 1040). So far that's 1040 ms of the 1200 of budget; 160 ms remain.

Now shipping, and here the two strategies diverge. Naive: shipping uses its local timeout of 300 ms without looking at the clock. shipping is degraded, so it takes the full 300 ms before timing out. Total elapsed: 1040 + 300 = 1340 ms. But the client had a deadline of 1200 ms: at 1200 ms the mobile app already gave up, closed the connection, and showed the user an error. The 140 ms orders spent waiting for shipping after the 1200 were work for the trash —orders held a thread, occupied a connection to shipping, and produced a result that arrived 140 ms late to a client that no longer existed—.

Budgeted: before calling shipping, orders computes the remaining time: 1200 − 1040 = 160 ms. The effective timeout of the call to shipping is min(300, 160) = 160 ms —clipped by the budget—. shipping times out at 160 ms instead of at 300, and the total elapsed is exactly 1200 ms: right inside the deadline. orders failed the call to shipping 140 ms earlier, yes —but those 140 ms were going to be work for the trash anyway—, and in exchange orders responded within its contract, without wasting the thread or the connection on a response no one was waiting for.

The difference —1340 ms vs 1200 ms— seems small in one request, but multiply it by thousands of checkouts during a shipping incident: the naive strategy has orders doing wasted work en masse, holding threads 140 ms extra for each one, right when the pool is already under pressure. The budget cuts that waste at the root.

Fail fast when the budget already ran out

The example showed the clipping (min(local, remaining)). The other half of the budget is missing, which is even more powerful: if the budget already ran out, don't call at all. In the code, that's the line if remaining <= 0: continue. Imagine that payments had taken not 760 ms but 1150 ms; on reaching shipping, the remaining time would be 1200 − 1190 = 10 ms, or even negative. Calling shipping with a 10 ms timeout is guaranteeing an immediate timeout —shipping doesn't respond in 10 ms even when healthy—, so it's a call doomed in advance. Worse: even a doomed call opens a connection, occupies a pool slot, gives shipping work. The budget says: if no useful time remains, don't even try; fail immediately with "deadline exceeded" and release everything.

This connects directly with the "downward" side of lesson 6's contract. The deadline isn't only used inside orders; it's propagated to shipping —typically in a header like X-Request-Deadline or the deadline mechanism of the RPC framework—. So shipping receives "you have 160 ms left" and can make the same decision: if its own internal work doesn't fit in 160 ms, shipping abandons early instead of working for a response that orders will discard. The budget is split across the whole chain: each link knows how much real time the end user has left, and none spends effort on expired work. This is what mature RPC systems do (gRPC propagates deadlines natively) and what Google's SRE Book describes as deadline propagation.

The budget and the timeout's other forces

It's worth seeing how the budget interacts with what you learned before, because sometimes it pulls in the opposite direction and the tension has to be resolved with judgment.

The local timeout comes from the dependency's p99 (lesson 5): it's the minimum time you must grant it so as not to cut healthy traffic. The budget, on the other hand, is a maximum imposed from above: it's all the time you have left. When the remaining budget is less than the next dependency's p99, you have a real conflict: clipping the timeout to remaining (as the budget does) means you'll cut even healthy responses of that dependency —because remaining < p99 implies more than half the healthy responses don't fit—. In the example, shipping received 160 ms when its p99 is higher; many of its healthy responses wouldn't have fit. That's not a bug in the budget; it's information: it's telling you that this chain doesn't honestly fit in this deadline, and that failing the last call is preferable to breaching the contract with the user.

When that conflict is frequent —not a rare worst-case, but something that happens often—, the answer isn't "clip the timeout more" (you'd cut healthy traffic constantly) or "ignore the deadline" (you'd breach the SLA), but redesign the chain: parallelize the independent calls (so their cost is the max, not the sum), take off the critical path whatever can be deferred (create shipping's shipment asynchronously after confirming the order —degradation, module 7—), or renegotiate the deadline with the client. The budget doesn't create the problem; it reveals it with numbers, and forces you to solve it in the design instead of hiding it behind timeouts that don't fit.

Common mistakes

Using local timeouts blindly, ignoring the client's deadline. What happens: each call uses its nominal timeout without subtracting the time already spent, so the chain can take the sum of all the timeouts —more than the deadline—. Why it happens: each call is configured in isolation, without a view of the total. How to spot it: orders keeps working (busy threads, open connections to dependencies) on requests whose client already closed the connection; metrics of "responses produced after the client's deadline." How to fix it: propagate the deadline and compute each effective timeout as min(local, remaining); clip the calls by the remaining time, not by their nominal value.

Calling a dependency with the budget already exhausted. What happens: even if no useful time remains, the call is made "just in case it responds fast," guaranteeing an immediate timeout but opening the connection and giving the dependency work. Why it happens: the "if remaining <= 0, don't call" cutoff isn't implemented. How to spot it: calls with a tiny effective timeout (≤ a few tens of ms) that almost always time out —work doomed in advance—. How to fix it: if the remaining budget is less than a reasonable minimum (or zero), fail fast with "deadline exceeded" without making the call; you save the connection, the pool slot, and the dependency's work.

Not propagating the deadline downward. What happens: orders respects the budget internally but calls shipping without telling it how much time remains, so shipping works believing it has all the time in the world. Why it happens: propagating the deadline requires passing it explicitly (header, RPC context) and coordinating it between services. How to spot it: shipping keeps processing and consuming resources on requests that orders already abandoned by timeout —wasted work in the dependency—. How to fix it: propagate the deadline on each hop (an X-Request-Deadline header, or the RPC framework's native mechanism like gRPC's deadlines); each link bounds its work by the real remaining time and abandons the expired. The budget belongs to the whole chain, not to a single service.

Exercises

Exercise 1 — Compute the effective timeout. The app gives orders a deadline of 1000 ms. orders calls catalog (local 300, takes 250), then payments (local 800, takes 500), then shipping (local 300). With a budget, what's the effective timeout of the call to shipping? What would happen if payments had taken 780 ms?

See solution

Base case: after catalog (250 ms) and payments (500 ms), the elapsed is 750 ms. The remaining time on reaching shipping is 1000 − 750 = 250 ms. shipping's effective timeout is min(local=300, remaining=250) = 250 ms. It's clipped by the budget: even though shipping's local timeout is 300 ms, only 250 ms are granted because that's all that remains before the client's deadline.

If payments had taken 780 ms: the elapsed after catalog (250) + payments (780) would be 1030 ms —the 1000 ms deadline already passed—. The remaining time on reaching shipping is 1000 − 1030 = −30 ms (negative). With a budget, the if remaining <= 0 rule fires: orders doesn't call shipping at all; it fails fast with "deadline exceeded," without opening the connection or giving shipping work. Calling shipping at that point would be doomed work (the client already gave up at 1000 ms), so the budget avoids it entirely. Note the lesson's nuance: here the budget is clipping below what shipping needs to respond healthily —a sign that, if this happens often, the chain doesn't fit in 1000 ms and needs a redesign—.

Exercise 2 — Parallelize to fit. orders has a deadline of 900 ms and makes three calls: catalog (250 ms), fraud-check (300 ms), and payments (500 ms). In sequence, the worst case is 1050 ms —doesn't fit—. Knowing that catalog and fraud-check are independent of each other (neither needs the other's result), propose a redesign that fits in 900 ms and compute the new worst case.

See solution

In sequence, the worst case is the sum: 250 + 300 + 500 = 1050 ms, which exceeds the 900 ms deadline. The budget would reveal the problem by clipping payments (or some call) below the healthy value, or failing the last call.

Redesign: parallelize the independent calls. Since catalog and fraud-check don't depend on each other, they're launched in parallel. Their contribution to the worst case is no longer the sum (250 + 300 = 550) but the max (max(250, 300) = 300 ms), because both run at once and you wait for the slower one to finish. Then, with the result of both, payments is called (500 ms).

New worst case: max(catalog, fraud-check) + payments = 300 + 500 = 800 ms. Now it fits in 900 ms, with 100 ms of slack.

This is exactly the kind of redesign the "budget < p99" conflict pushes you to do: when the sequential sum doesn't fit, parallelizing the independent calls turns a sum into a max, which is usually much smaller. (If catalog and fraud-check depended on each other, they couldn't be parallelized and you'd have to find another way out: defer, tighten, or renegotiate the deadline.)

Exercise 3 — Work for the trash. Explain, in terms of lesson 3's pool exhaustion, why the naive strategy (which takes 1340 ms when the deadline was 1200 ms) is especially harmful during an incident of shipping, and how the budget mitigates it.

See solution

During a shipping incident, shipping is degraded and takes its full timeout on every call. With the naive strategy, each checkout that reaches the shipping call waits 300 ms —140 ms of which are after the client already gave up at 1200 ms—. Those 140 ms extra are an orders thread held and a connection to shipping occupied, producing a response no one will receive. Multiplied by thousands of simultaneous checkouts during the incident, it's a huge amount of extra held threads and wasted connections, right when the pool is already under the pressure of the slow shipping (lesson 3). The work for the trash consumes the same scarce capacity that pool exhaustion drains, worsening the cascade.

The budget mitigates it in two ways. First, it clips shipping's timeout to remaining (160 ms in the example), releasing the thread 140 ms earlier per request —that recovered capacity, summed over thousands of requests, relieves the pool—. Second, and more powerful: when the budget runs out before reaching shipping, it doesn't even make the call, so it doesn't open the connection or give work to a shipping that's already overloaded —stopping hammering the degraded dependency gives it space to recover—. The budget turns "working for the trash under pressure" into "failing fast and releasing resources," which is exactly what a pool under stress needs. (Stopping hammering the dead dependency altogether is, taken to the extreme, module 5's circuit breaker.)

Closing the timeout arc

With the time budget, the timeout module is complete. You walked the whole pattern: what it is (a limit on the wait that aborts and releases), why it matters enough to be the first defense (the measured pool exhaustion), where to put it (connect and read, in the socket), what value to give it (the p99 + margin), what it promises outward (the time contract, your worst case), and how to chain it without blowing the client's deadline (the budget, min(local, remaining) and the fail-fast when no time remains). You have the timeout not as a loose configuration line, but as a system of judged decisions.

What comes next is applying it with your own hands. Lesson 8 puts you in front of Mercado's checkout without timeouts —the fragile version—, has you measure its exhausted pool and its down traffic, and then armor it: connection and read timeouts chosen from the p99, with a budget in the chain, measuring the before and after. You deliver the table of numbers that proves the pattern worked and the justification of each value you chose.

Summary and next step

In this lesson you took the timeout to a chain of calls and solved the problem the sum of local timeouts creates: when that sum exceeds the client's deadline, the naive strategy makes orders keep working after its client gave up —work for the trash—. You measured it: with a 1200 ms deadline, the naive strategy takes 1340 ms (140 ms extra, wasted), while the budget clips the last call to min(local, remaining) = 160 ms and finishes in exactly 1200 ms, within the contract.

You learned the two halves of the budget: clip each effective timeout by the remaining time (min(local, remaining)), and fail-fast when the budget already ran out (if remaining <= 0, don't call), saving the connection and the dependency's work. And you saw that the deadline is propagated downward (a header, gRPC's native deadlines) so each link abandons the expired work. Above all, you understood that when the remaining budget falls below the next dependency's p99, that's not a bug: it's the signal that the chain doesn't fit in the deadline and needs a redesign —parallelize the independent, defer the deferrable, or renegotiate the SLA—.

Before moving on you should be able to: compute a link's effective timeout given a deadline and the time already spent; explain why the naive strategy wastes resources during an incident; and describe how a deadline propagates through a chain.

What comes next is the capstone. Lesson 8 gives you Mercado's checkout without timeouts, has you measure its fragility, and guides you to armor it with everything from the module —connect + read, value from the p99, budget in the chain— measuring the before and after. It's where the pattern stops being reading and becomes yours.

Resources

• Google SRE Book, "Addressing Cascading Failures" and the material on deadline propagation — sre.google/sre-book/addressing-cascading-failures. Describes why propagating deadlines through the call chain avoids wasted work and cascades, the direct reference for this lesson's budget. Free and in English.
• gRPC documentation: "Deadlines" — grpc.io/docs/guides/deadlines. Shows how a mature RPC framework propagates deadlines natively between services, instead of isolated local timeouts —the budget implemented in the transport—. In English.
• Marc Brooker, "Timeouts, retries, and backoff with jitter", Amazon Builders' Library — aws.amazon.com/builders-library/timeouts-retries-and-backoff-with-jitter. Covers the relationship between timeouts, deadlines, and the total latency of a composite operation. Free and in English.
• Michael T. Nygard, Release It!, 2nd ed. (Pragmatic Bookshelf, 2018) — the Timeouts pattern and its interaction with call chains and end-to-end latency budgets. In English.