Module 6: Finops For Tokens

1. Module introduction: the cost gate already exists, tokens are what's new

Description

finops-and-cost-guardrails-guide already left a complete, finished, and verified cost gate for Andes Cargo: cost-policy/ (sibling of policy/, zero shared files), Infracost 2.16.1 already installed with its authentication limit already documented, COST-PROFILE.md with the volume assumptions for the three original billable resources (S3, Lambda, DynamoDB), and three CI jobs — cost-estimate, cost-check, cost-tags — that already ran, for real, against the complete project. This module doesn't reinstall any of that. It extends it with the one piece that gate, as it stands, can't solve on its own: the budget for a usage-based token workload, where there isn't even a Terraform resource representing "one invocation" for Infracost to read.

Connection to the module

This same guide's Module 2 already built two pieces this module reuses without rewriting: scripts/bedrock_cost_estimate.py (the deterministic per-token cost calculator, with its pytest suite) and GENAI-COST-PROFILE.md (the document that declared the chosen model, the cited price, and two volume scenarios — realistic and stress — for extract-shipment-manifest-fields). Module 5 already extended the inherited security gate with bedrock-least-privilege.rego, added to policy/, with no new job. This module does, in the cost domain, exactly what Module 5 already did in the security domain: it adds a new policy to a directory that already exists (cost-policy/), and a new execution piece (the token budget) to a job that already exists (cost-check) — never a new lane, never a new tool.

Analogy: the token budget is the mobile-data cap, but with no "plan" to represent it

You already know a cell phone plan's data cap: you contract "10 GB per month," the carrier knows exactly how much you've used at any moment, and warns you — or cuts you off — before you overbill. That cap exists because there's a plan: an explicit contract, with a fixed number, the whole system can query at any moment. The token budget this module builds is similar to that cap in purpose — warn, or cut off, before the bill surprises anyone —, but similar to prepaid phone service from twenty years ago in mechanics: no system in AWS has any "plan" declared, ahead of time, saying "Andes Cargo is going to consume X tokens this month." Nobody knows until someone explicitly declares it, by hand, in a document — exactly the gap GENAI-COST-PROFILE.md (Module 2) already filled with a declared assumption, and that this module turns, for the first time, into a number a pipeline can automatically enforce, not just read.

   MOBILE DATA CAP                          BEDROCK TOKEN BUDGET

   The carrier KNOWS your usage in          NO AWS system knows, ahead
   real time -- there's a native            of time, how many tokens you're
   counter for the service                  going to consume -- there's no
        │                                   native counter a pipeline can read
        ▼                                   before the bill already exists
   The "plan" (10 GB) is a fact                    │
   the system already knows                        ▼
        │                                   The "plan" (the budget) is a
        ▼                                   NUMBER A HUMAN DECLARES BY HAND
   Cuts off or warns automatically           (GENAI-COST-PROFILE.md, Module 2)
   before overbilling                              │
                                                    ▼
                                              This module builds the mechanism
                                              that enforces THAT declared number
                                              -- before `apply`, not after

What this module inherits, without reinstalling anything

Piece	Origin	What this module does with it
`cost-policy/` (directory, sibling of `policy/`)	`finops-and-cost-guardrails-guide`, Module 4	Adds `bedrock-budget.rego` (lesson 3) — a new file, zero existing files touched
`required-cost-tags.rego`	`finops-and-cost-guardrails-guide`, Module 4, lesson 4	Stays exactly the same — still evaluates `aws_s3_bucket`/`aws_lambda_function`/`aws_dynamodb_table`, never `aws_bedrock_guardrail`
Infracost `2.16.1`, installed with its authentication limit documented	`finops-and-cost-guardrails-guide`, Module 2	Not reinstalled. This module confirms, again, why Infracost can't solve the token budget on its own — lesson 2 explains it with new technical precision
`COST-PROFILE.md` (the three original billable resources)	`finops-and-cost-guardrails-guide`, Module 1	Stays intact. `GENAI-COST-PROFILE.md` (below) complements it, never replaces it
`scripts/bedrock_cost_estimate.py` + `pytest`	This same course, Module 2, lesson 7	Gains one new argument, `--monthly-budget` (lesson 4) — the same file, extended, never rewritten
`GENAI-COST-PROFILE.md`	This same course, Module 2, lesson 8	Its stress scenario (5,000 invocations/month) is the volume the pipeline checks against the budget (lessons 4 and 6)
`cost-estimate`/`cost-check`/`cost-tags` jobs in `ci.yml`	`finops-and-cost-guardrails-guide`, Modules 3 and 4	`cost-tags` evaluates `bedrock-budget.rego` automatically, with no YAML change (the same mechanism `policy-check` used with `bedrock-least-privilege.rego` in Module 5). `cost-check` gains one new step (lesson 4) — never a new job
The complete security gate (`policy-check`/`iac-scan`/`verify-artifact`)	This guide's Module 5	Keeps running in parallel, with no cross `needs:` toward the cost lane — lesson 8 confirms this with real `act`

Notice the pattern that repeats in every row: this module never creates a new directory, never installs a new tool, never opens a new CI job. Every new piece enters a place that already exists, following exactly the same "extend, don't rebuild" discipline Module 5 already demonstrated with the security gate.

Map of the eight lessons

  M6.1  Introduction (this lesson) -- what it inherits, what's missing
  M6.2  Why a token is the most extreme usage-based resource -- there
        isn't even a Terraform resource for "one invocation"
  M6.3  Hands-on: bedrock-budget.rego, added to cost-policy/
        -- EXECUTED, real FAIL against the current state
  M6.4  Hands-on: the per-token cost calculator, integrated
        into the gate -- EXECUTED, new --monthly-budget, real pytest
  M6.5  Cost allocation tagging for the AI workload -- EXECUTED,
        lesson 3's FAIL turns into a real PASS
  M6.6  Hands-on: a volume change that triggers the budget
        -- EXECUTED, lesson 4's step really fails
  M6.7  On-demand vs. Provisioned Throughput -- real numbers,
        verified at this very moment against AWS
  M6.8  Project: Andes Cargo's cost gate, extended to tokens
        -- EXECUTED, act pull_request in parallel with the security gate

Lessons 3 through 6 are pure hands-on, each building on the previous: the policy (3), the calculator integrated as a step (4), the really tagged HCL that closes lesson 3's FAIL (5), and the proof, in both directions, that the budget cuts off before apply (6). Lesson 7 is the module's only purely conceptual one — but with real numbers, not a rule of thumb. Lesson 8, the project, brings it all together with real act.

Why this matters: the cost gate, until now, never saw a real usage-based resource

It's worth stating this precisely, because it's the central tension that opens this module: the three billable resources finops-and-cost-guardrails-guide protected (S3, Lambda, DynamoDB) share something Bedrock does not have — each of them, at some point in its Terraform configuration, declares an attribute that comes close to "how much you're going to use this" (billing_mode = "PAY_PER_REQUEST" on DynamoDB, for example, is at least a mode declared in the HCL, even though the volume itself still lives in infracost-usage.yml). Bedrock, billed on-demand, has no equivalent attribute in bedrock.tf — not even a mode. Lesson 2 develops this difference with complete technical precision; for now, it's enough to understand this module exists exactly for that difference: the gate that already works for "how much capacity you reserved" can't, by design, work the same way for "how many times you're going to call a model" — and this module builds the mechanism that can.

Common mistakes

Assuming this module needs its own Infracost, its own version, or its own session (expecting reinstallation). What happens: someone, seeing that this module works with cost, looks for instructions to install Infracost again, or to authenticate a different session. How to spot it: if you're looking for an installation command in this module before lesson 4. How to fix it: no such command exists — Infracost 2.16.1 is already installed since finops-and-cost-guardrails-guide Module 2, and the authentication limit (infracost auth login, browser PKCE, never completed in this environment) is already documented there. This module doesn't repeat that installation anywhere.

Confusing GENAI-COST-PROFILE.md with COST-PROFILE.md, or thinking one replaces the other. What happens: someone, needing a volume assumption, doesn't know which of the two documents to look in. How to spot it: if your PR modifies COST-PROFILE.md to add a token assumption, or GENAI-COST-PROFILE.md to add an S3 assumption. How to fix it: COST-PROFILE.md (finops-and-cost-guardrails-guide, Module 1) covers Andes Cargo's three "classic" billable resources. GENAI-COST-PROFILE.md (this course, Module 2, lesson 8) covers, exclusively, extract-shipment-manifest-fields's per-token cost. They're complementary, never overlapping — the exact same relationship this guide's Module 2, lesson 8 already established in its Step 1.

Expecting cost-tags to need a YAML change to evaluate bedrock-budget.rego (not understanding -p cost-policy/). What happens: someone, while writing lesson 3's policy, looks for where in ci.yml the new file needs to be "registered." How to spot it: if you edit .github/workflows/ci.yml before lesson 4. How to fix it: conftest test tfplan.json -p cost-policy/ (the command cost-tags already runs, unchanged, since finops-and-cost-guardrails-guide Module 4) points to the complete directory — any new .rego file inside cost-policy/ gets evaluated automatically, the next time the job runs, without touching a single line of YAML. Exactly the same mechanism policy-check already demonstrated with bedrock-least-privilege.rego in Module 5.

Exercises

Exercise 1 — List, from memory, the three pieces this module inherits from finops-and-cost-guardrails-guide without reinstalling, and the two it inherits from this same guide's Module 2. Don't look at this lesson's table until you've attempted the complete list.

See solution

From finops-and-cost-guardrails-guide: cost-policy/ (with required-cost-tags.rego already inside), Infracost 2.16.1 already installed (with its authentication limit already documented), and the cost-estimate/cost-check/cost-tags jobs in ci.yml. From this same guide's Module 2: scripts/bedrock_cost_estimate.py (with its pytest suite) and GENAI-COST-PROFILE.md. If your list has all five pieces, with no sixth one invented, you have a clear grip on the exact base this whole module is built on.

Exercise 2 — Explain why required-cost-tags.rego (inherited, unchanged) never evaluates aws_bedrock_guardrail, and which policy does. Based on what you already know from finops-and-cost-guardrails-guide Module 4, why did that policy's billable_resource_types set never include a Bedrock guardrail's resource type?

See solution

Because required-cost-tags.rego was written in finops-and-cost-guardrails-guide, before this guide existed — at that point, Andes Cargo had no generative AI workload at all, so its billable_resource_types set (aws_s3_bucket, aws_lambda_function, aws_dynamodb_table) precisely reflected the only three billable resource types that existed then. aws_bedrock_guardrail is a resource type that file never had a reason to know about. The policy that does evaluate it is the one this module writes in lesson 3 — bedrock-budget.rego, a new file, never a modification to the inherited one.

Exercise 3 — Predict what would happen if, by mistake, someone wrote bedrock-budget.rego inside policy/ instead of cost-policy/. Using what you already know about the sibling-directory architecture (finops-and-cost-guardrails-guide Module 4, lesson 1; this guide's Module 5, lesson 3), which CI job would start evaluating that policy without anyone asking it to?

See solution

policy-check — the job that runs conftest test tfplan.json -p policy/, evaluating recursively every .rego file inside that directory, including any cost policy placed there by mistake. cost-tags, on the other hand, would never see it, because it points exclusively to -p cost-policy/. The result would be a cost-allocation policy being evaluated inside the security gate — a domain mix exactly the kind the sibling-directory architecture exists to prevent, the same lesson finops-and-cost-guardrails-guide Module 4, lesson 1 already previewed with its own "discarded option" diagram.

Summary and next step

This lesson mapped the module's eight lessons and confirmed, table by table, exactly what it inherits without reinstalling: cost-policy/ and the three jobs from finops-and-cost-guardrails-guide, plus bedrock_cost_estimate.py and GENAI-COST-PROFILE.md from this same guide. No new tool, no new job, no new directory — just two new files (bedrock-budget.rego, a --monthly-budget added to the existing script) in places that already exist.

Before moving on you should be able to: name the five inherited pieces without help; explain why required-cost-tags.rego never needs to change for this module to work; and predict which CI job would catch a misplaced policy.

Lesson 2 steps away from code for a moment to develop, with complete technical precision, why a token is the ecosystem's most extreme resource in terms of usage-based billing — one step further than what DynamoDB's PAY_PER_REQUEST already showed.

Resources

finops-and-cost-guardrails-guide, Module 4, lesson 1 — the origin of the cost-policy/ architecture as a sibling of policy/, which this module extends.
finops-and-cost-guardrails-guide, Modules 2 and 3 — the origin of Infracost 2.16.1 and the cost-estimate/cost-check jobs.
This course, Module 2, lessons 7 and 8 — the origin of bedrock_cost_estimate.py and GENAI-COST-PROFILE.md.
This course, Module 5, lesson 3 — the exact precedent of "a new file in an inherited directory, with no new job," applied there to security.

← Previous8. Project: Andes Cargo's security gate, extended Next →2. Why a token is the ecosystem's most extreme *usage-based* resource