When a dbt project needs a scheduler

Description

This lesson doesn't teach any new dbt command — it names, precisely, something you already lived through twice with nobody calling it by name. In module 6's mini-project, you ran dbt build with no --vars and saw ERROR=1, SKIP=7. In this module's lesson 5, you ran dbt docs generate with no --vars and saw the exact same error, word for word. Both cases share a cause: someone — you — has to remember, every time, to pass --vars '{"run_date": "..."}' with the correct date, before running any command that compiles fact_orders. As long as it's you running every command from your own terminal, forgetting it produces a visible, immediate, easy-to-fix error. This lesson explains why that same guarantee stops holding up as soon as someone else — or nothing, an automated process with no human supervision — has to run those commands every day, with no exceptions, with nobody watching.

Connection to the module. Lessons 2 through 6 built new pieces of the project — a macro, documentation, a lineage artifact. This lesson doesn't build anything: it gathers the evidence you already generated in this module and in module 6, and uses it to trace this whole guide's exact boundary. Module 6 already warned about this at its own boundary: "who decides when to run each run_date... is a scheduler's job." This lesson delivers on that promise with double evidence, not just a warning.

An analogy: the alarm you have to set yourself, every night

An alarm clock that doesn't program itself — one of those old, wind-up clocks, with no "repeat every day" function — works perfectly well as long as someone remembers to wind it and set the time every night, without missing a single one. The alarm clock was never the problem: it's a simple, reliable tool that does exactly what's asked of it. The problem shows up the day that person forgets, gets sick, or simply isn't there — and nobody set the alarm that night, so it doesn't go off the next morning, with no warning at all that something failed. An alarm clock with a programmable schedule solves a different problem: it doesn't get triggered by a person who remembers every night, it gets fired by a mechanism that was already configured once, and that keeps working exactly the same whether the person is awake, asleep, or on vacation. dbt build --vars '{"run_date": "..."}', run by hand from your terminal, is the wind-up alarm clock: it works perfectly as long as you're the one setting it. A scheduler is the programmable alarm.

The evidence: the same lapse, twice, in two different commands

Before naming the solution, it's worth seeing the complete problem, together, once. With fact_orders already built (dbt build --vars '{"run_date": "2026-08-09"}', the state it was left in at the end of lesson 5), run the two commands you already broke earlier, this time one after the other:

dbt build

What to expect (shortened — you already saw the complete detail in module 6):

16 of 23 ERROR creating sql incremental model main.fact_orders ................. [ERROR in 0.01s]
17 of 23 SKIP test accepted_values_fact_orders_product_id__P001__P002__P003__P004  [SKIP]
[... 6 more tests in SKIP ...]

[ERROR]: in model fact_orders (models/marts/fact_orders.sql)
  Compilation Error in model fact_orders (models/marts/fact_orders.sql)
  Required var 'run_date' not found in config:
  Vars supplied to fact_orders = {}

Done. PASS=15 WARN=0 ERROR=1 SKIP=7 NO-OP=0 REUSED=0 TOTAL=23

dbt docs generate

What to expect.

Found 7 models, 15 data tests, 1 snapshot, 4 sources, 502 macros

[ERROR]: Encountered an error:
Runtime Error
  Compilation Error in model fact_orders (models/marts/fact_orders.sql)
    Required var 'run_date' not found in config:
    Vars supplied to fact_orders = {}

Two commands, one single error text: Required var 'run_date' not found. It's not a coincidence — it's the same root cause, evaluated at the exact same point in the code ({% if is_incremental() %}), triggered by any command that needs to compile fact_orders.sql with the table already existing. This repetition is, precisely, this lesson's central evidence: Kiosko's project already has, today, an operational contract — "whoever runs it has to know to pass --vars with the correct date" — and that contract depends entirely on a person remembering it, every time, with no mechanism reminding them.

What a human has to remember today, with no scheduler at all

Take an honest inventory of everything correctly running Kiosko by hand, every day, requires:

1. Knowing which run_date matches today — nobody tells you; you have to calculate it or remember it.
2. Remembering --vars, with the exact JSON syntax — a single misplaced character ({"run_date": "2026-08-09" with no closing brace, for example) produces a different syntax error, one you have to diagnose again.
3. Knowing when to use --full-refresh instead — the three scenarios from module 6's lesson 7, which nobody evaluates automatically for you.
4. Running the commands in the right order — dbt build before dbt docs generate, if you want the catalog to reflect fresh data (this module's lesson 5).
5. Noticing whether something failed — by watching the terminal at the exact moment the command ran, because nothing warns you afterward if you're not watching.
6. Retrying by hand if something fails halfway through — with no mechanism retrying for you, or knowing which step to resume from.

None of these six points is hard to remember once. The problem isn't difficulty — it's that the complete guarantee depends, with no backup at all, on one specific person remembering it every single time, indefinitely, with no human slip-up in between.

What a scheduler adds, without building one here

A scheduler — dbt Cloud, or Airflow running dbt build as a task in a larger DAG — doesn't change a single line of fact_orders.sql, or of any macro, or of any test. What it adds is the layer that replaces each of the six points above with a mechanism, not with a person who remembers:

• Scheduling. The command fires on its own, at a fixed time or based on an event, with nobody needing to watch the clock or the terminal.
• Automatic partition calculation. The scheduler can pass run_date calculated from when the task runs, instead of a person typing it by hand every time — solving, along the way, points 1 and 2 from the list above.
• Managed retries. If a run fails — a Compilation Error, a network drop, an ERROR like the one you already saw twice in this lesson — the scheduler can automatically retry, a configured number of times, before telling a human something needs real attention.
• Sensors and cross-system dependencies. A scheduler can wait for a new file to arrive, or for another system to finish its own work, before triggering dbt build — something no dbt command, run by hand, can coordinate on its own.
• Alerts. If something fails and the retries run out, someone finds out — a message, an email, an alert — with nobody needing to be watching the terminal at the exact moment of the failure.
• Separate environments. A scheduler typically distinguishes between a development run and a production one, with its own credentials and configuration for each — a distinction this project's single-target profiles.yml (dev, since module 1) never needed to solve.

None of these six pieces get built in this guide. Naming them, with concrete evidence of why they're needed, is exactly where this guide's work ends and the next one's begins.

The explicit bridge: airflow-and-declarative-orchestration-guide

Everything you just saw — the dbt build you ran by hand since module 1, the --vars you had to remember since module 6, the dbt docs generate that fails the exact same way in this module's lesson 5 — is the literal starting point of airflow-and-declarative-orchestration-guide, the next guide in the data-engineering-ecosystem. That guide takes this exact same dbt build — without changing a single line of code — and turns it into a task in an Airflow DAG: with run_date calculated automatically from when the task runs (never typed by hand), retries configured if something fails, sensors that wait for Kiosko's raw data to really exist before triggering the run, and alerts if something keeps failing after the retries. You're not going to build any of that here — this lesson exists so you recognize, with evidence you've already lived through, the exact moment it's needed.

flowchart LR
    A["You, by hand, in your terminal\n(modules 1-7 of this guide)"] -->|"remember --vars\nevery run"| B["dbt build\nERROR if you forget"]
    B -->|"explicit bridge"| C["airflow-and-declarative-\norchestration-guide"]
    C --> D["automatic run_date\nretries, sensors, alerts"]

Common mistakes

Thinking a cron job solves the whole problem. What happens: someone, seeing the list of what a scheduler adds, concludes it's enough to schedule dbt build --vars '{"run_date": "..."}' in a crontab, with the date calculated by a simple script. Why it happens: cron does solve the scheduling — the list's first point — and that seems sufficient at a glance. How to spot it: ask yourself what happens if that run fails at 3 AM, with nobody watching — cron doesn't retry, doesn't send any alert on its own, and knows nothing about whether some other system Kiosko depends on already finished its own work. How to fix it: cron solves exactly one of this lesson's six points (scheduling) — a real scheduler like Airflow solves all six at once, with retries, sensors, and alerts as part of the same system, not as pieces you'd have to assemble yourself around cron.

Confusing "scheduling a run" with "orchestrating a pipeline". What happens: someone puts together a bash script that runs dbt build with today's date, and calls it the project's "scheduler." Why it happens: the script does automate the command, and automating feels like the whole goal. How to spot it: that script doesn't know what to do if dbt build fails halfway through — it doesn't retry, it doesn't distinguish a transient error (a network drop) from a real one (corrupted data) — and it doesn't coordinate with any other system Kiosko depends on in a real scenario, like a pipeline that loads the raw files before dbt reads them. How to fix it: automating a command is a small fraction of what a scheduler does — the real difference is in how it handles failures, cross-system dependencies, and visibility into what happened, not just in the command firing with no human involvement.

Believing this lesson means running dbt build by hand was a mistake throughout the whole guide. What happens: someone, reading about the limitations of running everything by hand, concludes modules 1 through 7 taught a bad practice that should have been avoided from the start. Why it happens: it's easy to read a lesson about limitations as a retroactive criticism of everything before it. How to spot it: go back to module 1's introduction — this whole guide's focus was always transformation as versioned, tested, documented code, never production orchestration, which the guide's design named as a boundary since the first module. How to fix it: running dbt build by hand, from your terminal, is exactly the correct way to learn and develop a dbt project — it's how any real analytics engineer tests their changes before a scheduler runs them in production. This lesson doesn't invalidate that work: it names the next step, which is a different problem (running that same project with no human supervision), not a correction of what you already did.

Exercises

Exercise 1 — Match each point on the list with the mechanism that solves it. Without looking at this lesson's section, for each of the six points in "What a human has to remember today," write in one sentence which of a scheduler's six pieces solves it.

See solution

1. "Knowing which run_date matches today" → automatic partition calculation.
2. "Remembering --vars, with the exact JSON syntax" → automatic partition calculation (the scheduler never types --vars by hand, it builds it on its own).
3. "Knowing when to use --full-refresh" → this stays, even with a scheduler, as a decision someone explicitly configures — no automatic mechanism guesses when you changed a model's logic — though the scheduler can schedule periodic --full-refresh runs if configured to do so.
4. "Running the commands in the right order" → dependencies between tasks in the scheduler's own DAG.
5. "Noticing whether something failed" → alerts.
6. "Retrying by hand if something fails" → managed retries.

Exercise 2 — Compare Required var 'run_date' not found against an error a scheduler would prevent. In 2-3 sentences, explain why this specific error — the one you've already seen three times in this guide — is exactly the kind of problem that disappears with a scheduler that calculates run_date automatically, with no person ever typing it by hand.

See solution

The Required var 'run_date' not found error happens, in all three cases you already saw (module 6, and this lesson's two times), because a person forgot to type --vars when running the command — it's, precisely, a human lapse, not an error in fact_orders.sql's code, which is perfectly well written. A scheduler that calculates run_date automatically — from the task's scheduled date, with no person typing it — eliminates the very possibility of this error: there's no manual step where anyone could forget anything, because that step no longer exists as a human action.

Exercise 3 — Argue why this guide doesn't build any Airflow DAG, even though you already know one is needed. In 2-3 sentences, using the module's boundary, explain why it makes sense for data-engineering-foundations-guide, data-modeling-for-analytics-guide, and this whole guide to leave orchestration for a separate guide, instead of mixing it into each one.

See solution

Every guide in the ecosystem has a deliberately narrow focus — this guide teaches turning a dimensional model into a versioned, tested, documented dbt project, not operating that project in production — and mixing Airflow in the middle would dilute that focus with no added clarity: learning ref(), tests, and macros is already enough new content per guide. Also, orchestration is a problem that applies equally to any data pipeline, not just dbt projects — foundations would also end up needing to schedule its Python scripts, data-modeling would also need to automate its queries — so it makes more sense to teach it once, cross-cutting, in a dedicated guide any earlier guide in the ecosystem can point to as its natural next step.

Summary and next step

This lesson didn't build anything new — it gathered the evidence you already generated in module 6 and in this very lesson: the same error, Required var 'run_date' not found, triggered by dbt build and by dbt docs generate, every time someone forgets to pass --vars. You named the six points a human has to remember today to correctly run Kiosko by hand, and the six pieces a scheduler adds so you stop depending on someone remembering them: scheduling, automatic partition calculation, retries, sensors, alerts, and separate environments. And you named, by its exact name, where this guide points when that stops being enough: airflow-and-declarative-orchestration-guide.

Before moving on you should be able to: explain, in your own words, the difference between "scheduling a command" (what cron does) and "orchestrating a pipeline" (what Airflow does); and name the six points a human has to remember today to correctly run fact_orders, with no scheduler at all.

Lesson 8 closes the module with the mini-project: everything built in lessons 2 through 7, run end to end over the complete project, with Kiosko's seventh version-control commit.

Resources

• dbt Developer Hub — "About incremental models," "Refreshing incremental models" section, already cited in module 6 and in this module's lesson 3, the reference for why run_date is required once fact_orders is incremental. docs.getdbt.com/docs/build/incremental-models. In English.
• dbt Labs — "What is analytics engineering?," already cited in module 1, with its distinction between development (what this guide teaches) and production operation (what airflow-and-declarative-orchestration-guide solves). getdbt.com/blog/what-is-analytics-engineering. In English.
• airflow-and-declarative-orchestration-guide, the next guide in the ecosystem — this lesson's explicit destination, where this guide's same dbt build becomes a task in a DAG with retries, sensors, and alerts.