Module 3: Ref Marts And Materializations

Inspecting the DAG with `dbt ls`

Next →

Description

You already used dbt ls in module 2, to confirm dbt recognized Kiosko's four sources without running anything. This lesson comes back to that same command, now applied to the complete DAG ref() built throughout this module — and adds the piece you didn't know yet: the +model and model+ operators, which ask dbt to print, literally, the dependency tree upstream or downstream of any node in the project.

By the end of this lesson you're going to be able to answer, without running a single model, two questions anyone new to a real dbt project constantly needs to answer: "what does this model need to exist?" and "what would break if I change this model?"

Connection to the module. Lessons 2 through 6 built Kiosko's DAG, piece by piece, and ran it several times with dbt run. This lesson adds no new model — it teaches you to read the structure that already exists, the same distinction between "declare/inspect" and "run" you already saw with source() in module 2.

Worked example: fact_orders's complete upstream tree

dbt ls (with the alias dbt list, identical) accepts the + operator before a model's name to ask for all its ancestors — everything that, directly or indirectly, has to exist before that model can be built:

dbt ls --select +fact_orders

What to expect.

kiosko_analytics.marts.dim_date
kiosko_analytics.marts.dim_store
kiosko_analytics.marts.fact_orders
kiosko_analytics.staging.kiosko.stg_orders
kiosko_analytics.staging.kiosko.stg_stores
source:kiosko_analytics.kiosko_raw.orders
source:kiosko_analytics.kiosko_raw.stores
kiosko_analytics.staging.kiosko.not_null_stg_orders_order_id
kiosko_analytics.staging.kiosko.not_null_stg_stores_store_id
kiosko_analytics.staging.kiosko.unique_stg_orders_order_id
kiosko_analytics.staging.kiosko.unique_stg_stores_store_id

Eleven lines, and each one is worth reading. +fact_orders included fact_orders itself, plus everything that has to exist beforehand: dim_date and dim_store (its two direct mart dependencies), stg_orders and stg_stores (which fact_orders and dim_store depend on, respectively), the two raw sources behind those two views (source:kiosko_analytics.kiosko_raw.orders, source:...stores), and — notice this — the four tests module 2 declared on order_id and store_id. dbt ls doesn't distinguish between "models" and "tests" when you ask for the complete ancestor tree: any resource that's part of the dependency chain shows up, with its own prefix (source: for sources, no prefix for models and tests).

Notice what does not show up: stg_events, stg_products, and their corresponding tests. Neither one takes part, directly or indirectly, in the chain that builds fact_orders — dbt ls --select +fact_orders confirms it with the same certainty dbt run would have used to automatically decide not to waste time building them, if you'd only asked it --select +fact_orders to run.

Worked example (continued): the downstream tree

The mirror operator, model+ (the + after the name), asks for everything that depends on that model — its descendants:

dbt ls --select fact_orders+

What to expect.

kiosko_analytics.marts.fact_orders

Just one line: fact_orders itself, nothing else. This isn't an error — it's real information: no model in the project, up to this point, depends on fact_orders. It's a leaf of the DAG, the final end of the build chain. That makes sense with what you know about the project: fact_orders is the star schema's central fact, the arrival point, not the starting point of any other model — yet. (In module 8's capstone, when Kiosko adds mart_daily_sales_obt, that new model is going to depend on fact_orders, and at that point dbt ls --select fact_orders+ is going to show two lines instead of one.)

Compare it with stg_orders, which does have descendants:

dbt ls --select stg_orders+

What to expect.

kiosko_analytics.marts.fact_orders
kiosko_analytics.staging.kiosko.stg_orders
kiosko_analytics.staging.kiosko.not_null_stg_orders_order_id
kiosko_analytics.staging.kiosko.unique_stg_orders_order_id

Three resources depend, directly or indirectly, on stg_orders: fact_orders (the only mart that uses it), and its own two tests (which, technically, depend on themselves, so they show up just as they did in +fact_orders). This answers this lesson's second promised question: "what would break if I change stg_orders?" — the answer, literal and verifiable, is "fact_orders, and nothing else" — not dim_store, not dim_date, not any of the other three staging models.

Reading the output format

Notice the three different formats that showed up in this lesson's examples:

FormatMeans
kiosko_analytics.marts.fact_ordersA model, inside models/marts/
kiosko_analytics.staging.kiosko.stg_ordersA model, inside models/staging/kiosko/ (one more folder level than the marts)
source:kiosko_analytics.kiosko_raw.ordersA source — notice the source: prefix, which no model has
kiosko_analytics.staging.kiosko.unique_stg_orders_order_idA generic test, with the same self-explanatory name format you already saw in module 2

The general pattern is project.folder[.subfolder].name — dbt reflects, in every resource's full name, the folder path where the file lives, which makes it possible to tell a staging model apart from a marts model at a glance, without opening any file.

Worked example: a narrower ancestor tree

+dim_store (without the rest of fact_orders's chain) shows only what that specific mart needs:

dbt ls --select +dim_store

What to expect.

kiosko_analytics.marts.dim_store
kiosko_analytics.staging.kiosko.stg_stores
source:kiosko_analytics.kiosko_raw.stores
kiosko_analytics.staging.kiosko.not_null_stg_stores_store_id
kiosko_analytics.staging.kiosko.unique_stg_stores_store_id

Five lines, much narrower than fact_orders's complete tree — because dim_store depends on a single chain (stg_stores → source: stores), with no crossing into stg_orders or dim_date. Comparing the size of these two trees — five lines for dim_store, eleven for fact_orders — is, on its own, a quick way to estimate how much of the project's surface depends on each model, with no need for any hand-drawn diagram.

Diagram: Kiosko's complete DAG at the end of this module

flowchart LR
    subgraph sources["sources (kiosko_raw)"]
        SO["source: orders"]
        SS["source: stores"]
    end
    subgraph staging["models/staging/kiosko/"]
        STGO["stg_orders (view)"]
        STGS["stg_stores (view)"]
    end
    subgraph marts["models/marts/"]
        DS["dim_store (table)"]
        DD["dim_date (table)"]
        FO["fact_orders (table)"]
    end
    SO --> STGO
    SS --> STGS
    STGS --> DS
    STGO --> FO
    DS --> FO
    DD --> FO

This diagram is, literally, what dbt ls --select +fact_orders showed you as text: dim_date has no arrow coming in — it depends on no source and no staging model, exactly as you learned in lesson 4 — and fact_orders is the only spot where three arrows converge at once. stg_events and stg_products don't even show up in this diagram, because this lesson only draws the subgraph that takes part in building fact_orders — the same cut +fact_orders already gave you as a text list.

Common mistakes

Confusing +model (ancestors) with model+ (descendants). What happens: someone writes dbt ls --select fact_orders+ expecting to see the whole chain that builds fact_orders, and is surprised to see a single line. Why it happens: the + symbol is the same in both cases, and only its position — before or after the name — changes the meaning; it's easy not to notice the difference at first glance. How to spot it: if model+'s result is shorter than expected (or just the model itself, as with fact_orders+ in this lesson), check whether you actually wanted the ancestor tree, not the descendant one. How to fix it: memorize the direction with a simple rule — the + points toward the side where the information you're looking for is: +model (the + "before," on the ancestors' side) brings everything that comes before; model+ (the + "after," on the descendants' side) brings everything that comes after.

Thinking dbt ls --select +fact_orders runs those models. What happens: someone runs this command expecting that, besides listing, it also builds the listed models — as if it were a shortcut for dbt run --select +fact_orders. Why it happens: both commands accept the same --select syntax, and it's easy to assume they do the same thing with different output formats. How to spot it: if you query kiosko.duckdb after a dbt ls, on a database that didn't have those models built, you still don't find them — dbt ls never opens a write connection to the warehouse, it only reads the dependency graph dbt already parsed from the project. How to fix it: use dbt ls to inspect the DAG before running anything, and dbt run/dbt build (with the same --select syntax) when you actually want to build those models — they're two separate commands, even though they share the selection syntax.

Forgetting dbt ls includes tests in the ancestor/descendant tree, and counting them as if they were models. What happens: someone counts dbt ls --select +fact_orders's lines (eleven, in this lesson's example) and concludes fact_orders depends on eleven models. Why it happens: without paying attention to the prefix or its absence, every line looks similar. How to spot it: check the names — unique_stg_orders_order_id and not_null_stg_stores_store_id are tests, not models; the real count of models in fact_orders's chain is five (fact_orders, dim_date, dim_store, stg_orders, stg_stores), plus two sources and four tests. How to fix it: if you need to count only models, add --resource-type model to the command, as you already did in this module's lesson 6 — filter the resource type before counting.

Exercises

Exercise 1 — Predict dim_date's ancestor tree. Without running anything, predict what dbt ls --select +dim_date would show. Use what you already know from lesson 4 about this model's dependencies.

See solution 
dbt ls --select +dim_date

What to expect.

kiosko_analytics.marts.dim_date

Just one line — dim_date itself, nothing else. As you learned in lesson 4, dim_date has no ref() or source() at all in its definition — it generates its own range with a recursive CTE — so it has no ancestor dbt can list. It's, in a sense, the exact counterpart of fact_orders+ (which also had no descendants): dim_date has no ancestors, fact_orders has no descendants, each one at a different end of the DAG.

Exercise 2 — Filter models only, no tests or sources. Using --resource-type model, write the command that shows only the models (not the tests or sources) in fact_orders's ancestor tree.

See solution 
dbt ls --select +fact_orders --resource-type model

What to expect.

kiosko_analytics.marts.dim_date
kiosko_analytics.marts.dim_store
kiosko_analytics.marts.fact_orders
kiosko_analytics.staging.kiosko.stg_orders
kiosko_analytics.staging.kiosko.stg_stores

Five lines — the chain's five real models — with none of the two sources or four tests that showed up in the worked example. --resource-type filters by resource type, exactly the way you already used --resource-type model in this module's lesson 6 to isolate only models when comparing materializations.

Exercise 3 — Explain, in your own words, why dbt ls --select stg_stores+ includes fact_orders. fact_orders has no direct ref('stg_stores') at all in its .sql file (it only has ref('dim_store')). In 2-3 sentences, explain why it shows up anyway in stg_stores's descendant tree.

See solution

The + operator (in either of its two forms) walks the complete DAG, not just one step's direct dependencies — fact_orders depends on dim_store, and dim_store depends on stg_stores, so an indirect but real chain exists between stg_stores and fact_orders. Since dbt ls --select stg_stores+ asks for everything that depends on stg_stores, directly or indirectly, fact_orders shows up in the result even though its .sql file never mentions stg_stores by name — the same logic would apply if you had a chain five or ten models long: the descendant tree always walks the complete chain, not a single hop.

Summary and next step

In this lesson you learned to read, with dbt ls and the +model/model+ operators, the complete DAG ref() built throughout this module — without running a single model. You confirmed fact_orders has eleven resources in its ancestor tree (five models, two sources, four tests) and none in its descendant tree — it's a DAG leaf — and that stg_orders and stg_stores do have real descendants, each one reaching all the way to fact_orders by different paths.

Before moving on you should be able to: tell +model apart from model+ from memory; and use --resource-type model to filter out tests and sources from any dependency tree you inspect.

Lesson 8 closes the module with a mini-project: rebuild Kiosko's complete star schema end to end, verify fact_orders's final count, and make the project's third version-control commit.

Resources

← PreviousChoosing materializations for KioskoNext →Mini-project: Kiosko's star schema in dbt