Module introduction: from a toy model to Kiosko's real data

Description

Module 1 ended with kiosko_analytics/ working end to end: dbt_project.yml, profiles.yml, a first trivial model (my_first_dbt_model, two rows written by hand in a SELECT), and the first git commit. That model served its purpose — demonstrating that dbt's whole mechanism works — but it never touched a single real piece of Kiosko data. Its two rows lived entirely inside the .sql file itself, with no FROM pointing at anything external.

This module bursts that bubble. You're going to delete models/example/ entirely and replace it with the first real contact between dbt and Kiosko's raw files: the same orders_2026-08-03.csv … orders_2026-08-09.csv, stores, products, and the events clickstream you already know from data-engineering-foundations-guide. You're going to declare those files as sources — the only entry point a dbt project recognizes for raw data — you're going to learn dbt-duckdb's specific mechanism for reading them directly from disk, without loading them first into any intermediate table, and you're going to build your project's first real layer: the staging models, one view per source table, no joins, no aggregations, just already-clean names and types.

Connection to the module. Everything you built in module 1 — dbt_project.yml, profiles.yml, the dbt debug/dbt run ritual — stays exactly the same; not a single line changes. What changes, from here on, is that models/ finally contains files that read real data. This module is, in that sense, where Kiosko's project stops being a scaffolding exercise and starts being a data project.

An analogy: the warehouse's receiving dock

Think of a large warehouse, with a single official loading dock. Everything that enters the warehouse — every box, every pallet — has to go through that dock, get logged in the receiving register (what arrived, from which supplier, on what date), and only then is it considered part of the official inventory. A new worker who needs raw material never goes out to fetch it directly from the supplier's truck parked outside; they request, inside the warehouse, material that's already been received and logged. If someone decided to skip receiving and grab boxes straight from the truck, the warehouse's inventory system never even finds out that merchandise exists — and the day the supplier switches trucks, or the truck arrives late, nobody in the warehouse has any way of knowing why pieces are missing.

source() is that single receiving dock. Kiosko's raw files — orders_2026-08-03.csv, events_2026-08-03.jsonl, stores.csv, products_v1.csv — are the trucks arriving from outside: data no dbt model produced, that exists independently of this project. Declaring them as source() is logging them in the official receiving register: it gives them a name inside the project, a description, and — as you're going to see in lesson 4 — an exact address for where to find them on disk. From there, any model that needs that raw data requests it by its registered name, never writing the file's path by hand in every place it's used. Lesson 3 goes deeper into exactly what breaks when someone decides to "grab the box straight from the truck" — that is, write a loose file path in a FROM, without going through source().

The map of this module

Lesson    Question it answers
────────  ──────────────────────────────────────────────────────────
L1        (this one) What this module is about, and why source() is the single door
L2        What raw files Kiosko has, and how they're declared as source()
L3        What the source() function does, exactly -- and why it matters
L4        How dbt-duckdb reads a CSV/JSONL directly, without loading it first
L5        What a staging model is, and why it's a view, not a table
L6        Why the convention is called stg_<entity>, not any other name
L7        Building stg_orders and stg_events for real, with dbt run
L8        Project: Kiosko's complete staging layer, all 4 tables

Notice the progression: lessons 2 through 4 solve how raw data enters the project (declaring, understanding the function, reading the physical file). Lessons 5 and 6 solve what shape the first layer you build on top of that raw data takes (one model per source table, materialized as a view, with a predictable name). And lessons 7 and 8 are where all of that becomes real, executed code, with verifiable row counts.

The data you're going to use (inherited, unchanged, from foundations)

This module doesn't invent new data. You're going to declare as sources exactly the same files data-engineering-foundations-guide already generated and that you already know:

Raw file(s)	What it contains	Rows
orders_2026-08-03.csv … orders_2026-08-09.csv (7 files, one per day)	One row per point-of-sale sale: order_id, store_id, product_id, quantity, unit_price, order_ts	40 total
events_2026-08-03.jsonl … events_2026-08-09.jsonl (7 files, one per day)	The app's clickstream: event_id, event_type (page_view/add_to_cart/purchase), session_id, event_ts	32 total
stores.csv	Catalog of the three stores: store_id, store_name, city	3
products_v1.csv	Catalog of the four products: product_id, product_name, category, unit_cost	4

The only new piece — not new data, a new column — is product_updated_at in products_v1.csv. Foundations never needed that column because it never historized anything; this guide is going to, in module 5, with a snapshot that detects changes by comparing that date. Lesson 2 explains why it makes sense to add it starting now, even though its real use starts five modules later.

This module's boundary: what you're NOT building yet

No staging model in this module does a JOIN or a GROUP BY. It's the module's hardest rule, and it's worth stating now, before writing a single line of SQL: a staging model exists to clean up — rename columns, cast types, nothing more — exactly one source table at a time. Combining orders with stores, or aggregating events by session, is the marts' job, and marts don't start until module 3, when you meet ref() and the dependency DAG dbt resolves for you. You're going to see, in lesson 5, what technically happens if you break this rule — dbt doesn't stop you, but the project becomes impossible to reason about — and why the dbt Labs team itself documents it as standard practice, not this guide's opinion.

Common mistakes

Believing "declaring a source" loads the file into a table. What happens: someone finishes lesson 2 — where orders is only declared as a source — and expects to be able to run SELECT * FROM orders directly in DuckDB, as if dbt had copied the CSV into a new table. Why it happens: in many other data tools, "registering a source" really does imply a loading step (a COPY INTO, a LOAD DATA). How to spot it: if you try to query the source's name directly outside a dbt model and it shows up in no catalog, that's the correct signal — source() doesn't create anything physical on its own, it's just a named reference. How to fix it: lesson 4 is what connects that name to a real physical file, with meta.external_location; up to that point, a declared source is just metadata in a YAML.

Writing a model that does a JOIN "because, hey, the data's already there." What happens: someone builds stg_orders.sql and, seeing that stores.csv is also declared as a source, adds a LEFT JOIN to bring in store_name right away, saving a step. Why it happens: technically the JOIN works with no error — dbt has no rule blocking it — so it feels like a reasonable optimization. How to spot it: if your file inside models/staging/ has more than one FROM/JOIN, or any GROUP BY, you've already broken this module's boundary, even if the model runs perfectly. How to fix it: lesson 5 shows, executed, exactly this case — an "incorrect" staging model that compiles and runs with no errors — so you see with your own eyes that the rule is a design one, not something dbt's syntax forces.

Exercises

Exercise 1 — Trace a piece of data's full path. Without writing code yet, describe in 2-3 sentences the complete journey a row from orders_2026-08-05.csv is going to take in this module: from the file on disk, to becoming a queryable row inside kiosko.duckdb.

See solution

The orders_2026-08-05.csv file exists on disk, generated by data-engineering-foundations-guide. This module declares it as the source source('kiosko_raw', 'orders') in sources.yml, with meta.external_location pointing at its physical path (alongside the other six daily files, via a pattern that reads all of them at once). The stg_orders.sql model selects from that source, renames/casts whatever columns need it, and when you run dbt run --select staging, dbt turns that SELECT into a real view (main.stg_orders) inside kiosko.duckdb — with the row never passing through any intermediate loading step.

Exercise 2 — Predict the row count. Using this lesson's table, how many rows do you expect to see in stg_orders once the module is complete? And in stg_events? Write down your prediction — you're going to be able to verify it literally in lesson 8.

See solution

40 rows in stg_orders (the sum of the seven daily orders files) and 32 rows in stg_events (the sum of the seven daily events files). A staging model, by doing no JOIN or GROUP BY, never loses or duplicates rows relative to its raw source — the count has to match exactly, and verifying that with a concrete number, not just the absence of an error, is the real proof the conversion was correct.

Exercise 3 — Argue the boundary in your own words. Using the receiving-dock analogy, explain in 2-3 sentences why a staging model that does a JOIN between orders and stores violates this layer's spirit, even if dbt allows it with no errors.

See solution

A staging model exists so every source table has, in the project, exactly one clean, reliable place to query it — the equivalent of every type of merchandise having its own already-labeled shelf inside the warehouse, with no mixing between products. If stg_orders already brings in store_name combined through a JOIN, any other model that only needs orders without stores no longer has a clean place to start from, and worse: if tomorrow you add a second table that also needs to cross orders with stores but in a different way, you're going to end up repeating the same JOIN in several places instead of declaring it once in a mart, where it belongs.

Summary and next step

This module closes the gap between the scaffolding you built in module 1 and Kiosko's real data. You saw the map of the 8 lessons, the four raw files you're going to declare (inherited unchanged from foundations, plus one new column in products_v1.csv that sets the stage for module 5), and the hard boundary that governs the entire staging layer: one model per source table, no joins or aggregations.

Before moving on you should be able to: explain in your own words what problem declaring a source solves versus writing a file's path by hand; and name, without looking at the table, Kiosko's four raw files and which table each one is going to form.

Lesson 2 is where the real work starts: you're going to organize Kiosko's raw files inside your project, and you're going to write this whole guide's first sources.yml.

Resources

• dbt Developer Hub — "About sources," the official reference for what a source is in dbt and why it exists as a concept separate from ref(). docs.getdbt.com/docs/build/sources. In English.
• dbt Developer Hub — "How we structure our dbt projects: staging," the official convention guide this layer follows. docs.getdbt.com/best-practices/how-we-structure/2-staging. In English.
• dbt-duckdb — official GitHub repository, with the documentation for meta.external_location, the central mechanism of lessons 3 and 4. github.com/duckdb/dbt-duckdb. In English.