Module 1: Your First Dbt Project

Connecting to Kiosko with `profiles.yml`

Next →

Description

profiles.yml is the file where dbt stores how to connect to a warehouse — the equivalent of lesson 5's electrical-connection paperwork analogy. Its structure is the same no matter the adapter: a profile name (which has to match dbt_project.yml's profile field), one or more targets (environments, like dev or prod), and inside each target, the warehouse's type and the credentials specific to that type. For DuckDB, those "credentials" are almost ridiculously simple compared to a cloud warehouse: no user, no password, no host — just a path to a file.

This lesson writes that file for Kiosko, explains why DuckDB needs so little to connect, and runs dbt debug for the first time with both files complete — the first time in this guide you're going to see All checks passed!.

Connection to the module. Lesson 5 left dbt debug failing at one exact point: profiles.yml file [ERROR not found]. This lesson closes that exact gap. By the end, your kiosko_analytics/ project is going to have all three pieces complete — dbt_project.yml, profiles.yml, models/ — and you're going to be ready to write your first real model in lesson 7.

An analogy: the delivery address, not the whole factory

Think about ordering something by mail within your own city, versus ordering something that crosses international borders. An international order needs customs, a tracking number validated by two countries, declaration forms — a lot of paperwork, because the package crosses different jurisdictions, each with its own rules. An order within your own city needs, basically, an address: the street, the number, and that's it — no border to cross, no customs, no two different systems negotiating with each other.

Connecting to a cloud warehouse (Snowflake, BigQuery, Redshift) is the international order: user, password, account, region, role — each credential exists because your laptop and the warehouse are completely different systems, on different machines, that need to authenticate to each other before exchanging anything. Connecting to DuckDB is the local order: DuckDB isn't a server somewhere else you have to authenticate to — it's a library that runs inside the same process as dbt, and the "warehouse" is, literally, a file on your disk. The only "address" needed is the path to that file. That's not less security — it's simply a different architecture: nothing crosses any network boundary.

Worked example: writing profiles.yml for Kiosko

Inside the same kiosko_analytics/ folder (next to dbt_project.yml, not inside models/), create profiles.yml:

# profiles.yml
kiosko_analytics:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: kiosko.duckdb
      threads: 4

Each line answers a specific question:

  • kiosko_analytics: (top level) — the profile's name. It has to match, character for character, the profile: "kiosko_analytics" field you already wrote in dbt_project.yml (lesson 5) — it's the only real coupling point between the two files.
  • target: dev — which of the environments defined under outputs to use by default when you run any command without specifying otherwise. A real project usually has several targets (dev, prod), each pointing at a different database; this guide uses only dev throughout all eight lessons.
  • type: duckdb — tells dbt-core which adapter to use to interpret the rest of this section. It's the field that only makes sense thanks to dbt-duckdb, the package you installed in lesson 4.
  • path: kiosko.duckdb — the path to the database file. It's a path relative to the folder you run the dbt command from — if the file doesn't exist yet, DuckDB creates it automatically on the first connection; there's no need to "create the database" with any prior step.
  • threads: 4 — how many models dbt can run in parallel when they have no dependencies on each other. It's not exclusive to DuckDB — it's a setting common to almost every dbt adapter.

Where dbt looks for profiles.yml, and why this guide keeps it next to the project

By convention, dbt looks for profiles.yml at ~/.dbt/profiles.yml — a folder in your user directory, outside any project versioned with git. That convention exists for a concrete security reason: in a real warehouse, profiles.yml contains passwords and access tokens, and that file should never end up in a git repository where anyone with access to the code could read the credentials.

Kiosko breaks that convention on purpose, for a different reason: as you saw in this lesson's analogy, DuckDB's profile has no secret in it — it's just a file path — so keeping profiles.yml inside the project's own folder makes kiosko_analytics/ fully self-contained and portable, ideal for a local learning project. To get dbt to look there instead of at ~/.dbt/, you have to tell it explicitly, with the DBT_PROFILES_DIR environment variable pointing at the project's folder:

export DBT_PROFILES_DIR=$(pwd)   # run from inside kiosko_analytics/

(The alternative is passing --profiles-dir . on every dbt command; DBT_PROFILES_DIR avoids repeating it. You're going to need this variable exported in every new terminal session where you work on the project, just like you need to reactivate kiosko_env — lesson 8 reminds you of this in the mini-project.)

Worked example (continued): dbt debug, complete this time

With DBT_PROFILES_DIR exported and profiles.yml already written, run the same command from lesson 5:

dbt debug

What to expect.

Running with dbt=1.12.2
dbt version: 1.12.2
python version: 3.12.3
os info: macOS-15.0.1-arm64-arm-64bit
Using profiles dir at /path/to/your/kiosko_analytics
Using profiles.yml file at /path/to/your/kiosko_analytics/profiles.yml
Using dbt_project.yml file at /path/to/your/kiosko_analytics/dbt_project.yml
adapter type: duckdb
adapter version: 1.11.0
Configuration:
  profiles.yml file [OK found and valid]
  dbt_project.yml file [OK found and valid]
Required dependencies:
 - git [OK found]

Connection:
  database: kiosko
  schema: main
  path: kiosko.duckdb
  config_options: None
  extensions: None
  settings: {}
  external_root: .
  use_credential_provider: None
  attach: None
  filesystems: None
  remote: None
  plugins: None
  disable_transactions: False
Registered adapter: duckdb=1.11.0
  Connection test: [OK connection ok]

All checks passed!

Now both Configuration checks are OK, and a new section shows up — Connection — that didn't exist in lesson 5, because now there's an actual profile to test. Notice database: kiosko: dbt-duckdb took the file's name (kiosko.duckdb, without the extension) as the database name inside the catalog — you never declared it anywhere, it's an adapter convention. Connection test: [OK connection ok] is the real test: dbt opened a real connection to the kiosko.duckdb file (which, if it didn't exist, DuckDB just created empty in this very step) and confirmed it can read from and write to it. The final line, All checks passed!, is the one you're going to look for every time you connect a dbt project to a new warehouse, in this guide or in any real project.

Confirm with your own file system that the file showed up:

ls -la kiosko.duckdb

You're going to see a file a few kilobytes in size — DuckDB reserves that minimal space from the first connection, even with no table created yet. The exact size may vary slightly depending on your DuckDB version; what matters isn't the byte count, but that the file exists — the first tangible proof that your dbt project is already talking to a real warehouse.

Common mistakes

A profile name that doesn't match between the two files. What happens: profiles.yml has a top-level key different from what profile: declares in dbt_project.yml — for example, kiosko_warehouse in one file and kiosko_analytics in the other. Why it happens: it's the same kind of typo as a misspelled variable name in code — silent until something tries to use it. How to spot it: dbt debug marks profiles.yml file [ERROR invalid], and the specific message literally says:

Profile loading failed for the following reason:
Runtime Error
  Could not find profile named 'kiosko_analytics'

How to fix it: check that profiles.yml's top-level key is exactly the same text as dbt_project.yml's profile: field — case-sensitive, no extra spaces. There's no other way dbt can "guess" which profile corresponds to which project; the text match is the mechanism.

Forgetting to export DBT_PROFILES_DIR in a new terminal. What happens: someone completes this lesson successfully, closes the terminal, opens it again the next day, runs dbt debug from inside kiosko_analytics/, and sees profiles.yml file [ERROR not found] again — the same error from lesson 5, even though the file really does exist. Why it happens: export sets an environment variable only for the current terminal session; it doesn't persist between sessions, just like activating a Python virtual environment. How to spot it: if profiles.yml exists in the folder but dbt debug still can't find it, suspect DBT_PROFILES_DIR first before touching the file. How to fix it: run export DBT_PROFILES_DIR=$(pwd) again from inside kiosko_analytics/ at the start of every new session — along with activating kiosko_env, this is going to be your two-command ritual every time you start working on the project.

Confusing path:'s relative path when you change folders. What happens: someone runs dbt run from a folder other than the project root (for example, from inside models/), and dbt creates a new, empty kiosko.duckdb in that wrong location, instead of using the one that already existed. Why it happens: path: kiosko.duckdb in profiles.yml is a relative path, and a relative path always resolves against the directory you run the command from, not against the location of profiles.yml itself. How to spot it: if your queries suddenly can't find tables you know you already created, check how many different kiosko.duckdb files exist in your project with find . -name "*.duckdb" — it's common to find more than one because of this mistake. How to fix it: always run dbt commands from the project root (kiosko_analytics/, where dbt_project.yml lives) — it's actually dbt's general requirement, not something specific to this guide.

Exercises

Exercise 1 — Diagnose the exact error. A teammate has this profiles.yml:

kiosko_warehouse:
  target: dev
  outputs:
    dev:
      type: duckdb
      path: kiosko.duckdb

And this dbt_project.yml (identical to lesson 5's, unchanged). When running dbt debug, exactly which line is going to fail, and with what message?

See solution

profiles.yml file [ERROR invalid] is going to fail, with the message Could not find profile named 'kiosko_analytics'. dbt_project.yml declares profile: "kiosko_analytics", but this profiles.yml's top-level key is kiosko_warehouse — they don't match, so dbt finds no connection configuration for the profile the project is asking for, even though the profiles.yml file itself is syntactically valid.

Exercise 2 — Change path and observe the result. Temporarily modify your profiles.yml so path points to otro_kiosko.duckdb instead of kiosko.duckdb, run dbt debug, and then check with ls what .duckdb files exist in your folder.

See solution

dbt debug passes every check the same as before — All checks passed! — because DuckDB doesn't need the file to exist beforehand: if otro_kiosko.duckdb doesn't exist, the connection creates it automatically, empty, at that moment. Running ls *.duckdb you're going to see two different files: kiosko.duckdb (the original, from earlier lessons) and otro_kiosko.duckdb (the new, empty one). This demonstrates that path isn't validated against something that already has to exist — it's, literally, the path where DuckDB is going to read and write, whether it exists yet or not. Set path: kiosko.duckdb back before continuing, and you can delete otro_kiosko.duckdb if you want to keep the folder clean.

Exercise 3 — Explain, in your own words, why DuckDB doesn't need a user or password. Using this lesson's analogy (the local order versus the international order), explain in 2-3 sentences why profiles.yml for DuckDB is so short compared to what, for example, Snowflake would need.

See solution

DuckDB isn't a separate server dbt connects to over a network — it's a library that runs inside the same Python process running dbt, and the "warehouse" is, literally, a file on the local disk. There's no network boundary to cross and no other system that needs to verify who you are, so there's no need for a user, password, or token — the only information dbt needs is where that file is (path). A warehouse like Snowflake, on the other hand, lives on a remote server operated by another company, and any connection from your laptop has to authenticate to that external system before it can read or write anything — hence the user, password, account, and role a Snowflake profiles.yml really does need.

Summary and next step

In this lesson you wrote profiles.yml for Kiosko, with a profile named kiosko_analytics that exactly matches dbt_project.yml's profile:, pointing at a local kiosko.duckdb file. You ran dbt debug with all three pieces complete — dbt_project.yml, profiles.yml, models/ — and saw, for the first time in this guide, All checks passed!: confirmation that dbt has a real, working connection to a warehouse. You also understood why DuckDB's profile is so simple compared to a cloud warehouse, and saw the exact error a mismatched profile name between the two files produces.

Before moving on you should be able to: write, from memory, the minimal structure of a profiles.yml for DuckDB; and explain exactly what DBT_PROFILES_DIR does and why this guide needs it.

With the connection working end to end, lesson 7 writes your first real dbt model — a .sql file inside models/ — and runs dbt run for the first time in this entire guide.

Resources

  • dbt Developer Hub — "About profiles.yml," the official reference for this file's complete structure, including the explanation of target and outputs. docs.getdbt.com/docs/core/connect-data-platform/profiles.yml. In English.
  • dbt Developer Hub — "Configure your profile," the page explaining where dbt looks for profiles.yml by default and how to use --profiles-dir / DBT_PROFILES_DIR to change it — the same URL that appears in this lesson's error message. docs.getdbt.com/docs/configure-your-profile. In English.
  • dbt-duckdb — official GitHub repository, with the complete list of configuration keys supported for type: duckdb (path, threads, extensions, settings, among others). github.com/duckdb/dbt-duckdb. In English.
← PreviousAnatomy of a dbt projectNext →Your first model and `dbt run`