Module 3: Exporting, Normalizing, and Structuring the Repository

7. Automating export with a script

Next →

Description

By the end of this lesson you will be able to run all the work from lessons 2 through 5 —exporting the workflows, normalizing them, renaming them, and dropping them into their folder— with a single command, thanks to a shell script. You'll know how that script is put together, how to run it before every commit, and how, if you want, to hook it to Git so it fires on its own. You'll also learn about the Enterprise alternative —n8n's native Git version control— and have an honest criterion for deciding when the script-based flow is enough and when it's worth paying for it.

This matters because a manual process, however well you understand it, is a process that sometimes doesn't get done. You already saw this in lesson 1: a backup that depends on you remembering to run five commands is, in practice, a backup that gets skipped the day you're busy. Automating turns "remember to do five things in order" into "run one command," and that difference is what makes the repo actually stay current, not just current in principle. It's the step that turns discipline into a habit that costs nothing.

Connection to the module: lessons 2, 3, and 4 taught you the pieces —exporting, protecting secrets, normalizing; 5 and 6 gave them shape and documentation. This lesson joins the mechanical pieces into a single machine: the export.sh script that orchestrates lesson 2's export command and lesson 4's normalizer, leaving the repository structured the way lesson 5 defined. It's the second-to-last stop: after this, in lesson 8 you're going to use this same script to produce the module's project deliverable. The script you write here is a tool that's going to stay with you beyond the guide.

Important reminder about n8n 2.0's restrictions. Throughout this guide we keep insisting that inside n8n 2.0's Code node you can't use require, access the filesystem, or run system commands. None of that applies here. This lesson's script is not a Code node: it's a shell script that runs in your terminal, on your computer, entirely outside n8n. There you have full access to git, n8n's CLI, jq, reading and writing files, everything. The Code node's restriction is on the code that runs inside a workflow; this script lives outside, and that's why it can do what a workflow can't.

Why a script, and what it is exactly

So far, every time you wanted to update the repository, you did a sequence: export with docker exec ... export:workflow, pull the files out of the container, normalize each with jq, rename them to readable names. Five or six commands, in order, without getting a single one wrong. It works, but it has two problems: it's tedious, and it's fragile —skip or swap one step and the result comes out wrong.

A script solves both. A script is nothing more than a text file containing a list of commands, in order, for the machine to run one after another when you ask it to. It's exactly the same sequence you'd do by hand, but written once and saved, so you never type it again. Think of it as a recipe's list of steps: instead of remembering "first export, then normalize, then rename" from memory every time, you write it once in the recipe and afterward you just say "follow the recipe."

The kind of script we're going to write is a shell script, or bash script. "Shell" is the name of the program that interprets the commands you type in the terminal —the same one that understands cd, ls, git. A shell script is a file with those same commands, which the shell reads and runs top to bottom. If you know how to type commands in the terminal, you already know almost everything needed to write a script: it's the same thing, saved in a file.

Anatomy of a shell script

Before writing ours, let's look at the pieces any shell script is made of, because there are few of them and they repeat in every one.

The shebang. A script's first line is usually this:

#!/usr/bin/env bash

It's called a shebang (after the #! symbols), and it tells the system which program to run the file with. Translated: "to run this, use bash." The /usr/bin/env bash is a portable way of saying "find bash wherever it's installed." Without this line, the system wouldn't know the file is a bash script. It's the label that says "this reads with bash."

The safety line. Right after, this line, which looks cryptic and is one of the most useful:

set -euo pipefail

These are three protections bundled together that make the script fail early and loudly instead of pressing on with a hidden error:

  • -e — exit on error: if any command fails, the script stops right there instead of continuing. Without this, an error in the export step would still go on to the normalize step over empty data.
  • -u — unset: if you use a variable that doesn't exist (from a typo, say), the script stops instead of silently using an empty value.
  • -o pipefail — if you chain commands with | (pipe) and one in the middle fails, the whole chain is considered failed, instead of the error getting lost.

The analogy: it's the script's seatbelt. It doesn't change what it does when everything goes fine, but when something goes wrong, it stops you cold instead of letting you keep driving toward the crash. Put it in all your scripts.

Variables. Storing a value under a name to reuse it:

CONTAINER="n8n"
OUT_DIR="./workflows"

They're assigned with no spaces around the =, and used with a $ in front: $CONTAINER, $OUT_DIR. They keep you from repeating the same value all over the script and let you change it in one place. If tomorrow your container has a different name, you change one line, not ten.

Commands and loops. The rest of the script is the commands you already know —docker exec, jq, mkdir— plus, whenever something needs repeating across several files, a for loop that says "for each file in this folder, do this." We'll see it in the example.

With these pieces you can already read any shell script. Now let's write ours.

The export.sh script

This is the lesson's heart: a script that does, in one pass, all the work of exporting and normalizing Cumbre's repository. It goes in scripts/export.sh, the spot lesson 5 reserved for it.

#!/usr/bin/env bash
# export.sh — exports, normalizes, and leaves the cumbre-automations repo ready to commit.
# Runs in your terminal, OUTSIDE n8n. Requires: docker, jq.
set -euo pipefail

# --- Configuration (adjust to your instance) ---
CONTAINER="n8n"                          # container name (check with: docker ps)
OUT_DIR="./workflows"                    # final, versioned folder
RAW_DIR="./.export-raw"                  # temp folder for the raw output (deleted at the end)
IN_CONTAINER="/tmp/wf-export"            # where the CLI exports to inside the container

# Volatile fields we remove for clean diffs (lesson 4).
VOLATILE='del(.pinData, .versionId, .active, .triggerCount, .meta.instanceId)'

# --- Step 1: export all workflows inside the container ---
echo "1/4  Exporting workflows..."
rm -rf "$RAW_DIR" && mkdir -p "$RAW_DIR"
docker exec -u node "$CONTAINER" n8n export:workflow --all --separate --pretty --output="$IN_CONTAINER"

# --- Step 2: bring the files from the container to your machine ---
echo "2/4  Copying from the container..."
docker cp "$CONTAINER:$IN_CONTAINER/." "$RAW_DIR/"

# --- Step 3: normalize and rename each one to its readable name ---
echo "3/4  Normalizing and renaming..."
mkdir -p "$OUT_DIR"
for f in "$RAW_DIR"/*.json; do
  name=$(jq -r '.name' "$f")             # reads the workflow's real name from the JSON
  slug=$(echo "$name" | tr '[:upper:] ' '[:lower:]-')   # to lowercase; spaces -> hyphens
  jq -S "$VOLATILE" "$f" > "$OUT_DIR/$slug.json"         # normalizes and writes with a readable name
  echo "     - $slug.json"
done

# --- Step 4: clean up the temp folder ---
echo "4/4  Cleaning up..."
rm -rf "$RAW_DIR"

echo "Done. Review the result with:  git status  and  git diff"

Let's read it block by block, because each one is a previous lesson turned into code:

The configuration. The four variables at the top gather everything that changes between machines into one place: the container's name, the folders. If your instance has a different name, you touch one line. The VOLATILE variable stores lesson 4's jq filter —the same five fields we remove— so it doesn't get repeated.

Step 1 is lesson 2's export command: docker export:workflow --all --separate --pretty. Notice one change from when you ran it by hand: here it's docker exec -u node without -it. Remember from lesson 2 that -it gave you an interactive terminal; in a script, which runs on its own with nobody typing, we don't want an interactive terminal, so we drop it. -u node stays, because permissions still matter. It exports inside the container, to /tmp/wf-export.

Step 2 resolves the Docker detail from lesson 2 —the files land inside the container— with docker cp, which brings them to your machine, into the temporary .export-raw folder.

Step 3 is the densest one, and it's lesson 4 plus 5 combined. The loop for f in "$RAW_DIR"/*.json says "for each .json file in the raw folder, call it f and do the following." Inside, three things: it reads the workflow's real name with jq -r '.name' (the -r gives the text without quotes); it turns it into a readable kebab-case slug with tr (lowercase, spaces to hyphens); and it normalizes with jq -S "$VOLATILE", writing the result straight to workflows/<slug>.json. In a single pass, each workflow ends up normalized and with a readable name. This is where lesson 5's ugly id-name problem disappears: the script reads the name from inside the JSON and uses it.

Step 4 deletes the temp folder, so no trash is left behind.

Notice the elegance of what we achieved: six lines of commands that by hand were a whole work session now run on their own, always the same way. The script is lesson 1's reproducibility made real.

Making it executable and running it

A freshly created script is just a text file; it needs execute permission before the system treats it as a program:

chmod +x scripts/export.sh

chmod +x means "make it executable." It's a one-time step: once marked, it stays that way. Then you run it:

./scripts/export.sh

The ./ in front tells the shell "the script is here, in this folder." One detail that confuses people: the paths inside the script (./workflows, ./.export-raw) are relative to where you're standing when you run it, not to where the script lives. So always run it from the repository's root (cumbre-automations), not from inside scripts/. If you run it from somewhere else, the folders are going to land where you don't expect. A more advanced script anchors itself to its own location so it doesn't depend on this, but to start, the simple rule —"run from the repo root"— is enough. What to expect: the terminal prints the four steps with their messages —1/4 Exporting..., and at the end the list of files written and Done— and your workflows/ folder ends up with the normalized, well-named workflows. Now you run git status and git diff to review what changed, and you commit, calmly, whatever you want. The script prepares; you decide what enters the history.

A script that warns you when something's missing

The export.sh above assumes docker and jq are installed. If they aren't, the script is going to fail partway through with a confusing error —"command not found"— that doesn't tell the user what to install. A good script checks its dependencies up front and warns clearly if something's missing, instead of crashing halfway. It's a courtesy to whoever runs it, including you six months from now. It's added right after set -euo pipefail:

# --- Dependency check ---
for cmd in docker jq; do
  if ! command -v "$cmd" >/dev/null 2>&1; then
    echo "Error: missing '$cmd'. Install it before running this script." >&2
    exit 1
  fi
done

Read it: for each tool the script needs (docker, jq), command -v "$cmd" asks "does this command exist?" If it does not exist (the ! flips the answer), it prints a clear error naming what's missing —the >&2 sends the message to the error channel, the convention for messages that aren't normal output— and exit 1 stops the script immediately. The 1 is a nonzero exit code, which by convention means "I finished badly"; a 0 would mean "everything's fine."

The difference between a script with this check and one without is the difference between an error that says "jq is missing, install it" and one that says "line 23: jq: command not found" in the middle of a half-finished run. The first gets fixed in a minute; the second sends people searching the internet. Anticipating the stumble and warning clearly is the same voice as the installation guides, applied to code.

Running it before every commit: by hand or with a hook

The discipline this script enables is simple: before committing a workflow change, run export.sh. You edit in n8n's editor, run the script, review the diff, commit. That way the repository always reflects the instance's real state, normalized and clean.

There are two ways to make sure that "before every commit" actually happens.

Way 1 — by hand, as a habit. You simply run it yourself before committing. It's the simplest, the most transparent, and what I recommend to start. The cost: it depends on you remembering. The upside: total control, no surprises.

Way 2 — a pre-commit git hook. Git lets you trigger a script automatically at certain moments; those triggers are called hooks. The pre-commit hook runs right before a commit gets finalized. If you put your script there, it runs on its own every time you commit. It lives in a .git/hooks/pre-commit file:

#!/usr/bin/env bash
# .git/hooks/pre-commit — re-exports and normalizes before every commit.
./scripts/export.sh
git add workflows/

(You also have to make it executable with chmod +x .git/hooks/pre-commit.)

It sounds ideal, but it has two traps you should know before adopting it, because not everyone mentions them:

Trap 1: hooks don't get versioned. The .git/hooks/ folder lives inside .git/, which Git doesn't version —it's the repo's internal machinery, not its content. That means your hook exists only on your machine; if a teammate clones the repo, they don't have it. There are tools for sharing hooks across a team (like setting core.hooksPath to a versioned folder, or using a hooks manager), but it's extra complexity worth weighing.

Trap 2: re-exporting on commit can bring surprises. The hook re-exports from the live instance at the moment of the commit. If between when you edited and when you commit someone else touched another workflow on the instance, the hook is going to drag it into your commit without you expecting it. It's subtle and confusing.

That's why my honest recommendation is: start with Way 1, by hand. Run the script as a deliberate, conscious step, not as magic happening behind your back. The automatic hook is an optimization that makes sense once the flow is already well-oiled and you understand its traps well, not before.

The convenient alternative: a Makefile or an npm script

Typing ./scripts/export.sh isn't hard, but there are ways to give it an even shorter, more memorable name, especially if the repo is going to have several scripts.

A Makefile. make is a veteran tool that runs named "targets" defined in a Makefile file. With this in the root:

export:
	./scripts/export.sh

.PHONY: export

you run the script with just make export. (The command line needs a tab at the start, not spaces —make is strict about that; and .PHONY tells make that export is an action, not a file it produces.)

An npm script. If your project already uses Node, you can define the script in package.json:

{
  "scripts": {
    "export": "./scripts/export.sh"
  }
}

and run it with npm run export. The advantage of these two forms is that they give you a uniform vocabulary —make export, make normalize, make check— easy to remember and to document in the README. They don't change what the script does; they give it a more comfortable handle. For Cumbre, with a single script, running it directly with ./scripts/export.sh is perfectly enough; the Makefile starts paying off once you have three or four repeated tasks.

The Enterprise alternative: n8n's native Git

Everything we've built in this module —exporting via CLI, normalizing, structuring, automating with a script— achieves the market's result, "version-controlled, documented JSON," using only the free, self-hosted Community edition. But let's be honest: n8n offers a feature that does part of this without you writing a single command, and it lives in the Enterprise plan.

It's called Git version control (n8n's "environments / source control"). It connects your instance directly to a Git repository and lets you push and pull the workflows from n8n's own interface, with buttons. The instance knows how to talk to Git natively: no CLI, no script, no docker cp. It's more convenient, and it's built for teams that promote changes between environments frequently.

The honest question is: when is it worth paying for? Here's the criterion, with no marketing spin in either direction:

SituationWhat's best
You're learning, or it's a personal project / a small client'sCLI + script flow. Free, and you produce exactly the same artifact: versioned, normalized, documented JSON.
A small team, occasional changes, tight budgetCLI + script flow. The script automates the tedious part; Enterprise's extra convenience doesn't justify the cost yet.
A team promoting changes between dev/staging/prod daily, with many peopleConsider Enterprise. Once the friction of coordinating manual exports among several people exceeds the license cost, native Git pays for itself in saved time.
Audit requirements, fine-grained access control, complianceEvaluate Enterprise. It brings guarantees the manual flow doesn't provide on its own.

The conclusion isn't "the CLI flow is for people who can't afford better" nor "Enterprise is an unnecessary luxury." It's that both produce the same deliverable, and the difference is convenience and scale, not capability. Knowing how to do the script-based flow means you understand exactly what native Git does under the hood —and that understanding serves you whether you never pay for Enterprise or you end up administering it someday. Module 6 revisits this decision in more detail, with cross-environment promotion already on the table.

Common mistakes

Confusing the shell script with a Code node and avoiding system tools (conceptual). What happens: someone, conditioned by n8n 2.0's Code node restrictions, avoids using docker, jq, or reading files in their export.sh, believing they're forbidden. Why it happens: the guide insists so much on the Code node's restrictions that it's easy to believe they're universal. How to spot it: if you're limiting what your script can do "just in case," check where it runs. How to fix it: the script runs in your terminal, outside n8n; there you have full access to the system. The Code node's restrictions only apply to code inside a workflow. Use system tools freely.

Forgetting chmod +x and hitting "permission denied" (practical). What happens: you write the script, run ./scripts/export.sh, and the system responds "permission denied." Why it happens: a freshly created file doesn't have execute permission; the system doesn't treat it as a program until you grant it. How to spot it: the "permission denied" message when running your own script is almost always this. How to fix it: chmod +x scripts/export.sh once, and you're done. It's a step people forget often the first few times and then it becomes automatic.

Writing output over input inside the loop (practical). What happens: in the normalize step, someone does jq ... "$f" > "$f" —reading and writing the same file— and empties it, the same trap as lesson 4. Why it happens: it's the same > redirection error that empties the file before jq reads it. How to spot it: if after running the script the files end up empty, this is it. How to fix it: the script writes to a different folder ($OUT_DIR) from the one it reads ($RAW_DIR), precisely to avoid this. Never read and write the same file in a single command; keep input and output separate.

Adopting the git hook before understanding its traps (conceptual). What happens: someone puts export.sh in a pre-commit hook on day one, and gets confused when a commit drags in changes to workflows they didn't touch, or when a teammate who cloned the repo doesn't have the hook and their repo gets out of sync. Why it happens: the hook sounds like "total automation" and gets adopted without knowing its two traps —it doesn't get versioned, and it re-exports the live state. How to spot it: if unexpected changes show up in your commits, or if the hook doesn't exist for your team, those are the known traps. How to fix it: start by running the script by hand as a deliberate step; adopt the hook only once you understand and accept its two limitations.

Exercises

Exercise 1 — Read the script. Without running it, read this lesson's export.sh and answer: (a) what does the set -euo pipefail line do and why is it worth it? (b) why does the script use docker exec -u node without -it, unlike when you ran it by hand? (c) where does the script get each file's readable name from?

See solution

(a) set -euo pipefail makes the script fail early and loudly: -e stops it if a command fails, -u if you use a nonexistent variable, -o pipefail if a command fails in the middle of a pipe. It's worth it because it stops an error in the export step from silently continuing into normalizing over empty data. It's the script's seatbelt.

(b) Because -i and -t request an interactive terminal, meant for when you are typing and watching the output. In a script, which runs on its own with nobody at the keyboard, an interactive terminal doesn't make sense and can cause problems. -u node stays because permissions still matter.

(c) From the JSON itself: jq -r '.name' "$f" reads the name field from inside each exported file, which is the workflow's real name in n8n. Then it converts it to kebab-case with tr. That's how it solves lesson 5's ugly id-name problem, reading the good name from inside.

Why it works: if you could answer all three, you can already read a shell script, which is 80% of knowing how to write them. A script has no magic: it's the same sequence of commands you'd do by hand, with a seatbelt and a loop for repeating.

Exercise 2 — Decide CLI or Enterprise. For each case, say whether you'd recommend the CLI-with-script flow or considering Enterprise's native Git, and why in one sentence: (a) you alone, automating a small client's workflows; (b) a team of eight people promoting changes across three environments several times a day; (c) a personal learning project.

See solution

(a) CLI + script flow. You produce the same versioned, documented artifact, for free; for a small client, Enterprise's convenience doesn't justify its cost.

(b) Consider Enterprise. With eight people and daily promotions between environments, the friction of coordinating manual exports starts costing more than the license; there, native Git pays for itself in saved time.

(c) CLI + script flow, without a doubt. While learning, doing the flow by hand also teaches you what native Git does under the hood, knowledge that serves you whether you pay for it later or not.

Why it works: the decision isn't about capability —both produce "documented JSON"— but about scale and friction. The criterion is "is the license cost lower than the time I lose coordinating by hand?" For one person or a small team, almost never; for a large team with frequent promotions, sometimes yes.

Exercise 3 — Extend the script. The current export.sh exports and normalizes workflows, but doesn't touch credentials. Think through (no need to write it in full) what the script should NOT do regarding credentials, and why. Could the script export credentials with --decrypted to "back them up too"?

See solution

The script must not export credentials into the repository in any form, and much less with --decrypted. A script that runs export:credentials --decrypted and leaves the result inside the repo's folder would be writing plaintext secrets right where a git add would catch them —exactly the disaster lesson 3 exists to prevent. Automating a mistake doesn't fix it; it repeats it faster and more often.

If the script were ever going to touch credentials, it would be for an encrypted backup (without --decrypted) written outside the repository tree, to a secure destination, and even then with care. But the cleanest approach is for the workflow-export script to not touch credentials at all: they're two different flows with different security rules, and mixing them invites an accident. Lesson 3's .gitignore is the safety net in case something slips through, but the first line of defense is that the script simply never generates secrets inside the repo.

Why it works: this exercise proves you internalized lesson 3 to the point of recognizing a bad automation pattern. Automation amplifies whatever you feed it: it amplifies a good practice until it becomes an effortless habit, and it amplifies a security lapse until it becomes a systematic leak. That's why the script is designed with security as a constraint, not an afterthought.

Summary and next step

In this lesson you turned lessons 2 through 5's manual work into a single command. You understood what a shell script is —a saved list of commands, the same sequence you'd do by hand— and its anatomy: the shebang that says what to run it with, set -euo pipefail as a seatbelt, the configuration variables, and the for loop. You wrote export.sh, which checks its dependencies and warns clearly if one's missing, exports with lesson 2's CLI, pulls the files out of the container with docker cp, and in a loop normalizes (lesson 4) and renames to readable names by reading each JSON's name (lesson 5), all in one pass. You made it executable with chmod +x and ran it with ./scripts/export.sh. You saw the two ways to make sure it runs before every commit —by hand (recommended to start) or with a pre-commit git hook (with its two traps: it doesn't get versioned, and it re-exports the live state)— the convenient handles of make and npm run, and n8n's native Git Enterprise alternative, with the honest criterion for when it's worth paying for: it isn't a matter of capability, but of scale and friction, because both produce the same deliverable.

Before moving on you should be able to: explain why the script can use docker and jq even though the Code node can't; read export.sh and say what each step does; name the git hook's two traps; and decide between the CLI flow and Enterprise based on scale.

Lesson 8 is the project that closes the module. You're going to take an n8n instance with several workflows and credentials —Cumbre's full one— and produce, end to end, the cumbre-automations repository: workflows exported, normalized, and well named; secrets out with a correct .gitignore; professional structure; handoff documentation; and the export.sh that makes it all reproducible. The deliverable is a repository that would pass another developer's review, which is exactly what job postings ask for when they write "version-controlled, documented JSON."

Resources

← Previous6. Documenting a workflow for handoffNext →8. Project: from instance to documented repository