4. Redirecting output to files: >, >>, 2>, and /dev/null

Description

By the end of this lesson you will be able to send any command's output exactly where you want: save it to a new file, append it to one that already exists, separate errors from results, combine both streams when it suits you, discard noise you do not care about, feed a program's input from a file, and watch an output on screen while saving it to disk at the same time.

This is not a syntax trick. It is the skill that separates someone who runs a script and crosses their fingers from someone who automates with no surprises: a cron job that fails silently because someone sent everything to /dev/null without thinking about what they were covering up, a deployment that overwrote the previous night's log with a careless >, an analysis pipeline that mixes errors with data and ruins the final count. All three are redirection bugs, not logic bugs.

Connection to the module: in the previous lesson you saw that every process is born with three streams — standard input, standard output, and standard error — identified by descriptors 0, 1, and 2. Now you are going to use those same numbers to tell the terminal, with precision, where each one goes.

The mental model: a valve, not a switch

Think of every command as a pipe with three ports: one input and two outputs. By default, both outputs end at the screen and the input comes from the keyboard. Redirection operators do not create new streams — they never add a port where there was none — they turn a valve so a stream that already existed ends up somewhere else: a file, another stream's same destination, or a drain that discards everything.

That image — turning a valve, not opening a new pipe — is what you need to understand why order matters in some lines: if a valve points "wherever the other one points right now," reversing the order you turn the two in changes the result. You are going to see exactly that case below, with 2>&1.

Everything that follows works the same in bash and in zsh (the default shell on macOS since 2019) except for one specific detail flagged where it shows up.

> — writing to a file (and the real danger)

> takes a command's standard output and writes it to a file. If the file does not exist, it creates it. If it exists, it truncates and replaces it with no prompt — no confirmation, no recycle bin, no undo.

ls -l /etc > listing.txt

What to expect: nothing on screen — ls -l /etc's full output went straight into listing.txt — and cat listing.txt shows the whole listing.

The danger shows up when you repeat the same command, or a similar one, without noticing the destination file already had something valuable:

echo "Deploy started" > deploy.log
# ...hours later, someone runs something similar pointing at the same file...
echo "Debugging session" > deploy.log
cat deploy.log

What to expect:

Debugging session

The first line disappeared with no warning. > did exactly what it was asked: truncate and write.

The protection is the shell's noclobber option:

set -o noclobber
echo "test" > deploy.log

What to expect (with deploy.log already existing, as it was left above):

bash: deploy.log: cannot overwrite existing file

If you really want to overwrite despite noclobber, the >| operator forces the write once — and it works the same in bash and zsh, so it works for you no matter which one your terminal runs:

echo "test" >| deploy.log

noclobber does not protect you from everything — it says nothing about >>, and it only acts when the operator is > — but it eliminates the most expensive mistake: overwriting a file you thought nobody was touching anymore.

>> — appending without destroying

>> opens the file in append mode: it writes at the end and, if the file does not exist, it creates it. It is the tool for accumulating, not for replacing.

echo "Deploy started at 09:00" > session.log
echo "Deploy finished at 09:04" >> session.log
cat session.log

What to expect:

Deploy started at 09:00
Deploy finished at 09:04

The practical rule: use > the first time a script writes a file within a run — to start clean — and >> for every subsequent write within that same run. Mixing the two up is the most common cause of "why does my log only have the last line?"

2> — separating the errors

Descriptor 2 is standard error. 2> redirects it without touching standard output (descriptor 1), which keeps going to the screen.

ls -l /etc /no-such-directory 2> errors.log

What to expect on screen: /etc's normal listing — that kept going as standard output, never got redirected. And in errors.log:

ls: /no-such-directory: No such file or directory

This is exactly what the previous lesson set up: the two streams are independent, and now you have the operator that points at each one separately.

2>&1, the order, and the &> shortcut

Sometimes you want both streams in the same destination: a single log with results and errors in the order they happened. 2>&1 means "send descriptor 2 to wherever descriptor 1 points right now." The key phrase is right now: the shell processes redirections left to right, so order decides the result.

ls -l /etc /no-such-directory > all.log 2>&1

What to expect: first > all.log points standard output at the file. Then 2>&1 points descriptor 2 to wherever descriptor 1 already is — that is, the same file. all.log ends up with the listing and the error, mixed in order.

Reverse the order and it breaks:

ls -l /etc /no-such-directory 2>&1 > all.log

Here 2>&1 gets processed first, while descriptor 1 still points at the screen: the error ends up pointing there. Only afterward does > all.log move standard output to the file. all.log only has the listing; the error prints on screen, exactly where you did not want it.

The shortcut &> (or &>> to append) does the same as > file 2>&1 in a single piece, with no risk of reversing the order:

ls -l /etc /no-such-directory &> all.log

/dev/null: when silencing is legitimate and when it hides a problem

/dev/null is a special file that discards anything you write to it. It takes no space, stores nothing, cannot be read back. Sending something there tells the system "I do not care about this, throw it away."

find / -name "*.conf" 2> /dev/null

What to expect: the list of .conf files you actually could read, with the screen not filling up with Permission denied lines for every system directory you have no access to.

The criterion for not turning this into a dangerous habit: silencing is legitimate when you already know exactly what error you are going to get, why you are going to get it, and you have confirmed it does not change the result you care about. In the find example, you know beforehand that searching from / as a regular user is going to run into protected system directories — that noise is expected and does not affect the list you are after.

It is hiding a problem when you silence an entire command (2>/dev/null with no thought about what error is expected) inside a production script, a deployment, or a cron job, "because it was cluttering the logs." There you did not remove the noise: you removed the evidence. The command might really be failing — an expired credential, a full disk, a database that is down — and you are not going to find out until the effect shows up much later, with no trace of the cause.

< — feeding input from a file

Just like > redirects output, < redirects standard input: instead of waiting for you to type on the keyboard, the command reads from a file.

wc -l < access.log

What to expect:

1284

Compare it to wc -l access.log (with no <): the count is the same, but that second form also prints the file's name (1284 access.log), because there you passed it as an argument and wc knows which one it is reading. With < the command does not even find out there is a file involved — it only sees a standard input with content inside. The difference matters when you chain commands that must only print the number, with no name stuck next to it.

tee and tee -a: watching and saving at the same time

Everything above forces you to choose: either you see the output on screen, or you save it to a file. tee breaks that dilemma — it reads its standard input and copies it to two places at once: the screen and one or more files.

ls -l /etc | tee listing.txt

What to expect: the same listing you would see with a normal ls -l /etc shows up on screen, and at the same time it gets saved, complete, to listing.txt.

(The | symbol connecting ls's output to tee's input is the pipe — you are going to explore it in depth in the next lesson. For now, just watch it doing its job: passing what comes out of one command into the next one's input.)

By default tee truncates the file, same as >. To append instead of replacing, use -a:

echo "Deploy started at 09:00" | tee -a process.log
echo "Deploy finished at 09:04" | tee -a process.log

What to expect: both lines show up on screen the moment they run, and both end up accumulated in process.log — useful when you want to follow a long process live without losing the full record to review afterward.

Common mistakes

1. Believing 2>&1 works the same no matter where you put it (conceptual). What happens: someone writes command 2>&1 > file.log expecting to see errors and results together in the file, and the errors still show up on screen. Why: 2>&1 does not say "join stdout and stderr forever" — it says "point descriptor 2 wherever descriptor 1 points today," and the shell resolves it the moment it reads it, left to right. If descriptor 1 still points at the screen when 2>&1 is processed, that is where it stays. How to spot it: the log file has fewer lines than you expected and errors still show up in the terminal. How to fix it: put 2>&1 at the end (> file 2>&1) or use the &> shortcut, which does not let you accidentally reverse the order.

2. Overwriting an important file with > without noticing. What happens: you run a command with > pointing at a file that already had valuable content — a log, a backup, a config — and lose it with no warning. Why: > truncates immediately, before running the command, and does not compare or ask. How to spot it: check the file's size or modification date (ls -l file) the moment you suspect something — if it has less content or a more recent date than you expected, it already got overwritten. How to fix it going forward: turn on set -o noclobber in your session (or in your .bashrc/.zshrc) for files you truly do not want to lose, and reserve >| for the times you do want to force the write, knowingly.

3. Using /dev/null to "make the script stop complaining" without knowing what error you are covering up. What happens: a deployment script or a cron job ends with 2>/dev/null stuck on the end of every line because at some point it generated annoying noise, and months later a real failure — permissions, connection, a full disk — goes completely unnoticed. Why: /dev/null does not distinguish expected noise from a serious error; it discards everything that reaches descriptor 2, with no judgment. How to spot it: if you cannot explain, command by command, which specific message you expect to show up there and why it is harmless, that is a sign you are covering it up instead of filtering it. How to fix it: redirect to a file (2>> errors.log) instead of /dev/null while you are not sure, check that file periodically, and only then decide, error by error, which ones really are expected noise.

Exercises

1. You have this command, which is supposed to save both the result and the errors of grep into search.log:

grep -r "TODO" src/ 2>&1 > search.log

Checking it, you notice the errors (for example, "Permission denied" in some directory) still show up in the terminal instead of ending up in search.log. Why does this happen, and how do you fix it two different ways?

grep -r "TODO" src/ > search.log 2>&1

grep -r "TODO" src/ &> search.log

2. You want to count how many lines of a sales.csv file contain the word "refund," without the result including the file's name next to the number. Write the command using input redirection (<) instead of passing the file as an argument.

grep -c "refund" < sales.csv

3. A coworker runs this command on a production server and asks you why it "does nothing":

./backup.sh > /dev/null 2>&1

What is actually happening, and what would you change so you could diagnose a failure if the backup starts failing tomorrow?

./backup.sh &> backup.log

Summary and next step

You can now decide, with precision, where every one of a command's streams goes: create a file with > — and protect yourself from overwriting it with noclobber — append with >>, separate errors with 2>, combine them in the right order with 2>&1 or with the &> shortcut, discard expected noise into /dev/null without hiding real failures, feed input from a file with <, and watch and save at the same time with tee and tee -a.

Everything you did in this lesson moved streams between a command and a file. What comes next is moving a stream directly from one command to another, with no disk involved: the pipe, |.

Before moving on you should be able to:

• explain, without looking at this lesson, why command 2>&1 > file does not do the same as command > file 2>&1;
• decide whether a specific case of "silencing with /dev/null" is legitimate or is hiding a problem;
• write from memory the correct way to save stdout and stderr together to a file, both with the shortcut and the explicit form.

Resources

• Redirections — Bash Reference Manual — the official reference for every bash redirection operator, including >| and &>.
• tee invocation — GNU Coreutils Manual — official documentation for tee, its options (-a, -i), and its exact behavior.
• NoClobber — Greg's Wiki (BashFAQ) — explains noclobber in detail, its limits (it does not protect >> or pipes), and how to combine it with >|.
• tee(1) — Linux manual page — the full manual page for tee, useful for looking up all its options from the terminal with man tee.