Computer Science Department Unix Primer

Redirection and Pipes

The glue behind UNIX's toolkit approach is input/output stream redirection.

Most of the UNIX command line utilities work as filters. Conceptually, the command line works like plumbing. Pipes and redirects just channel data between the various components of the command line. A program that acts as a filter takes input from the STDIN stream (typically your terminal's keyboard), generates output to the STDOUT stream (typically printing to the terminal), and if there are any problems, it will generate error messages to the STDERR stream. STDOUT and STDERR usually are both displayed to your terminal.

A good way to learn streams is with the cat command. If you type cat with no arguments, you will find that anything you type will be repeated back at you after you hit enter. (Press ^d on a clean line to exit).

$ cat
hello
hello
echo
echo

Now use the > character to force cat to redirect to a file.

$ cat > a.txt
hello
echo
$ ls
a.txt
$ cat a.txt
hello
echo

This is output redirection; you just redirected the cat program's STDOUT to a file called a.txt. Now you can try the opposite, to redirect the standard input to cat from a file:

$ cat < a.txt
hello
echo

The fun part comes when you redirect the output of one program to the input of another program. A good example would be to use a couple of the advanced utilties described earlier.

$ cat unsorted.txt 
peaches
bananas
oranges
pears
apples
cranberries
blueberries
$ sort unsorted.txt 
apples
bananas
blueberries
cranberries
oranges
peaches
pears
$ grep b unsorted.txt 
bananas
cranberries
blueberries
$ grep b unsorted.txt | sort
bananas
blueberries
cranberries

Here, we've used the grep and sort commands to grep out fruits that contain the letter b in their names then sort them. The output can still be redirected once again to a file.

$ grep b unsorted.txt | sort > sorted_b.txt

One trap to avoid with redirection is that you don't try to read and write to a file at the same time. For instance, trying to sort unsorted.txt back into itself has the unexpected result of emptying the file and losing its contents. This is because the > redirection will clobber (yes, that's the technical term) the existing contents.

To accomplish output redirection that doesn't clobber existing contents, you use >>. This will append new output to the end of the file.