Chapter #6: How to Use Command Substitution and Output Redirection in Bash

In the previous chapters, we've covered some core concepts, like working with variables, taking user input, and writing simple scripts. Today, we're going to dive into something that will make your Bash scripting even more powerful: command substitution and output redirection.

Bash is a scripting language that makes automation and system administration both efficient and fun. With the right tools, you can automate almost anything on your Linux system, and today, we're introducing a couple of those tools.

Redirection is all about taking the output from one command and sending it somewhere else, like to another command or to a file. This might sound a bit tricky at first, but don't worry, we'll break it down step by step and show you how these techniques can make your scripts more flexible and dynamic.

Command substitution is a feature that allows you to capture the output of a command and use it inside another command, or store it in a variable. This lets you create scripts that can react to their environment and execute more efficiently.

By the end of this chapter, you'll know how to make your scripts more dynamic and useful by redirecting command output to files and using the output of one command in others.

Let’s get started!

Using Command Substitution in Bash

If you’ve been following along with the last few lessons, you might remember hearing about command substitution in one of the examples.

Every command in Bash produces an output, this is the result after executing the command. For instance, when you run the ls command, the output is the list of files in that directory. Command substitution allows us to capture that output and pass it to other tools, save it in a file, or use it however we need in our script.

In simple terms, command substitution lets you take the result of a command and assign it to a variable, text, or any other context within a script.

You might recall from a previous lesson when we used the date command and assigned its output to a variable. That’s what we mean by command substitution.

Let’s look at another example:

all=$(ls -a)
echo "All the files in this directory: $all"

Here’s what we’re doing: in the first line, we declare a variable called all, and it stores the output of the ls -a command. This command lists all the files in the directory, including hidden ones.

Notice the syntax for command substitution: $(command). This is the recommended way to do it. You could also write it like this: all=ls -a, but using $(command) is the best practice.

What happens is that the shell first executes the command inside the parentheses (), takes the result (or output), and stores it in the all variable.

To visualize this, you can think of it like this:

Understanding Command Substitution

By using command substitution, we’ve made our script more dynamic. Instead of manually listing all the files, we’re automatically retrieving them. We can make this example even more dynamic.

Let's add another variable that tells us the directory we are in, and then shows the files inside it. If you recall, the command to show the current working directory is pwd.

Let’s update our previous example:

all=$(ls -a)
dr=$(pwd)
echo "All the files in $dr are: $all"

If you run this, you’ll get a result like this:

Dynamic Directory Listings

Now, we’ve made the script even more dynamic. No matter where you run the script, it will first display the current working directory and then list all the files within it!

In the previous example, we assigned the output of a command to a variable, but we can also use it directly within a command.

For example:

echo "Today is $(date +%A)"

Here, we’re passing the output of the date +%A command directly into the echo statement. This command displays the day of the week.

If you run this, you’ll see something like:

Today is Friday

This is another way to use command substitution directly in a command without needing to store it in a variable.

Using Output Redirection in Bash

The same idea we used for command substitution applies to output redirection. Redirection means taking the output of a command and sending it to a file instead of displaying it on the screen.

In command substitution, we passed the output to another command or assigned it to a variable. In redirection, we send the output to a file instead.

Let’s make this clearer with an example:

ls > file.txt
echo "Done! Check the file.txt"

In the first line, we use the ls command along with the greater-than operator (>) followed by a filename. This redirects the output of the ls command to a file called file.txt.

The second line simply prints a message saying that the execution has finished.

If we run this code, we'll first see the message "Done! Check the file.txt."

As a result, a new file named file.txt will be created in the same location where our script is running. This file will contain the output of the ls command.

You can check the contents of the file using the cat command like this:

cat file.txt

The > operator creates a new file if it doesn’t already exist. But, if the file already exists, it will overwrite the content inside.

So, every time you run the script, it will replace the old content with the new output. This is what we mean by overwrite.

Behind the scenes, redirection works similarly to command substitution: it first creates the file (if it doesn't exist) and then stores the output inside it.

Appending to a File

As we discussed earlier, the > operator overwrites the content of a file. But if you want to add new content to a file without replacing what’s already there, you should use >> instead.

Appending allows you to keep the previous content and add the new output each time the script runs.

Example:

ls >> file.txt
echo "Done! Check the file.txt"

This is the same as before, but with >> replacing the > operator. Now, if we run the script twice, the file file.txt will contain the output of ls two times, instead of replacing the old content.

To verify, you can check the file contents using.

cat file.txt

Redirecting Errors

Another important concept to keep in mind when working with redirection in Bash is that sometimes, instead of the expected output, we might encounter errors depending on the script we’re writing.

For example, let’s revisit the last script and add a line with an incorrect command:

ls >> file.txt
ls -z >> file.txt
echo "Done! Check the file.txt"

In the second line, we added a command that doesn’t exist: ls -z. If you try this command in the terminal, you'll get an error like:

ls: invalid option -- z

This means the command isn't working. Now, if we run the script and check the content of file.txt, we’ll see that the script executed, but the error is captured too. This isn't ideal, since we only want to see the correct output, not the error.

Fortunately, Bash provides a way to redirect errors separately. We can do this using the 2> syntax.

In Bash, 1> refers to standard output (the expected output of the command), while 2> refers to standard error (the errors that might occur).

Let’s modify the script like this:

ls >> file.txt
ls -z >> file.txt 2>> errors.txt
echo "Done! Check file.txt and errors.txt"

Here, the second line is updated to 2>> errors.txt, which tells Bash to redirect any errors to errors.txt. Now, when we run the script, we’ll have two files:

• file.txt for the successful output
• errors.txt for the errors

Each will contain the following content:

• file.txt: Only the successful output from ls.
• errors.txt: The error from ls -z.

You can think of it like this:

Also, you can use either >> or 1>> for standard output, both mean the same thing.

Summary

In this chapter, we explored two essential Bash features: command substitution and output redirection. By mastering these techniques, you can create more dynamic and flexible scripts that can interact with their environment, handle data more efficiently, and manage errors gracefully.

As you continue to explore Bash scripting, these tools will prove invaluable for automating tasks, processing data, and handling unexpected situations.

In the next lesson, we'll take our scripting skills even further by diving into control flow and loops, allowing us to make our scripts even more powerful and responsive.

Keep experimenting with these concepts, and soon you’ll be writing scripts that can do almost anything! Happy scripting!