Chapter #10: How to Read Files in Bash with cat, While Loops & More

Till now, we've covered some key concepts in Bash, and our journey to mastering shell scripting continues.

Before diving deeper into Bash itself, we need to equip ourselves with an essential tool that will give us greater control over our system. You've likely heard the saying, "everything in Linux is a file," and it's true. To truly understand and gain more control over your system, it's crucial to grasp this idea.

In Bash scripting, working with files and directories is a fundamental skill. As a sysadmin, you'll need to master how to interact with them. A significant portion of your work in Linux will involve dealing with files and directories, whether you're reading from them or writing to them.

In this lesson, we'll explore how to handle files and directories with practical examples and real-world scenarios, so you’ll be well-prepared to interact with them effectively.

Reading from a File

One of the essential operations you need to master is reading from files using Bash scripts. As a system administrator, or even as a developer, there will be times when you need to read from a file, either to view its contents or to pass that content to another tool or operation for further handling.

In Bash, there are several commands available for reading files. A quick reminder, which we might not have covered in the previous lesson: Bash commands fall into two main categories.

External Commands: These are commands that aren’t part of Bash itself. They are separate programs, typically stored in directories like /bin or /usr/bin. All the commands/programs in these directories are considered external commands.

Since these commands aren't built into Bash, executing them requires launching a child process, which makes them slower than native Bash commands.

For example, consider the ls command. Here’s how it works behind the scenes:

The internal commands, or often we call them built-in commands, are the commands that are in the language itself. To run them, we don’t need a child process, they execute directly in the current shell (which makes them fast compared to the first ones!).

How to know if a command is built-in or external? Well, you can use the command type, which is itself a built-in command that shows you the type of the command.

Example:

That’s just a quick reminder that you need to know. I remember in one of my interview questions getting this:"What is the difference between external and internal commands in Bash, and how can you determine if a command is external?"

I answered the first part about the difference, but I didn’t know the command type. Now you know; don’t be like me!

Let's return to our subject for this lesson, which is how to read a file in bash. For this purpose, there are a lot of commands that achieve almost the same goal, which is to view the content of the file.

Reading Files in Bash

For this purpose, there are several commands that can help you achieve the goal of viewing a file’s contents.

Cat command

The cat command is one of the most popular commands used to read a file's content in Bash. As we learned earlier, it's an external command.

It’s simple to use, just type cat followed by the file name.

cat file.txt

This will show the content of the file like this:

Learn Bash with Pro.TecMint
Linux is Awesome it Forces you to Think...

The cat command also has several options you can use. For instance, if you want to display line numbers, use the -n flag, like this:

cat -n file.txt

Output:

     1	Learn Bash with Pro.TecMint
     2	
     3	Linux is Awesome it Forces you to Think...

If you'd prefer to count line numbers but skip the empty lines, use the -b flag instead, like this:

cat -b file.txt

Output:

     1	Learn Bash with Pro.TecMint

     2	Linux is Awesome it Forces you to Think...

With the cat command, you can also combine flags with redirection. In this case, we're using the -n flag to show line numbers, and we're redirecting the result to a new file called filewithnumbers.txt.

cat -n file.txt > filewithnumbers.txt

If you read it, it will show content like this:

You can even combine multiple files, either when reading them or during redirection.

cat file.txt filewithnumbers.txt 

This reads both files at once. Similarly, you can redirect them to a new or existing file, depending on your needs:

cat file.txt filewithnumbers.txt > all.txt

Here, we're redirecting the output of both files into a third file called all.txt. It will look like this:

As we’ve already learned, there's a difference between > and >>. Make sure you understand the distinction: > overwrites the file, while >> appends to it. Choose the right one based on your needs.

Here's an example to demonstrate this:

echo "New line" > file.txt    # Overwrites file.txt with "New line"
echo "Another line" >> file.txt  # Appends "Another line" to file.txt

That’s everything you need to know about the cat command. Just remember what each flag does, and you'll be good to go!

Use a While Loop to Read Files Line by Line

In some situations, we may need to read the contents of a file line by line, either for text manipulation or any text-related operation.

To achieve this, we need a method to process the file line by line. A loop is an effective way to accomplish this.

Let’s begin by using the read command to get the first line of the file:

read line1 < file.txt  
echo "The first line: $line1"  

The read command fetches the first line from the file file.txt and stores it in the variable line1. The < symbol indicates that the file is being used as input for the read command.

In the second line, we print the message "The first line" followed by the content stored in line1.

Running this will display something like this:

In most cases, when using the read command, it is common to include the -r flag. This flag prevents backslashes from being treated as escape characters, ensuring that the text and backslashes are printed exactly as they are.

Here's an example:

read -r line1 < file.txt  
echo "The first line: $line1"  

The output will be:

The first line: \ Learn Bash with Pro.TecMint \

This example only reads a single line. But what if we want to read multiple lines and manipulate them? For that, we need to use a loop.

Here’s how you can read and print lines one at a time:

while IFS= read -r line
do  
   echo "Line: $line"  
done < file.txt  

The IFS= ensures that leading/trailing spaces are preserved, and read -r prevents escape sequences from being interpreted.

Notice that the file.txt is placed at the end of the loop. This is because the while loop processes the entire content of the file as input.

If you run this example, you should see an output similar to this:

For instance, to skip empty lines:

while IFS= read -r line
do  
   # Skip empty lines
   if [ -z "$line" ]; then
      continue
   fi
   echo "Line: $line"
done < file.txt

Further File Operations

While reading files is crucial, don’t forget that there are other file operations you’ll often perform as a sysadmin. For example:

You can use the touch command to create an empty file:

touch newfile.txt

You can use mv to move or rename files:

mv oldfile.txt newfile.txt

Use rm to delete files:

rm file.txt

Standard Input, Output, and Error

When working with files and redirection, it’s important to understand stdin, stdout, and stderr:

• stdin (Standard Input): This is where input comes from, often provided via the keyboard or redirected from a file.
• stdout (Standard Output): This is where the output goes, typically displayed in the terminal.
• stderr (Standard Error): This is where error messages are sent.

For example, you can redirect error messages (stderr) to a file using 2>:

ls /nonexistentdir 2> error.log   # Redirects error message to error.log

By now, you should have a solid understanding of how to handle files in Bash, from reading files with the cat command to processing them line by line with loops.

You've learned how to redirect output and handle errors, giving you a broader toolkit to work with files in your system administration tasks.