Chapter #11: How to Use IFS and Read to Parse Files in Bash

We’re picking up right where we left off - working with files in Bash. There’s a lot more you can do beyond just reading a file line by line.

File handling is a core part of Bash scripting. Whether you're automating tasks, building utilities, or managing system processes, you're going to be dealing with files all the time. So, knowing how to manipulate them efficiently is key.

We have separated this into multiple lessons so you can concentrate on each lesson and focus on what you learn from it.

In the last lesson, we learned how to read a file using the cat command with some examples and simple Bash scripts that used a while loop to read a text file line by line.

In this lesson, we will continue learning about file and text processing commands and techniques that you should know as a sysadmin.

Internal Field Separator (IFS)

IFS, or Internal Field Separator, is what Bash uses to decide how to split a string into words. By default, it splits on spaces, tabs, and newlines. But the cool part? You can change it to whatever you want.

I know you don’t understand this from this introduction, but when you do an example, you will understand what we mean here.

If you remember the previous example that we did in the last lesson, it looked like this:

while read -r line1 && read -r line2  
do  
   echo "Line 1: $line1"
   echo "Line 2: $line2"
done < file.txt  

In this script, we're reading two lines at a time from a file called file.txt. Then we print them out one after the other.

The output looks like this:

Line 1: Learn Bash with Pro.TecMint
Line 2: Linux is Awesome it Forces you to Think...

This example is simple and good, but what if I ask you to write a script that takes this file:

Learn Bash | Learn Linux | Learn Docker

Let’s say you’ve got a single line in a file that contains three parts, and you want to store each part in its own variable:

• "Learn Bash" should go into one variable
• "Learn Linux" into another
• And "Learn Docker" into a third one

Then, you want to print them out.

How can you do that?

The trick here is to use the IFS (Internal Field Separator). Here’s how you can do it:

IFS="|"
while read -r part1 part2 part3
do
   echo "Part 1: $part1"
   echo "Part 2: $part2"
   echo "Part 3: $part3"
done < file.txt

Now, assuming your file.txt contains a line like this:

Learn Bash|Learn Linux|Learn Docker

When you run the script, you'll get this output:

Part 1: Learn Bash
Part 2: Learn Linux
Part 3: Learn Docker

Nice and clean. The IFS="|" tells Bash to split the line wherever it sees a pipe (|), which makes it super easy to assign each part to a variable.

In the while loop, we use the read command followed by some variable names.

By default, the read command reads an entire line of text. But when we set IFS (which stands for Internal Field Separator), it splits that line into parts based on the character we choose. That’s why we use read in this way - it helps us break a line into separate pieces.

I think you’re getting the hang of it now, but let’s test that a little.

Say we have a line like this:

Learn Bash$Learn Linux$Learn Docker

Now, use what you learned from the previous example, but apply it here. The only difference is the delimiter. (A delimiter is just a character or symbol that splits strings apart. In this case, it’s the dollar sign $.)

In the last example, the delimiter was a pipe symbol |. This time, it’s $.
Same script structure, just a different delimiter.

Starting to see how powerful IFS is?

Here’s the key idea: no matter what you’re doing in Bash, IFS is always there in the background. By default, it’s set to a space.

Need proof of that? Cool, check out this example:

names=(TecMint Bash Linux)

echo "${names[@]}"

We declare an array names with three values, and in the next line, we print it.
If we run this, we should get the following:

$ bash ifs.sh 
TecMint Bash Linux

You already know that the @ symbol is used to print all the elements of an array, nothing fancy, right?

Cool. Now let’s try something different. What happens if we change the IFS (Internal Field Separator) to something else? For example, we’ll set it to an empty string like this:

IFS=''
names=(TecMint Bash Linux)

echo "${names[*]}"

Notice I used * instead of @. Both symbols print all the values in the array, but there’s a key difference:

• When you use *, the array is treated as one big string.
• When you use @, each element is treated separately, like individual words.

So what happens when you run this code?

Since we changed IFS to an empty string (no space, no separator), Bash doesn’t know how to separate the array elements anymore. It just sticks them together:

TecMintBashLinux

No spaces, no separators, just one long string. That’s because IFS is what Bash uses to decide how to split things up, and by default, it's a space.

This little experiment shows you that IFS is always there, doing its job behind the scenes. And you can tweak it whenever you need to control how your data gets split or joined.

Check for Empty Lines

Now that we’ve learned how to use IFS in Bash and understand its role, let’s look at another interesting aspect of working with files - handling empty lines.

Let’s go back to an example we’ve seen before:

while read -r line1 && read -r line2  
do  
   echo "Line 1: $line1"
   echo "Line 2: $line2"
done < file.txt  

And here’s what file.txt might look like:

Learn Bash with TecMint

Linux is Awesome it Forces you to Think

Notice that there's an empty line between the two blocks of text.

When we run the script, you’ll see that Bash still processes those lines, including the empty one and your output will look something like this:

$ bash read.sh 
Line 1: Learn Bash with Pro.TecMint
Line 2: 

Okay, now you see that we have a problem here: there’s an empty line between the two lines, and we need a trick to skip it. We only want to keep the non-empty lines.

As we’ve already discussed, the read command doesn’t have a built-in option to skip empty lines, so we need a solution for that.

The solution we’ll use is a test condition to check whether a line is empty or not, like this:

while read -r line1
do
    [[ -z "$line1" ]] && continue  

    while read -r line2 
    do
        [[ -z "$line2" ]] && continue   
        echo "Line 1: $line1"
        echo "Line 2: $line2"
        break  
    done
done < file.txt

It looks like a big, scary script, right? Don’t worry, it’s actually simple! Let’s break it down:

• First, we read the first line using read -r line1, and then we check if the line is empty using -z, which checks whether the variable (the line in the file) is empty or not.

This syntax:

[[ -z "$line1" ]]

means: "Test if the line is empty." If it is empty, the continue command will skip the rest of the loop and start the next iteration.

• After we’ve processed the first line, we start another loop to read the next line:

while read -r line2

As before, we check if this line is empty and skip it until we find one that isn’t.

• Once we have both lines and they’re not empty, we print them out, and then use break to exit the inner loop once we’ve processed line2.

The result looks something like this:

Using this approach, we get rid of any empty lines that may exist at the beginning or in between lines.

For example, if we have something like this in the file:

Learn Bash with Pro.TecMint

Linux is awesome. It forces you to think.

But, after executing our script, we’ll get the following output:

Line 1: Learn Bash with Pro.TecMint
Line 2: Linux is awesome. It forces you to think.

Here’s the new thing you’ve learned from this section: You can use the if statement like this:

[[ -z "$line1" ]]

Instead of the longer version:

if [[ -z "$line1" ]]; then
    continue   
fi

Both are the same, but the shorter version is more concise and easier to write.

Another thing to note is the -z test, which checks if the value of a variable is empty.

• break is used to exit the loop once you get the result you want.
• continue is used to skip the rest of the loop and move to the next iteration.

Remember, in our examples, we used a text file, but this method can be applied to any kind of file.

In this lesson, we learned how to handle empty lines in files. In the next lessons, we’ll continue to explore the power of Bash scripts.