When writing Bash scripts, you will sometimes find yourself in situations where you need to read a file line by line. For example, you may have a text file containing data that should be processed by the script.

In this tutorial, we will discuss how to read a file line by line in Bash.

Reading a File Line By Line Syntax

The most general syntax for reading a file line-by-line is as follows:

while IFS= read -r line
do
  echo "$line"
done < input_file

or the equivalent single-line version:

while IFS= read -r line; do echo $line; done < input_file

How does it work?

The input file (input_file) is the name of the file you want to be open for reading by the read command. The read command reads the file line by line, assigning each line to the line variable. Once all lines are processed the while loop will terminate. The internal field separator (IFS) is set to the null string to preserve leading and trailing whitespace which is the default behavior of the read command.

Reading a File Line By Line Examples

Let’s take a look at the following example. Suppose, we have a file named distros.txt containing a list of some of the most popular Linux distributions and their package managers separated with comma (,):

distros.txt

Ubuntu,apt
Debian,apt
CentOS,yum
Arch Linux,pacman
Fedora,dnf

To read the file line by line you would run the following code in your terminal:

while IFS= read -r line
do
  echo "$line"
done < distros.txt

The code will read the file by line, assign each line to a variable, and echo the variable. Basicity you would see the same output as if you would display the file content using the cat command.

What if you want to print only the distributions that use apt? One way would be to use the if statement and check if the line contains the apt substring:

while IFS= read -r line
do
  if [[ "$line" == *"apt"* ]]; then
    echo "$line"
  fi
done < distros.txt

Ubuntu,apt
Debian,apt

When reading file line by line, you can also pass more than one variables to the read command which will split the line into fields based on the IFS. The first field is assigned to the first variable, the second to the second variable, and so on. If there are more fields than variables, the leftover fields are assigned to the last variable.

In the following example, we are setting IFS to a comma (,) and passing two variables distro and pm to the read command. Everything from the beginning of the line until the first comma will be assigned to the first variable (distro) and the rest of the line will be assigned to the second variable (pm):

while IFS=, read -r distro pm
do
  echo "$pm" is the package manager for "$distro"
done < distros.txt

apt is the package manager for Ubuntu
apt is the package manager for Debian
yum is the package manager for CentOS
pacman is the package manager for Arch Linux
dnf is the package manager for Fedora

Alternative File Reading Methods

Using a Process Substitution

Process substitution allows you to pass output from command as a filename:

while IFS= read -r line
do
  echo "$line"
done < <(cat input_file )

Using a Here String

Here String is a variant of Here document. The string (cat input_file ) will keep the newlines:

while IFS= read -r line
do
  echo "$line"
done <<< $(cat input_file )

Using File descriptor

You can also provide the input to the loop using a file descriptor:

while IFS= read -r -u9 line
do
  echo "$line"
done 9< input_file

When working with file descriptors use a number between 4 and 9 to avoid conflict with shell internal file descriptors.

Conclusion

In Bash, we can read a file line-by-line by providing the filename as an input to a while read loop.

If you have any questions or feedback, feel free to leave a comment.

Show Comments