How to Handle Files in Python

In any programming language, file handling is an important aspect. And Python also supports working with files in different modes—such as reading and writing to files, and more.

By the end of this tutorial, you’ll be able to:

• open and read files in Python,
• read lines from a text file,
• write and append to files, and
• use context managers to work with files in Python.

How to Read File in Python

To open a file in Python, you can use the general syntax: open('file_name','mode').

• Here, file_name is the name of the file.

Note: If the file that you’d like to open is in the current working directory, you can mention only the name of the file. If it’s in another folder in your working environment, you should include the path to the file.

• The parameter mode specifies the mode in which you’d like to open the file.

The default mode for opening a file is read—denoted by the letter 'r'. However, it’s a recommended practice to specify the mode explicitly.

Before we begin, let’s take a look at the file lib.txt, which we’ll use in this example.

📁 Download the text file and code used in this tutorial from this GitHub repo.

The code snippet below shows how you can open a text file 'lib.txt' in Python using open() function, and read its contents.

file = open('lib.txt','r')
contents = file.read()
print(contents)
file.close()


# Output
Hello, there!
Here are a few helpful Python libraries:
1) NumPy
2) pandas
3) matplotlib
4) seaborn
5) scikit-learn
6) BeautifulSoup
7) Scrapy
8) nltk
9) Bokeh
10) statsmodels

In the above example,

• The open() function returns a file object, and we choose to call it file.
• Next, we call the read() method on file.
• The variable contents now contains the content of the file. And we print it out.
• Finally, we close the file.

However, if you forget to close the file, there will be a possible wastage of resources. If you’re working with a large number of such files, there can be substantial memory usage. This is because you’ve opened several files but haven’t closed any of them.

Now, let’s learn a better way to open files using context managers. The code snippet below shows how you can use them.

with open('lib.txt','r') as f:
  contents = f.read()
  print(contents)

When using contact managers to work with files, you don’t have to use the close() method. The files are automatically closed after the I/O operation is complete.

How to Read Lines from File in Python

In our sample text file, we had only a few lines. So reading in all of the file contents at once was not a problem.

However, when you need to read in large files, using the read() method, as shown above, may not be very efficient.

In fact, if the text file is of a very large size, you may soon run out of memory. That’s why you may want to read in read only lines from a text file, and you’ll learn how to do this in this section.

Using Python’s readline() Method to Read Lines from File

The readline() method reads one line at a time, from the file.

Run the following code snippet.

with open('lib.txt','r') as f:
  line = f.readline()
  print(line)
  line = f.readline()
  print(line)


# Output
Hello, there!

Here are a few helpful Python libraries:

You can see that after the first readline() method call, the first line in the file is printed out. And the second call to the readline() method returns the second line in the file.

This is because, after the first method call, the file pointer is at the beginning of the second line.

In Python, you can use the the tell() method to get the current location of the file pointer. And to move the file pointer to a specific location, you can use the seek() method.

In the code snippet below, we use f.seek(0) after the first method call. This moves the file pointer to the beginning of the text file. That is why, both the times, the first line in the file is printed out.

with open('lib.txt','r') as f:
  line = f.readline()
  print(line)
  f.seek(0)
  line = f.readline()
  print(line)


# Output
Hello, there!

Hello, there!

Using Python’s readlines() Method to Read Lines from File

There’s another closely related method called readlines().

When you run the following code snippet, you’ll see that the readlines() method returns a list of all the lines in the file.

with open('lib.txt','r') as f:
  lines = f.readlines()
  print(lines)


# Output
['Hello, there!n', 'Here are a few helpful Python libraries:n', 
'1) NumPyn', '2) pandasn', '3) matplotlibn', 
'4) seabornn', '5) scikit-learnn', '6) BeautifulSoupn', 
'7) Scrapyn', '8) nltkn', '9) Bokehn', '10) statsmodelsn', 'n']

Using Python’s for Loop to Read Lines from File

In order to read in the lines from a text file, you could also use the for loop.

Once you have a file object, you can use for loop to iterate through the contents of the file—one line at a time and print them out, as shown below. Notice how we’re accessing only one line at a time and not reading in the entire file’s contents.

with open('lib.txt','r') as f:
  for line in f:
    print(line, end='')

Note: When using Python’s print() function, the default separator is a newline—'n' character. But in the original file, we don’t have these new lines. So set the separator argument to an empty string: sep = '' in order to print the contents of the file as they are.

How to Read Chunks of Content from File in Python

In Python, you can also choose to read in the contents of the file in terms of small chunks.

Read through the code below:

• Here, we set the chunk_size to 50. This means that the first 50 characters in the file will be read in, and we also print them out.

• Now, call the tell() method on the file object f. You can see that the file pointer is now at position 51— which is as expected.

chunk_size = 50
with open('lib.txt','r') as f:
  chunk = f.read(chunk_size)
  print(chunk)
  current = f.tell()
  print(f"Current position of file pointer: {current}")

# Output
Hello, there!
Here are a few helpful Python librar
Current position of file pointer: 51

You can also use this technique to read in the entire file in terms of small chunks.

The following code snippet shows how you can do this.

chunk_size = 50
with open('lib.txt','r') as f:
  chunk = f.read(chunk_size)
  print(chunk,end='')

  while(len(chunk)>0):
    chunk = f.read(chunk_size)
    print(chunk,end='')

# Output
Hello, there!
Here are a few helpful Python libraries:
1) NumPy
2) pandas
3) matplotlib
4) seaborn
5) scikit-learn
6) BeautifulSoup
7) Scrapy
8) nltk
9) Bokeh
10) statsmodels

Here, we use a while loop to read the contents of the file. We read in the file’s contents in chunk of size 50 until we reach the end of the file. ✅

How to Write to File in Python

In order to write to a text file in Python, you should open it in the write mode—specifying 'w'.

The code snippet below shows how to do it.

with open('new_file.txt','w') as f:
  f.write('Hello, Python!')

You’ll see that 'new_file.txt' has been created in your working directory.

Now, run the above code cell once again.

In your terminal run the following command:

cat new_file.txt

# Output: Hello, Python!

Ideally, we’ve written to the file twice. So Hello, Python! should have been printed twice, yes?

But you’ll see that it has been printed only once. Well, this is because when you open a file in write (w) mode, you basically overwrite the contents of the file with new content.

If you’d like to add to the end of the file without overwriting existing content, you should open the file in the append mode. And you’ll see how to do this in the next section.

How to Append to File in Python

If you’d like to append content to a file—without overwriting, open it in the append mode.

To do this, use `'a'—a for append—and specify the mode explicitly.

Next, run the following code cell twice.

with open('new_file.txt','a') as f:
  f.write('Hello, Python!')

Notice how the text is printed out twice now, as we appended to the file.

cat new_file.txt

# Output: Hello, Python!Hello, Python!

Conclusion

Let’s quickly summarize what we’ve gone over in this tutorial.

• You’ve learned the common file I/O operations such as reading, writing, and appending to a file.
• In addition, you’ve also learned how to use the seek() method to move the file pointer to specific position, and
• how to use the tell() method to retrieve the current position of the file pointer.

I hope you found this tutorial helpful. Now that you’ve learned how to work with text files in Python, learn how to work with JSON files in Python.