When working with Linux systems, you’ll often need to inspect and understand the actions performed by processes and the system calls carried out by their execution.

When it comes to performing such tasks, the Linux kernel provides features such as ptrace to debug and diagnose processes.

This article discusses how to use the strace tool to trace, monitor, and debug processes interacting with the Kernel.

What Are System Calls?

Before we discuss how to use strace, you need to understand what we are looking for and how they work. That means we should l go over the basics of Linux System calls.

A system call is a programmatic method through which a program can request a service from the system’s Kernel. That is the process we will use to inspect the actions between user processes and the Linux kernel.

Anytime a user executes a program that makes a read, write, kill, exit, bind, etc., request, they are making a system call. There is a wide range of system calls used by programs to perform various tasks such as networking, reading and writing to files, initializing and terminating processes, and much more.

Think of system calls as functions—they behave similarly—because they can accept arguments and return values. The main difference between system calls and normal operation is that system calls can directly interact with the Kernel. System calls use a trap mechanism to navigate between user space and the Kernel.

In the Linux system, this mechanism is well hidden from the users by libraries such as Glibc.

NOTE: There is a lot more to system calls and kernel interactions than what we’ve discussed in this tutorial. Please refer to the manual pages for more information.

https://linkfy.to/syscalls

https://linkfy.to/trapmanual

How to Install strace on Linux

Although strace tools don’t come pre-installed by default in major Linux distributions, it is available in most official repositories of these distributions; you can install it easily using default package managers.

NOTE: Although we won’t cover how to install strace on all systems, we’ll discuss how to do so with major package managers such as apt, dnf, pacman, and yum

1: Debian (apt) Installation

Install strace using the command:

apt-get install strace -y

2: RedHat Family (dnf and yum)

To install strace using yum package manager, enter the command:

For dnf package manager, enter the command:

3: Arch Linux (pacman)

For Arch Linux users, you can install strace with the command:

Now that you have strace installed and running, we can move on and learn how to use

Basic Strace Usage: A How-to Guide

Let’s discuss basic strace usage and understand the basic output of the command and how we can use it.

NOTE: Strace output such as system calls names, corresponding arguments, and return values get handled by the standard error file descriptor (stderr).

The basic way to use strace is by calling the strace utility followed by the program’s name, whose behavior we want to understand.

Here’s an example of that using the ls command:

<img alt="" data-lazy- data-lazy-src="https://kirelos.com/wp-content/uploads/2020/12/echo/str1.png" data-lazy- height="419" src="data:image/svg xml,” width=”778″>

Wow! That is a lot of output for a simple command such as ls.

Although we cannot discuss all the output from the strace command, we can distill and understand its meaning.

If you consider the first line in the output above, you will notice the following features.

  • The name of the system call
  • The arguments passed to the system call enclosed in parenthesis.
  • The return value from the system call

Hence, in the first line, the system call is execve (execute program using the specified array of arguments), the arguments of the system call are (“/bin/ls”, [“ls”, “/”], 0x7fffc4b277a8 /* 13 vars */) and a return value of 0.

https://linkfy.to/execve

The execve system calls execute the binary we want to use, in this case, located in (/bin/ls) and the array of arguments being the path we want to list contents.

You will also notice a notation enclosed with a forward slash and an asterisk. For our example:

The above output indicates the number of variables added as a result of calling the process. The environment inside the execv function is accessed by using the environ external variable defined as:

int main(int argc, char *argv[], char *envp[])

The final output is the return value, which is 0 in this case.

You’ll also notice that most lines of the strace output follow a similar pattern we discussed above.

How to Trace Specific System calls

Although strace gives a lot of information regarding programs system calls, most instances will call on you to filter specific system calls. To do this, we pass the -e flag to the strace command followed by the name of the system call we need.

How about looking at the read system calls for the ls command. For example:

<img alt="" data-lazy- data-lazy-src="https://kirelos.com/wp-content/uploads/2020/12/echo/str2.png" data-lazy- height="149" src="data:image/svg xml,” width=”632″>

You will notice that this only displays only read system calls.

The read system call accepts three arguments: file descriptor, buffer, and the number of bytes. The system call then reads up to the count bytes from the passed file descriptor argument into the buffer.

https://linkfy.to/readsyscall

Summary of System Calls

Strace also allows us to get a summary of system calls made by a process. By passing the -c or –summary-only argument, we can get an output such as the one shown below:

<img alt="" data-lazy- data-lazy-src="https://kirelos.com/wp-content/uploads/2020/12/echo/str3.png" data-lazy- height="385" src="data:image/svg xml,” width=”576″>

The command filters and arranges the output more efficiently than the normal strace output. To get both summary and normal strace output, pass the -C argument.

How To Use Strace With Running Processes

At other times, you will need a trace of a running process. Up to this point, we have only used strace a single command. To trace a running process, we can use the -p argument followed by the Process ID (PID) process to attach strace to it.

You can get the PID of a running process by using the top and grep, ps, htop, pidof, or other system monitoring tools.

For example, to get the PID of the apache process, we can use:

That should give you the PID of the apache2 process (PID 3514 in this case), and we can use it to attach it to strace.

That should display an output similar to the one shown below.

<img alt="" data-lazy- data-lazy-src="https://kirelos.com/wp-content/uploads/2020/12/echo/str4.png" data-lazy- height="287" src="data:image/svg xml,” width=”657″>

Strace will continuously trace the attached process and show output as the attached process executes system calls. To terminate the trace, press CTRL C, which detaches the process from the strace.

How to Save Strace Output to Files

We can also redirect the output of strace to a file as an argument. Using the -o flag followed by the file path as an argument, we can save strace logs.

For example:

strace -p 3514 -o ~/Desktop/apache_trace

Once the file is saved, you can later monitor and analyze it.

Conclusion

In this guide, we learned how to install and use strace on Major Linux distributions. Now that you understand system calls and how processes work, you can use strace to monitor and debug a running system process running.

The concepts learned in this tutorial are very useful, mainly because you can use what you’ve learned to monitor if anyone is tampering with system processes.

About the author

<img alt="John Otieno" data-lazy-src="https://kirelos.com/wp-content/uploads/2020/12/echo/john-150×150.png5fe91e4139018.jpg" height="112" src="data:image/svg xml,” width=”112″>

John Otieno

Computer science student and resident of Kenya