7 Practical Strace Command Examples for Linux Debugging

What is the strace Command?

Strace is a diagnostic tool for Linux that intercepts and records system calls made by a process. System calls are how programs talk to the kernel, opening files, reading data, spawning threads, or making network connections. When something breaks and logs give you nothing, the strace command shows you the raw conversation between your program and the operating system.

It works without recompiling or modifying your code. You attach it to a running process or launch a command under it. Strace prints every syscall with its arguments and return value. That alone often reveals why something is failing.

1. Tracing a Command with the strace Command

The simplest way to use strace is to prefix any command with it. This shows every system call that command makes from start to finish.

strace ls

You will see a flood of output. Every syscall like execve, openat, stat, and write appears with its arguments and return value. It is a lot to take in at first. But that stream of data is exactly what you need when a program silently fails. You can see which file it tried to open, and whether that file existed.

Try strace cat /etc/hostname. Watch how it first checks if the file exists with stat, then opens it with openat, reads it with read, and writes it to your terminal with write. Each syscall tells one step of the story.

2. Filtering Output with the strace Command -e Option

Strace without filters is noisy. A simple command like ls might produce 100 syscalls. Most of them are library loading and memory mapping that you do not care about. Use the -e flag to trace only the syscalls you want.

strace -e openat,read,write ls

This only shows openat, read, and write calls. If you are debugging a file permission issue, this is the filter you want. You see exactly which files the program tries to open and whether those calls succeed or return errors like EACCES or ENOENT.

The -e flag also supports negative filtering. Pass -e !write to exclude write syscalls and cut down noise that way. Combine it with other flags for targeted debugging sessions.

3. Attaching to a Running Process with the strace Command -p Flag

You do not have to launch a program from scratch. If a process is already running and misbehaving, attach strace to its PID.

sudo strace -p 2345

Replace 2345 with your target process ID. It hooks into the running process and starts printing syscalls in real time. This is incredibly useful for debugging services like Nginx or PostgreSQL that are already running when a problem surfaces.

Find the PID with pgrep first. For example, pgrep nginx returns the process IDs, and you attach it to the one that interests you. Hit Ctrl+C to detach when you have enough data.

4. Saving Output to a File with the strace Command -o Flag

The terminal output from strace can scroll by too fast to read. Redirecting it to a file lets you inspect it at your own pace.

strace -o /tmp/strace.log ls

All syscall traces go to /tmp/strace.log instead of the terminal. You can then open that file in less or your editor and search for specific calls. I use this constantly when debugging build failures. A failed make might spit out one cryptic error, but the strace log shows every missing header file and permission denied that led to it.

Combine -o with -e to save only what you need. strace -o /tmp/log.txt -e openat,stat make writes just the file access traces from a build process.

5. Tracing Child Processes with the strace Command -f Flag

Most servers and services spawn child processes. Strace by default only traces the parent. To follow children, add the -f flag, which traces all forked processes.

strace -f -o /tmp/nginx_trace.txt nginx

Each line in the output includes the PID of the process that made the syscall. This helps you tell worker processes apart from the master process. When debugging a web server that returns 500 errors on some requests but not others, tracing all child processes with -f shows you exactly which one fails and why.

There is also -ff which splits output into separate files, one per child process. That is useful when you have many workers and need to inspect each one independently.

6. Displaying Only Errors with the strace Command -z Flag

Sometimes you do not need to see every syscall. You only want to know which ones failed. The -z flag of the strace command prints only syscalls that returned an error.

strace -z -o /tmp/errors.txt ./myapp

This filters out all the successful calls and leaves you with a concise list of failures. If a configuration file is missing or a permission is wrong, the failed syscall shows you the exact error code and the file path involved. This is the fastest way to diagnose startup failures in applications.

7. Getting Timestamps and Call Statistics with the strace Command

It can show you when syscalls happened and how long they took. This helps with performance debugging.

Use -t to prepend a timestamp to each call:

strace -t ls

Use -tt for microsecond precision or -ttt for Unix epoch timestamps. These are useful when correlating strace output with application logs.

For a summary instead of a full trace, use the -c flag:

strace -c ls

This prints a table showing how many times each syscall was called, the total time spent, and how many errors occurred. It is great for finding performance bottlenecks. If a program calls stat ten thousand times on the same path, you have a caching problem.

Practical Debugging Example with the strace Command

Here is a real scenario. Your application says it cannot open a config file. The error message is vague. Run it under the strace command:

strace -e openat,stat -o /tmp/debug.log ./myapp

Open the log file and search for ENOENT. You will see exactly which path the application tried to open and what path actually exists. Nine times out of ten, the issue is a hardcoded path that does not match your system, or a trailing whitespace in a variable. Strace does not guess, it shows you the facts.

Another common use case. A web server is slow. Use strace -c against one of its worker processes to see where time is going. If poll or epoll_wait dominates, the issue is network latency. If read on a disk file dominates, you have an I/O bottleneck.

When Not to Use the strace Command

Strace is powerful, but it comes with overhead. Every traced syscall slows the process down, sometimes by an order of magnitude. Do not run it on production under load unless you have no other choice. Use it on staging or during maintenance windows.

Also, strace does not trace everything. It shows system calls, but not what happens inside the program between calls. For that you need a debugger like GDB. And strace does not show signals that are blocked or ignored. Pair it with ltrace if you need library call tracing too.

Further Reading

The man page has every option in detail. Run man strace or check the online strace man page for the full reference. The strace guide on Linux Blog has more examples worth reading.

For more Linux debugging tools on this site, check out the htop features guide for real time system monitoring, the btop monitoring tutorial for a modern alternative, and the tcpdump tutorial for network level debugging.