Treasure Trove

Few Linux utilities are as popular as netstat, a fantastic tool used to display network connections, routes, and bandwidth statistics gleaned from network interfaces. Not only is netstat very effective and full of features, but I can't count the number of Linux books and Linux training courses that cover its stock commands as must-have knowledge for any newcomers to Linux.

Don't get me wrong, netstat certainly has its place within those tutorials for both advanced and novice users, but I like to think I can turn to a more powerful tool sometimes. You could say that I've neatly slotted netstat into a place where I know I can use it if I want, but I can achieve the same results in most cases with one of its counterparts – and usually much more efficiently.

If I were making a package comparison and using DNS clients, I might say that netstat is the nslookup to my host command. That said, I will now backtrack a little and state that my alternative to netstat lacks many of its features and, in fact, isn't really a network information tool at all. However, because of this mighty command's formidable capabilities, I rarely turn to netstat at all.

Shopping Lists

My trusty tool is lsof, which is short for list open files. In this article, I'll run through the power of this little tool. In fact, thanks to its functionality, lsof is one of the first packages that I install when building a new server.

Additionally, for Debian and Ubuntu users, I'll unveil a little trick that I've been using on production servers for many years, which thanks to the functionality that lsof provides, has meant that I've hit the desired levels of uptime again and again.

On Debian-based boxes, you can install lsof as simply as:

# apt-get install lsof

I've also seen it built into other Linux flavors.

Among other methods, lsof works by reading the processes from the operating system held within the process table. Apparently, it also queries other aspects of the system to relate which information in the process table is associated with which files on the filesystem. In other words, it cleverly brings together two otherwise hard-to-query and disparate areas of a system: the processes and the files these processes use. It's not just bog standard files either; the highly sophisticated lsof has the visibility of NFS (Network Files System) connections, too.

How Does It Help?

Lsof can output a mountain-sized amount of information about what a system is currently doing, which unquestionably needs to be filtered down to become immediately useful and to satisfy my fondness for brevity.

This information includes, for example, the PID (process number unique to a process), the command the process is executing, the owner of the process (the user that spawned it), all the files a process is currently using, and which shared libraries the process has called as dependencies. (I'll cover shared libraries later.) Then, you'll see a file's access mode, whether a file is locked, its device numbers, its inode number, and a file's local and remote network addresses, to name just a little of its output.

Network

Without meaning to disparage the powerful netstat in any way, I'll start by looking at similarities with netstat, or at least some of the functionality that both packages share, including a simple lookup of TCP port 22, a common server port for SSH daemons. The time-honored netstat can see what's going on with the command:

# netstat -lp

Netstat tries to list all listening daemons (not outbound connections, too) with the -l switch and then offer the PID followed by the name of the program responsible. In Listing 1, it has discovered sshd, the SSH daemon.

The reminder about why I prefer lsof for such a simple query is compounded by the speed of the generated output from netstat: lsof just works, whereas netstat seems to take forever. I'm nitpicking here but the netstat command took a staggering 5.111 seconds according to the time command.

To be fair to netstat, you tend to filter after the command has been run using grep (to filter the output) or something similar. So, to see just port 22 for SSH you might enter:

# netstat -pl | grep ssh

However, lsof only needs a little nudge, and in 0.125 seconds I have the information relating to port 22 only:

# lsof -i:22

The output dutifully shows an IPv4 and a IPv6 daemon running in Figure 1. (Note that I run the command intentionally as root to open the trove of privileged ports and processes on a system.)

Figure 1: Gloriously simple lsof output for TCP port 22.