From Initq

The lsof command lists open files, sockets, and pipes. To learn more about a Unix system, run lsof on them to see what files are held open (such as libraries or log files) and what ports daemons listen to.

# lsof | less -S

However, lsof does not show all the information required to debug a problem. See also netstat for routing table and send and receive queue size information.

Search for Open Files

A single open file can prevent a filesystem from being unmounted. lsof should be run as the superuser (root) to see all open files. The following example shows an open file under the mount point /mnt being used by vim.

# lsof /mnt
vim 1481 user 3u VREG 14,6 4096 306536 /mnt/.test.swp

For more information about a particular process, use the -p option to lsof:

# lsof -p 1481
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
…

To close an open file, the process holding the file open will need to be closed, either by quitting out of it, or via a command like kill. Avoid using the -9 or -KILL options to the kill command if possible.

Show Listen Addresses

Daemons may either bind to the global 0.0.0.0 IPv4 address, or to specific addresses, such as 127.0.0.1 (localhost). A daemon bound to the localhost address will only be reachable from the system itself. Use the -i and -nP options to lsof to show listening ports without lookups on hostnames and services. For example, the following shows the Apache httpd daemon running on localhost at the non-standard port of 8000. Other systems will not be able to connect to this httpd processes: good for security, bad for remote connectivity.

# lsof -i -nP | grep httpd
httpd 2318 apache 16u IPv4 0x019922bc 0t0 TCP 127.0.0.1:8000 (LISTEN)
httpd 2319 apache 16u IPv4 0x019922bc 0t0 TCP 127.0.0.1:8000 (LISTEN)
httpd 2322 apache 16u IPv4 0x019922bc 0t0 TCP 127.0.0.1:8000 (LISTEN)

In contrast, the following OpenSSH sshd process will accept connections from other systems, as it is bound to the 0.0.0.0 address, as indicated by the * preceeding the port number.

# lsof -i -P | grep sshd
sshd 2361 root 3u IPv4 2658 TCP *:22 (LISTEN)

Certain applications listen on many different ports, such as the Berkeley Internet Name Daemon (BIND) named daemon, version 9.

# lsof -i -nP | grep ^named
named 284 named 5u IPv6 0x01388be0 0t0 UDP *:53
named 284 named 6u IPv6 0x01664e80 0t0 TCP *:53 (LISTEN)
named 284 named 7u IPv4 0x01388b10 0t0 UDP 127.0.0.1:53
named 284 named 8u IPv4 0x01870570 0t0 TCP 127.0.0.1:53 (LISTEN)
named 284 named 9u IPv4 0x01388a40 0t0 UDP *:49164
named 284 named 10u IPv6 0x01388970 0t0 UDP *:49165
named 284 named 11u IPv4 0x0186fd54 0t0 TCP *:953 (LISTEN)
named 284 named 13u IPv4 0x01387ee0 0t0 UDP 192.0.2.1:53
named 284 named 14u IPv4 0x01999ce4 0t0 TCP 192.0.2.1:53 (LISTEN)

A process may not work for other reasons, such as a firewall, access service control like tcp_wrappers, or some other misconfiguration. Use ping, telnet, or nmap to check from a remote system whether something else may be blocking the request, or run tcpdump to see whether connections leave the source or arrive at the target system.

Mangle the Current Working Directory

lsof in conjunction with the GNU Project Debugger (GDB) can alter the current working directory of another process, for example if a shell is left open on a mount point that must be remounted. Use gdb at your own risk! A simpler solution: kill the bash process with a HUP signal.

$ lsof | egrep '^C|/nfs/server'
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
bash 17328 jmates cwd VDIR 14,2 102 1456154 /nfs/server

1. Attach gdb to the above bash process

Either launch gdb and attach, or attach directly via the gdb bash 17328 command.

$ gdb -q
(gdb) attach 17328
Attaching to process 17328.
Reading symbols for shared libraries . done
Reading symbols for shared libraries .... done
0x900137a4 in read ()

1. Change the working directory

After the chdir(2) call, use lsof to examine the current working directory.

(gdb) call (int) chdir("/")
$1 = 0
(gdb) shell lsof -p 17328 | fgrep cwd
bash 17328 jmates cwd VDIR 14,2 1224 2 /

1. When done, change the directory back

(gdb) call (int) chdir("/nfs/server")
$2 = 0
(gdb) shell lsof -p 17328 | fgrep cwd
bash 17328 jmates cwd VDIR 14,2 102 1456154 /nfs/server
(gdb) detach
Detaching from process 17328 thread 0xd03.
(gdb) quit

For a list of other functions and system calls available, use the info functions command under gdb, then consult the man pages or source to see how the function should be called.

$ cat show-functions
info functions
detatch
$ gdb -batch -x show-functions bash 17328 > bash.fun

Again, use gdb at your own risk!

Overview

   LiSt Open Files is a useful and powerful tool that will show you opened files. In Unix everything is a file: pipes are files, IP sockets are files, unix sockets are files, directories are files, devices are files, inodes are files...

Useful Examples

   So in this tangle of files lsof listst files opened by processes running on your system.

   When lsof is called without parameters, it will show all the files opened by any processes.

        lsof | nl

   Let us know who is using the apache executable file, /etc/passwd, what files are opened on device /dev/hda6 or who's accessing /dev/cdrom:

        lsof `which apache2`
        lsof /etc/passwd
        lsof /dev/hda6
        lsof /dev/cdrom

   Now show us what process IDs are using the apache binary, and only the PID:

        lsof -t `which apache2`

   Show us what files are opened by processes whose names starts by "k" (klogd, kswapd...) and bash. Show us what files are opened by init:

        lsof -c k
        lsof -c bash
        lsof -c init

   Show us what files are opened by processes whose names starts by "courier", but exclude those whose owner is the user "zahn":

        lsof -c courier -u ^zahn

   Show us the processes opened by user apache and user zahn:

        lsof -u apache,zahn

   Show us what files are using the process whose PID is 30297:

        lsof +p 30297

   Search for all opened instances of directory /tmp and all the files and directories it contains:

        lsof +D /tmp

   List all opened internet sockets and sockets related to port 80:

        lsof -i
        lsof -i :80

   List all opened Internet and UNIX domain files:

        lsof -i -U

   Show us what process(es) has an UDP connection opened to or from the host www.akadia.com at port 123 (ntp):

        lsof -iUDP@www.akadia.com:123

   lsof provides many more options and could be an unvaluable foresinc tool if your system get compromised or as daily basis check tool.