cheat sheet

lsof & ss

Diagnose what's holding a port, which files a process has open, and the state of every TCP/UDP socket using lsof and the modern iproute2 ss utility.

lsof & ss — Open Files and Sockets

What it is

lsof (list open files) and ss (socket statistics) are the two diagnostic utilities most Linux developers reach for when something networking-related misbehaves. lsof was written by Vic Abell in 1994 and treats Unix's "everything is a file" maxim literally — it lists every open file descriptor on the system: regular files, sockets, pipes, devices, and deleted-but-still-held files. ss is part of the iproute2 suite, written by Alexey Kuznetsov, and is the modern replacement for the deprecated netstat; it reads directly from the kernel's netlink interface, which makes it dramatically faster on busy servers. Reach for lsof when you need cross-cutting "who has what open" answers (files, sockets, deleted inodes), and ss when you specifically need a fast, kernel-accurate view of TCP/UDP/UNIX sockets.

Choosing between them

The two tools overlap on sockets but diverge on everything else, and they perform very differently at scale.

TaskUseWhy
What's listening on port 8080?ss -tlnp | grep 8080 or lsof -i :8080Both work; ss is faster
Process holding a deleted log filelsof | grep deletedss only knows sockets
Files open under /var/loglsof +D /var/logss doesn't track files
Count connections by TCP statess -tanlsof is far slower for this
Filter by remote address/portss dst 10.0.0.1Kernel-side filter, very fast
macOS / BSD without iproute2lsof -iss is Linux-only
Quick socket summaryss -sBuilt-in counter

On a server with tens of thousands of connections, ss -tan returns in milliseconds while lsof -i can take several seconds. Default to ss for socket questions and only fall back to lsof when you need its file-side superpowers.

On modern Linux distros there's a third option: lsfd (ships in util-linux ≥ 2.38) is a Linux-native, libsmartcols-based successor to lsof with JSON output, a real filter expression language, and namespace/cgroup awareness. It's not a drop-in replacement — see the lsfd section below for the CLI and a translation table.

Install

Both tools are typically pre-installed on every modern Linux distribution; the package names differ when they're missing.

bash
# Debian/Ubuntu
sudo apt install lsof iproute2

# Fedora/RHEL
sudo dnf install lsof iproute

# Alpine
sudo apk add lsof iproute2

# macOS — lsof ships with the base system; ss is Linux-only
brew install lsof   # already present, but Homebrew has a newer build

Output: (none — exits 0 on success)


lsof — list open files

lsof walks /proc/<pid>/fd/ for every running process and prints one row per open file descriptor. Because Unix represents nearly every kernel resource as a file, this single tool reveals network sockets, log files, mmaped libraries, pipes, and devices.

Syntax

lsof takes filters that narrow the result set; with no arguments it prints every open file on the system, which on a typical server is tens of thousands of rows.

bash
lsof [OPTIONS] [filter...]

Output: (none — exits 0 on success)

Essential options

OptionMeaning
-i [proto][@host][:port]Internet sockets (e.g. -i :22, -i tcp@10.0.0.1)
-p PIDFiles opened by a process
-u USERFiles opened by a user
-c COMMANDFiles opened by processes named COMMAND
+D DIRFiles open under a directory tree (recursive)
-d FDFilter by file descriptor (e.g. -d 0-2 for stdio)
-tTerse output: PIDs only — for piping to kill
-nNo DNS lookups (much faster)
-PNo port-name lookups (show :80 not :http)
-r NRepeat every N seconds (lightweight monitor)
-aLogical AND between filters (default is OR)

Output columns

lsof prints one row per open descriptor with these columns. Reading them is most of the skill — once you can spot deleted files or sockets in LISTEN state at a glance, the tool becomes very fast to use.

bash
lsof -i :22

Output:

text
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd     800 root    3u  IPv4  12345      0t0  TCP *:ssh (LISTEN)
sshd     800 root    4u  IPv6  12346      0t0  TCP *:ssh (LISTEN)
sshd    8800 root    4u  IPv4  98765      0t0  TCP 10.0.0.10:ssh->10.0.0.5:52314 (ESTABLISHED)
ColumnMeaning
COMMANDProcess name (truncated to 9 chars by default; use +c 0 for full)
PIDProcess ID
USEROwning user
FDFD number + access mode (r, w, u); special: cwd, txt, mem, rtd
TYPEIPv4, IPv6, REG, DIR, CHR, FIFO, unix, sock
DEVICEMajor,minor for block/char; inode hash for sockets
SIZE/OFFFile size or offset
NODEInode number, or protocol for sockets (TCP, UDP)
NAMEPath, or host:port->host:port (STATE) for sockets

What's holding port 8080?

The most common single-line use of lsof. Add sudo if the socket is owned by another user — without it, the row will be missing.

bash
sudo lsof -i :8080
sudo lsof -iTCP:8080 -sTCP:LISTEN     # only the LISTEN socket

Output:

text
COMMAND  PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
node    9200 alice   19u  IPv6  87654      0t0  TCP *:http-alt (LISTEN)

Find files held open by a process

-p PID lists every descriptor for a process — useful when a service has spawned children and you want to know which library, log, or socket each one owns. Combine with -d to narrow to specific FDs.

bash
lsof -p 1234
lsof -p 1234 -d 0,1,2          # only stdin/stdout/stderr
lsof -p 1234 -ad txt           # only executable + libraries
lsof -p 1234,5678,9012         # multiple PIDs

Output (lsof -p 1234):

text
COMMAND  PID USER   FD   TYPE DEVICE  SIZE/OFF     NODE NAME
nginx   1234 root  cwd    DIR  259,1      4096        2 /
nginx   1234 root  rtd    DIR  259,1      4096        2 /
nginx   1234 root  txt    REG  259,1   1369712  1310771 /usr/sbin/nginx
nginx   1234 root  mem    REG  259,1    100728   524301 /lib/x86_64-linux-gnu/libnsl.so.1
nginx   1234 root    0u   CHR    1,3       0t0        6 /dev/null
nginx   1234 root    1u   CHR    1,3       0t0        6 /dev/null
nginx   1234 root    2w   REG  259,1     45221  1572873 /var/log/nginx/error.log
nginx   1234 root    6u  IPv4  23456       0t0      TCP *:http (LISTEN)

Held-open deleted files

When a service rotates or truncates a log file but the writing process still holds the old inode open, disk space is not reclaimed until the process exits or closes the FD. lsof | grep deleted reveals these zombie files — the classic "df shows full but du can't find anything" case.

bash
sudo lsof | grep deleted
sudo lsof +L1                  # files with link count < 1 (deleted but open)

Output:

text
java     4521 alice    7w   REG  259,1  10485760000  1572881 /var/log/app/server.log (deleted)
mysqld   2100 mysql   18u   REG  259,1   1048576000  1572883 /var/lib/mysql/ib_logfile0 (deleted)

The fix is usually to send the process a signal that makes it close and reopen its log file (e.g. kill -HUP nginx), or — last resort — restart the service.

Files open under a directory tree

+D walks the directory recursively and reports every file beneath it that any process has open. Use this before unmounting a filesystem or deleting a directory to find what's still using it.

bash
sudo lsof +D /var/log
sudo lsof +D /mnt/usb           # who's holding the USB drive open?

Output:

text
COMMAND   PID     USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
rsyslogd  611     root    7w   REG  259,1   234560 1572880 /var/log/syslog
nginx    1234 www-data   2w   REG  259,1    45221 1572873 /var/log/nginx/error.log
nginx    1234 www-data   3w   REG  259,1   128304 1572874 /var/log/nginx/access.log
journald  423     root    8w   REG  259,1   524288 1572879 /var/log/journal/abc/system.journal

Network connections

-i is the flagship socket filter and accepts a small grammar: protocol, @host, and :port in any combination. Always add -n -P for speed — DNS and service-name lookups can take seconds per row on a slow resolver.

bash
lsof -i                          # all internet sockets
lsof -i -n -P                    # fast: no name lookups
lsof -i tcp                      # all TCP
lsof -i tcp:443                  # port 443 only
lsof -i @10.0.0.1                # to/from a host
lsof -i tcp@10.0.0.1:443         # full filter
lsof -iTCP -sTCP:ESTABLISHED     # only established TCP
lsof -iTCP -sTCP:LISTEN          # only listening

Output (lsof -iTCP -sTCP:ESTABLISHED -n -P):

text
COMMAND   PID  USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd     8800  root    4u  IPv4  98765      0t0  TCP 10.0.0.10:22->10.0.0.5:52314 (ESTABLISHED)
nginx    1235 nginx   12u  IPv4 100001      0t0  TCP 10.0.0.10:443->203.0.113.42:49182 (ESTABLISHED)
chrome   4521 alice   23u  IPv4 100002      0t0  TCP 10.0.0.10:55210->203.0.113.42:443 (ESTABLISHED)

Files by user or command

-u filters by username, -c by command name. By default these are OR'd with other filters; add -a to make them AND.

bash
lsof -u alice                            # everything alice has open
lsof -u alice -a -i                      # alice's network sockets only
lsof -u '^root' -i                       # exclude root (note the caret)
lsof -c nginx                            # all nginx processes
lsof -c nginx -c apache2                 # multiple commands

Output: (none — exits 0 on success)

Terse output for piping

-t prints only PIDs, one per line — ideal as input to kill. Combined with -i this is the cleanest "kill whatever is on this port" recipe.

bash
lsof -t -i :8080                              # PIDs only
kill -TERM $(lsof -t -i :8080)                # kill them
kill -9   $(lsof -t -i :8080) 2>/dev/null     # force, ignore "no such PID"

Output (lsof -t -i :8080):

text
9200

Continuous monitoring

-r N re-runs the query every N seconds, prefixing each batch with =====. Useful for watching connection churn without firing up top or watch.

bash
lsof -i :443 -r 2 -n -P

Output: (none — exits 0 on success)


ss — socket statistics

ss is the iproute2 replacement for the deprecated netstat. It reads kernel data through netlink rather than parsing /proc/net/*, which makes it both faster and more accurate (no torn reads on a busy box).

Syntax

ss flags select which socket families to include and how to format them; filter expressions further narrow by state, address, and port. The default invocation lists established TCP connections only — surprisingly minimal.

bash
ss [OPTIONS] [FILTER]

Output: (none — exits 0 on success)

Essential options

OptionMeaning
-tTCP sockets
-uUDP sockets
-xUNIX domain sockets
-wRaw sockets
-lListening sockets only
-aAll sockets (listening + connected)
-nNumeric (no DNS / service lookups)
-pShow owning process (needs root for others')
-eExtended info (UID, inode, socket cookie)
-oShow timer info (retransmit, keepalive)
-iInternal TCP info (cwnd, RTT, ssthresh)
-mSocket memory usage
-sOne-page summary
-4 / -6Restrict to one address family
-KForcibly close matching sockets (root)
-ZShow SELinux context
-M / --mptcpDisplay MPTCP (multipath TCP) sockets — iproute2 ≥ 5.16
--cgroupAppend the cgroup v2 path that owns each socket — iproute2 ≥ 5.18
-HSuppress the header row (script-friendly)
-N <netns>Switch into a named network namespace before querying

The canonical invocation

ss -tulnp is what most engineers type first: TCP + UDP, listening only, numeric, with process info. Add sudo to see sockets owned by other users.

bash
sudo ss -tulnp

Output:

text
Netid State   Recv-Q Send-Q   Local Address:Port    Peer Address:Port  Process
tcp   LISTEN  0      128            0.0.0.0:22           0.0.0.0:*     users:(("sshd",pid=800,fd=3))
tcp   LISTEN  0      511            0.0.0.0:80           0.0.0.0:*     users:(("nginx",pid=1234,fd=6))
tcp   LISTEN  0      511            0.0.0.0:443          0.0.0.0:*     users:(("nginx",pid=1234,fd=8))
tcp   LISTEN  0      128          127.0.0.1:5432         0.0.0.0:*     users:(("postgres",pid=2100,fd=5))
udp   UNCONN  0      0              0.0.0.0:68           0.0.0.0:*     users:(("dhclient",pid=345,fd=6))

Summary statistics

ss -s prints a one-screen overview of socket counts by family and TCP state. It is the fastest way to spot, for example, a TIME-WAIT explosion or unexpected CLOSE-WAIT accumulation.

bash
ss -s

Output:

text
Total: 142
TCP:   38 (estab 24, closed 8, orphaned 0, timewait 8)

Transport Total     IP        IPv6
RAW       0         0         0
UDP       6         4         2
TCP       30        22        8
INET      36        26        10
FRAG      0         0         0

Filtering by state

State filters are evaluated kernel-side, so they're cheap even on systems with millions of sockets. The TCP states ss knows are established, syn-sent, syn-recv, fin-wait-1, fin-wait-2, time-wait, closed, close-wait, last-ack, listening, and closing.

bash
ss -tn state established
ss -tn state time-wait
ss -tn state listening
ss -tn state connected           # everything except listen + closed
ss -tn state bucket              # time-wait + syn-recv

Output (ss -tn state established):

text
Recv-Q Send-Q   Local Address:Port    Peer Address:Port
0      0          10.0.0.10:22           10.0.0.5:52314
0      0          10.0.0.10:443      203.0.113.42:49182
0      0          10.0.0.10:443      203.0.113.99:51033

Filtering by address and port

The filter language supports comparisons against src, dst, sport, dport, and combinations with or / and. Operators must be quoted to keep the shell from interpreting them.

bash
ss -tn dst :443                                # connections to remote port 443
ss -tn dport = :443                            # same, explicit form
ss -tn 'dport >= :8000 and dport <= :8999'     # port range
ss -tn dst 10.0.0.1                            # specific remote host
ss -tn dst 10.0.0.0/24                         # subnet
ss -tn 'sport = :80 or sport = :443'           # multiple source ports
ss -tn '( dport = :22 or sport = :22 )'        # either direction

Output (ss -tn dst :443):

text
Recv-Q Send-Q   Local Address:Port    Peer Address:Port
0      0         10.0.0.10:55210      203.0.113.42:443
0      0         10.0.0.10:55214      203.0.113.99:443

Process column

-p appends a users:(("name",pid=N,fd=M)) block to each row. Add -e for the UID and inode, which lets you cross-reference with /proc/<pid>/.

bash
sudo ss -tnp                            # all TCP with process
sudo ss -tlnp sport = :8080             # who owns port 8080?
sudo ss -tnpe state established         # extended info + process

Output (sudo ss -tlnp sport = :8080):

text
State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port  Process
LISTEN 0      128           0.0.0.0:8080         0.0.0.0:*   users:(("node",pid=9200,fd=19))

Counting by state

A common diagnostic recipe is to histogram TCP state — a healthy server has lots of ESTAB and modest TIME-WAIT; lots of CLOSE-WAIT usually means the application is leaking sockets.

bash
ss -tan | awk 'NR>1 {print $1}' | sort | uniq -c | sort -rn

Output:

text
     24 ESTAB
      8 TIME-WAIT
      4 LISTEN
      2 CLOSE-WAIT
      1 SYN-SENT

TCP internals

-i adds a second line per connection with congestion-window, RTT, and retransmit counters — handy when investigating slow connections or congestion.

bash
sudo ss -tni state established

Output:

text
ESTAB 0 0 10.0.0.10:443  203.0.113.42:49182
     cubic wscale:7,7 rto:204 rtt:3.21/1.5 ato:40 mss:1448 cwnd:10 bytes_acked:1432 segs_out:9 segs_in:7

Cgroup ownership

--cgroup annotates each row with the cgroup v2 pathname of the owning process — invaluable on systemd-managed hosts where every service lives in its own slice and you want to attribute a socket back to a unit without an extra lookup.

bash
sudo ss -tlnp --cgroup
sudo ss -tn state established --cgroup | head

Output:

text
State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port  Process  Cgroup
LISTEN 0      511           0.0.0.0:443          0.0.0.0:*   users:(("nginx",pid=1234,fd=8))  /system.slice/nginx.service
LISTEN 0      128           0.0.0.0:22           0.0.0.0:*   users:(("sshd",pid=800,fd=3))    /system.slice/ssh.service

MPTCP (multipath TCP)

-M (or --mptcp) lists MPTCP master sockets — a single logical TCP connection that the kernel is multiplexing across several subflows. Pair with -i to see each subflow's congestion window. Requires iproute2 ≥ 5.16 and a kernel with CONFIG_MPTCP=y.

bash
ss -Mn
sudo ss -Mnip                  # with TCP internals + owning process

Output:

text
State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port
ESTAB  0      0          10.0.0.10:443      203.0.113.42:55301
       tcp-ulp-mptcp flags:Mec token:0000(id:0)/abcd1234(id:0) seq:1234 sfseq:1 ssnoff:0 maplen:0

UNIX domain sockets

ss -x enumerates UNIX-domain sockets, the IPC mechanism behind Docker, system buses, and many local daemons. Path-based sockets show as filesystem paths; abstract sockets (Linux-only) start with @.

bash
ss -xl                       # listening UNIX sockets
ss -xa                       # all UNIX sockets
sudo ss -xlp                 # with owning processes
ss -x 'src @docker'          # abstract-namespace match

Output (ss -xl):

text
Netid  State    Recv-Q Send-Q  Local Address:Port   Peer Address:Port
u_str  LISTEN   0      4096    /run/systemd/private   *
u_str  LISTEN   0      128     /var/run/docker.sock   *
u_str  LISTEN   0      4096    /run/dbus/system_bus_socket  *

Force-close a socket

-K asks the kernel to abort matching sockets — useful for clearing a stuck TIME-WAIT glut or evicting a misbehaving client, but it requires root and a recent kernel (≥ 4.9).

bash
sudo ss -K dst 10.0.0.99           # close all connections to a host
sudo ss -K state time-wait         # clear time-wait

Output: (none — exits 0 on success)



lsfd — the Linux-native lsof successor

lsfd (list file descriptors) ships in util-linux since v2.38 (2022) and is the actively maintained, Linux-specific successor to lsof. It is not a drop-in replacement — the CLI and output columns differ — but on Linux it is faster, namespace-aware, cgroup-aware, and integrates with libsmartcols so columns, JSON output, and filter expressions are all first-class. Reach for lsfd on a modern distro when you want scriptable output or richer per-FD metadata; keep lsof for portability to macOS and BSD.

Install

bash
sudo apt install util-linux-extra         # Debian/Ubuntu (lsfd lives in the -extra split)
sudo dnf install util-linux-core          # Fedora/RHEL
sudo apk add util-linux                   # Alpine

Output: (none — exits 0 on success)

Default invocation

With no arguments, lsfd prints every open file descriptor on the system, one per row. Columns are tabular by default; pass -J for JSON or -r for raw.

bash
lsfd | head
lsfd -J | jq '.[][] | select(.type == "REG")'  # JSON pipeline

Output:

text
COMMAND  PID  USER  ASSOC  MODE TYPE   SOURCE     MNT-ID INODE NAME
systemd    1  root  cwd    ---  DIR    dm-0           29     2 /
systemd    1  root  exe    ---  REG    dm-0           29  4720 /usr/lib/systemd/systemd
systemd    1  root  0      r--  CHR    devtmpfs        6     6 /dev/null
sshd     800  root  3u     rw-  SOCK   sockfs          7 12345 TCP 0.0.0.0:22 LISTEN

Filter expression language

Unlike lsof's positional filters, lsfd uses a single expression via -Q/--filter borrowed from libsmartcols. Operators are familiar: ==, !=, =~ (regex), and/&&, or/||, plus boolean column predicates like DELETED.

bash
lsfd -Q '(PID == 1) or (PID == 2)'         # files held by pid 1 or 2
lsfd -Q '(PID == 1234) and (FD == 1)'      # stdout of one process
lsfd -Q 'ASSOC == "exe"'                   # every running executable
lsfd -Q 'DELETED'                          # held-open deleted files
lsfd -Q 'TYPE != "REG"'                    # everything except plain files
lsfd -Q 'NAME =~ "/var/log/.*"'            # regex on path

Output (lsfd -Q 'DELETED'):

text
COMMAND  PID  USER  ASSOC  MODE TYPE SOURCE      MNT-ID    INODE NAME
java    4521 alice  7      -w-  REG  dm-0            29  1572881 /var/log/app/server.log
mysqld  2100 mysql  18     rw-  REG  dm-0            29  1572883 /var/lib/mysql/ib_logfile0

Custom columns

-o selects columns; -l lists every column known to the build. The column set is much richer than lsof's — NS.TYPE, XMODE, POS, INET.LADDR, SOCK.PROTONAME, and namespace IDs are all available.

bash
lsfd -l                                          # list every column
lsfd -o PID,COMMAND,FD,TYPE,NAME -Q 'TYPE == "SOCK"'
lsfd -o +SOCK.PROTONAME,INET.LADDR,INET.RADDR    # append columns

Output (lsfd -o PID,COMMAND,FD,TYPE,NAME -Q 'TYPE == "SOCK"' | head):

text
PID  COMMAND  FD  TYPE  NAME
800  sshd     3   SOCK  TCP 0.0.0.0:22 LISTEN
1234 nginx    6   SOCK  TCP 0.0.0.0:80 LISTEN
1234 nginx    8   SOCK  TCP 0.0.0.0:443 LISTEN

Counters and summaries

--summary=only skips the per-FD rows and prints only an aggregate count per type — handy for watch and dashboards.

bash
lsfd --summary=only
lsfd --summary=only -Q '(UID == 1000)'      # per-user totals

Output:

text
   COUNT TYPE
   12451 REG
    2104 SOCK
     980 CHR
     520 FIFO
     312 DIR
      88 unknown

lsof → lsfd quick map

lsof invocationlsfd equivalent
lsof -p 1234lsfd -Q 'PID == 1234'
lsof -u alicelsfd -Q 'UID == 1000'
lsof | grep deletedlsfd -Q 'DELETED'
lsof -c nginxlsfd -Q 'COMMAND == "nginx"'
lsof -i :22lsfd -Q 'NAME =~ ":22"' (or fall back to ss)
lsof +D /var/loglsfd -Q 'NAME =~ "^/var/log/"'

lsfd doesn't have a dedicated "internet sockets" filter — for port-level questions on Linux, ss remains the right tool. Use lsfd when you want a single tool to answer file and socket questions with structured output.


Common pitfalls

  1. No process column without rootlsof -i and ss -p only show the users:(...) field for sockets your user owns. Always prefix with sudo when diagnosing a foreign service.
  2. Slow DNS lookups — both tools resolve addresses and port names by default. Add -n -P to lsof and -n to ss for an immediate 10–100× speedup on hosts with sluggish resolvers.
  3. Truncated command nameslsof clips commands to 9 characters. Pass +c 0 for full names, otherwise chrome_crashpa is really chrome_crashpad_handler.
  4. ss state names — the filter language uses kebab-case (time-wait, close-wait), not the upper-case names printed in output (TIME-WAIT, CLOSE-WAIT). Both forms are accepted in modern builds but only kebab-case is portable.
  5. lsof | grep racing with itselflsof may take seconds to enumerate /proc on a busy box; processes can come and go between rows. For listing the current set of open sockets, prefer ss.
  6. macOS lsof -i requires sudo — unlike Linux, macOS hides socket info from unprivileged users entirely, not just other users' sockets.
  7. ss is Linux-only — scripts that need to run on macOS or FreeBSD should fall back to lsof -i or the platform's netstat -an.
  8. Logical OR by default in lsoflsof -u alice -i :22 shows files that are alice's or on port 22. Use -a to AND them: lsof -a -u alice -i :22.

Real-world recipes

What's holding port 8080?

The classic "address already in use" scenario. Either tool works; pick the one you remember faster.

bash
sudo ss -tlnp sport = :8080
sudo lsof -iTCP:8080 -sTCP:LISTEN -n -P

Output:

text
State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port  Process
LISTEN 0      128           0.0.0.0:8080         0.0.0.0:*   users:(("node",pid=9200,fd=19))

Kill whatever is on a port

Useful inside dev scripts that need to clear a port before re-launching a server.

bash
sudo kill -TERM $(sudo lsof -t -iTCP:8080 -sTCP:LISTEN)

# Alternative using ss + awk
sudo ss -tlnp 'sport = :8080' | awk -F'pid=' 'NR>1 {split($2,a,","); print a[1]}' | xargs -r sudo kill -TERM

Output: (none — exits 0 on success)

Find the deleted-but-open log eating disk

When df says the disk is 95% full but du -sh /var/log/* accounts for only a fraction, a process is holding deleted log files open.

bash
sudo lsof +L1 | awk '$NF=="(deleted)" {print}'

Output:

text
java     4521 alice    7w   REG  259,1  10485760000  1572881 /var/log/app/server.log (deleted)
mysqld   2100 mysql   18u   REG  259,1   1048576000  1572883 /var/lib/mysql/ib_logfile0 (deleted)

Free the space without restarting the service by truncating its FD directly:

bash
sudo : > /proc/4521/fd/7

Output: (none — exits 0 on success)

Count established connections per remote IP

Useful for spotting a noisy client or a runaway scraper.

bash
ss -tn state established | awk 'NR>1 {print $5}' | cut -d: -f1 | sort | uniq -c | sort -rn | head

Output:

text
     42 203.0.113.42
     17 10.0.0.5
      8 192.168.1.101
      3 198.51.100.7

Show who's currently SSH'd in

ss gives a quick snapshot; pair with who for human-readable login times.

bash
sudo ss -tnp state established sport = :22
who

Output:

text
State Recv-Q Send-Q  Local Address:Port  Peer Address:Port  Process
ESTAB 0      0          10.0.0.10:22       10.0.0.5:52314   users:(("sshd",pid=8800,fd=4))
ESTAB 0      0          10.0.0.10:22       10.0.0.7:54812   users:(("sshd",pid=9100,fd=4))

alice    pts/0   2026-05-24 10:14 (10.0.0.5)
bob      pts/1   2026-05-24 10:34 (10.0.0.7)

Find which interface a connection uses

ss -tin shows the TCP info line including the local interface for established connections — handy on multi-homed hosts.

bash
sudo ss -tin state established dst 203.0.113.42

Output:

text
ESTAB 0 0 10.0.0.10:443  203.0.113.42:49182
     cubic wscale:7,7 rto:204 rtt:3.21/1.5 ato:40 mss:1448 cwnd:10

Identify the process listening on every privileged port

Audit script — every TCP socket below port 1024 with its owning process.

bash
sudo ss -tlnp | awk 'NR==1 || $4 ~ /:[0-9]+$/' | awk 'NR==1 || ($4 ~ /:([0-9]|[1-9][0-9]|[1-9][0-9][0-9]|10[0-1][0-9]|102[0-3])$/)'

Output:

text
State  Recv-Q Send-Q  Local Address:Port  Peer Address:Port  Process
LISTEN 0      128           0.0.0.0:22           0.0.0.0:*   users:(("sshd",pid=800,fd=3))
LISTEN 0      511           0.0.0.0:80           0.0.0.0:*   users:(("nginx",pid=1234,fd=6))
LISTEN 0      511           0.0.0.0:443          0.0.0.0:*   users:(("nginx",pid=1234,fd=8))

Files held open under a mountpoint before unmount

Run this before umount so you know what to stop.

bash
sudo lsof +D /mnt/usb
sudo fuser -vm /mnt/usb    # complementary tool: same answer, terser

Output (lsof +D /mnt/usb):

text
COMMAND   PID  USER  FD   TYPE DEVICE SIZE/OFF NODE NAME
vlc      4810 alice  18r   REG    8,1 12345678   42 /mnt/usb/movie.mkv
bash     8821 alice  cwd   DIR    8,1     4096    2 /mnt/usb

Live-watch TIME-WAIT count

A spike in TIME-WAIT often indicates many short connections — usually a misconfigured client doing connect/disconnect per request instead of keep-alive.

bash
watch -n 1 'ss -tan state time-wait | wc -l'

Output: (none — exits 0 on success)

Tips

ss and lsof answer overlapping questions, but for any script you write, prefer ss for socket queries — it's faster, has a kernel-side filter language, and is the only one that scales to high-connection workloads. Reserve lsof for its file-side superpowers.

The deleted-file recovery trick (: > /proc/PID/fd/N) works for files but not sockets. For sockets, use ss -K to ask the kernel to abort them — but be aware this can hang or crash applications that aren't prepared for an abrupt close.

Both tools support -Z to show SELinux context, which is essential when diagnosing "permission denied" errors on RHEL/CentOS systems where AVC denials are silent in normal logs.

Sources