cheat sheet
tr & xargs
Translate, squeeze, and delete characters with tr. Build and execute command lines from stdin with xargs. Includes parallel execution, NUL safety, and pipeline recipes.
tr & xargs — Transform and Execute
What it is
tr and xargs are POSIX-standard utilities that fill two essential gaps in shell pipelines: tr translates, squeezes, or deletes individual characters in a byte stream, while xargs converts lines from stdin into arguments for an arbitrary command — optionally running invocations in parallel. Both are installed on every Unix and Linux system as part of GNU coreutils. Reach for tr when you need character-level transformations (case conversion, stripping non-printable characters) and xargs when you want to fan out a list of items from stdin into parallel or batched command executions.
tr — Translate Characters
tr reads stdin and translates, squeezes, or deletes characters. It operates on individual bytes/characters, not lines.
Syntax
tr [OPTIONS] SET1 [SET2]
Output: (none — exits 0 on success)
| Option | Meaning |
|---|---|
-d SET | Delete characters in SET |
-s SET | Squeeze consecutive chars in SET to one |
-c SET | Complement (all chars NOT in SET) |
-C SET | Complement for multibyte chars |
Translation (SET1 → SET2)
When two sets are given, tr maps each character in SET1 to the corresponding character in SET2 — if SET2 is shorter it is padded with its last character. This covers case conversion, character substitution, and classic ciphers like ROT13.
echo "Hello World" | tr 'a-z' 'A-Z' # lowercase → uppercase
echo "Hello World" | tr 'A-Z' 'a-z' # uppercase → lowercase
echo "hello" | tr 'aeiou' '*' # replace vowels
echo "abc" | tr 'abc' 'xyz' # map a→x, b→y, c→z
# ROT13
echo "secret" | tr 'A-Za-z' 'N-ZA-Mn-za-m'
Output:
HELLO WORLD
hello world
h*ll*
xyz
frperg
Delete characters
-d SET removes every character that appears in SET from the stream, without a replacement. Common uses are stripping carriage returns from Windows files, removing digits, or joining all lines by deleting newlines.
echo "hello 123" | tr -d '0-9' # remove digits
echo "line\r" | tr -d '\r' # remove carriage returns (CRLF → LF)
echo "abc xyz" | tr -d ' ' # remove all spaces
echo "a,b,,c" | tr -d ',' # remove commas
cat file | tr -d '\n' # join all lines (remove newlines)
Output:
hello
line
abcxyz
abbc
Squeeze repetitions
-s SET collapses any run of consecutive identical characters that appear in SET into a single instance. The most common use is normalising inconsistent whitespace — multiple spaces or blank lines — before further processing.
echo "hello world" | tr -s ' ' # collapse spaces → single space
echo "aabbcc" | tr -s 'a-z' # squeeze each run of repeated letters
echo "line1\n\nline3"| tr -s '\n' # remove blank lines
Output:
hello world
abc
line1
line3
Complement
-c inverts SET1 so the operation applies to every character not in the set. Combined with -d it strips everything except the allowed characters; combined with a translation it replaces all "other" characters with a single substitute (e.g., turn non-alphanumeric into underscores).
# -c SET means "all characters NOT in SET"
echo "abc 123 !?" | tr -cd '0-9\n' # keep only digits (and newlines)
echo "abc 123" | tr -c '[:alnum:]' '_' # replace non-alphanumeric with _
Output:
123
abc_123
Character classes
| Class | Characters |
|---|---|
[:alpha:] | Letters |
[:lower:] | Lowercase letters |
[:upper:] | Uppercase letters |
[:digit:] | Digits 0–9 |
[:alnum:] | Letters + digits |
[:space:] | Whitespace (space, tab, newline…) |
[:blank:] | Space and tab only |
[:punct:] | Punctuation |
[:print:] | Printable characters |
tr '[:lower:]' '[:upper:]' # canonical case conversion
tr -d '[:punct:]' # strip punctuation
tr -s '[:space:]' '\n' # split words onto separate lines
tr -cd '[:print:]\n' # strip non-printable characters
Output: (none — exits 0 on success)
Practical tr recipes
# Convert Windows CRLF to Unix LF
tr -d '\r' < windows.txt > unix.txt
# Tokenise text into one word per line
tr -sc '[:alpha:]' '\n' < file.txt
# Count word frequency (combined with sort/uniq)
tr -sc '[:alpha:]' '\n' < essay.txt | tr '[:upper:]' '[:lower:]' \
| sort | uniq -c | sort -rn | head -20
# Generate a simple password (alphanumeric only)
tr -dc '[:alnum:]' < /dev/urandom | head -c 16
# Remove all blank lines
tr -s '\n' < file.txt
# Convert tabs to spaces (4-space equivalent)
expand -t 4 file.txt # better use expand; tr can't count
Output:
# tr -dc '[:alnum:]' < /dev/urandom | head -c 16 example:
mK9rZx4QpL2nWvA8
xargs — Build and Execute Commands
xargs reads items from stdin (delimited by whitespace or NUL) and passes them as arguments to a command.
Syntax
xargs [OPTIONS] [CMD [INITIAL-ARGS]]
Output: (none — exits 0 on success)
Basic usage
echo "a b c" | xargs echo "args:" # args: a b c
cat files.txt | xargs rm # delete listed files
find . -name "*.log" | xargs wc -l # count lines in all logs
ls *.txt | xargs grep "pattern" # search all .txt files
Output:
# echo "a b c" | xargs echo "args:"
args: a b c
# find . -name "*.log" | xargs wc -l
142 ./app.log
58 ./error.log
37 ./access.log
237 total
NUL safety (handles spaces in filenames)
By default xargs splits stdin on whitespace, which breaks filenames containing spaces or tabs. Using find -print0 paired with xargs -0 switches both sides to NUL-terminated records, making the pipeline safe for any filename the filesystem allows.
# Always prefer -0 / -print0 combination
find . -name "*.txt" -print0 | xargs -0 grep "pattern"
fd -0 -e log | xargs -0 rm
printf '%s\0' "file 1.txt" "file 2.txt" | xargs -0 cat
Output: (none — exits 0 on success)
Limit arguments per call
-n N caps how many arguments are passed to each invocation of the command, causing xargs to call it multiple times if there are more inputs. This is essential when a command has a maximum argument count limit, or when you want to process items one at a time with -n 1.
echo "a b c d e" | xargs -n 2 echo # 2 args per call
# → echo a b
# → echo c d
# → echo e
cat urls.txt | xargs -n 1 curl -LO # download one URL at a time
Output:
a b
c d
e
Parallel execution
-P N runs up to N invocations of the command simultaneously, turning a sequential list-processing loop into a parallel one with a single flag. Combine with -n 1 so each process handles exactly one item; use $(nproc) to match your CPU core count automatically.
# -P N — run up to N processes in parallel
find . -name "*.png" -print0 | xargs -0 -P 8 -n 1 convert {} {.}.webp
# Compress files in parallel
find . -name "*.log" -print0 | xargs -0 -P 4 -n 1 gzip
# Run tests in parallel
cat test_list.txt | xargs -P $(nproc) -n 1 ./run_test.sh
Output:
# find . -name "*.log" -print0 | xargs -0 -P 4 -n 1 gzip
gzip: ./app.log: 78.3% -- replaced with ./app.log.gz
gzip: ./error.log: 71.1% -- replaced with ./error.log.gz
gzip: ./access.log: 82.4% -- replaced with ./access.log.gz
gzip: ./debug.log: 69.8% -- replaced with ./debug.log.gz
Placeholder {}
-I PLACEHOLDER substitutes the placeholder string with the input item at any position in the command, and implies -n 1. This lets you place the argument in the middle of the command or reference it multiple times — unlike the default mode where arguments are always appended at the end.
cat files.txt | xargs -I {} cp {} {}.bak # backup each file
cat hosts.txt | xargs -I {} ssh {} "uptime" # ssh to each host
find . -name "*.gz" | xargs -I {} -n 1 gunzip {} # decompress each
# Use a different placeholder name
cat list.txt | xargs -I FILE sh -c 'echo "Processing FILE"; wc -l FILE'
Output: (none — exits 0 on success)
Interactive confirmation
-p prompts you to confirm each command before executing it — useful when running destructive operations like rm on a large batch. -t (trace) prints each command before running it without pausing, which is handy for debugging or auditing what xargs is doing.
ls *.tmp | xargs -p rm # prompt before each call
ls *.bak | xargs -t rm # trace: print command before running
Output:
# ls *.bak | xargs -t rm
rm old.bak session.bak temp.bak
Handle empty input
By default, xargs invokes the command once even when stdin is empty, which can cause unintended side effects (e.g., xargs rm with no input deletes nothing, but a bare rm with no arguments errors). The -r flag (GNU extension) suppresses the invocation entirely when there is no input.
# Without -r, xargs runs CMD with no args if stdin is empty
# With -r, it does nothing if stdin is empty
find . -name "*.bak" | xargs -r rm
Output: (none — exits 0 on success)
Combine with shell
xargs invokes commands directly — it does not go through a shell, so shell syntax like pipes, &&, and variable expansion is unavailable. Wrap the invocation in sh -c '...' to access full shell features inside the per-item command.
# When you need shell features (pipes, ;, &&)
cat files.txt | xargs -I {} sh -c 'wc -l {} && echo "---"'
# Process with a function
process() { echo "Handling: $1"; }
export -f process
cat items.txt | xargs -P4 -n1 bash -c 'process "$@"' _
Output: (none — exits 0 on success)
Practical xargs recipes
# Find and delete old logs
find /var/log -name "*.log.gz" -mtime +90 -print0 | xargs -0 rm -v
# Mass rename: add prefix
ls *.txt | xargs -I {} mv {} prefix_{}
# Check if all listed packages are installed
cat packages.txt | xargs dpkg -l 2>/dev/null | grep "^ii"
# Download a list of URLs
cat urls.txt | xargs -n 1 -P 5 wget -q
# Run linter on all changed files
git diff --name-only HEAD | grep '\.py$' | xargs -r flake8
# Copy matched files preserving directory structure
find src/ -name "*.conf" -print0 \
| xargs -0 -I {} install -D {} "/backup/{}"
# Batch API call (1 item at a time, 4 parallel workers)
cat ids.txt | xargs -P 4 -n 1 -I ID curl -s "https://api.example.com/item/ID"
Output:
# find /var/log -name "*.log.gz" -mtime +90 -print0 | xargs -0 rm -v
removed '/var/log/syslog.2.gz'
removed '/var/log/auth.log.3.gz'
removed '/var/log/kern.log.4.gz'
# cat packages.txt | xargs dpkg -l 2>/dev/null | grep "^ii"
ii curl 7.88.1-10 amd64 command line tool for transferring data
ii wget 1.21.3-1 amd64 retrieves files from the web
ii jq 1.6-2.1 amd64 lightweight command-line JSON processor
Always use
find -print0 | xargs -0(NUL delimiters) rather than newline-delimited output when filenames may contain spaces, tabs, or newlines. This is especially important in scripts running against user-supplied directories.
parallel(GNU Parallel) is a powerfulxargs -Preplacement with progress bars, job logging, and advanced input templating. Install withapt install parallel; same-j Nflag for concurrency.
tr — Deep dive
The headline syntax is tr SET1 SET2, but the precise semantics around set length mismatches, character classes, and complement+delete combinations trip up most newcomers. This section documents the corners that matter when pipelines move from interactive prototyping to scripts that must run unattended.
Set length handling
When SET2 is shorter than SET1, tr pads SET2 by repeating its last character — a behaviour that is convenient for mapping many characters to one (e.g., turn all vowels into *) but a footgun when you intended a one-to-one map. The -t (truncate-set1) flag inverts the behaviour by trimming SET1 to the length of SET2, leaving any extra source characters untouched.
# Without -t: 'd' and 'e' both map to 'Z' (last char of SET2)
echo "abcde" | tr 'abcde' 'xyZ'
# With -t: 'd' and 'e' pass through unchanged
echo "abcde" | tr -t 'abcde' 'xyZ'
Output:
xyZZZ
xyZde
Ranges, octal, and hex escapes
tr understands a-z style ranges (locale-sensitive in older versions; use LC_ALL=C for byte-stable behaviour), and octal escapes for non-printable bytes. Common escapes: \t (tab), \n (newline), \r (carriage return), \0 (NUL), \\ (literal backslash), and \nnn (three-digit octal).
# Lowercase letters reversed by range
echo "abcxyz" | LC_ALL=C tr 'a-z' 'z-a'
# Replace NUL bytes with newlines (handy after find -print0)
printf 'a\0b\0c\0' | tr '\0' '\n'
# Replace tabs with two spaces (note: tr cannot expand 1-to-N, use sed/expand for that)
printf 'a\tb\tc\n' | tr '\t' ' '
# Use octal escape for the same NUL
printf 'a\0b\0c\0' | tr '\000' '\n'
Output:
zyxcba
a
b
c
a b c
a
b
c
Character class reference
The POSIX classes below behave identically across GNU coreutils and BusyBox tr. Classes are written inside the set with their [:name:] form — the surrounding […] is part of the class syntax, not a regex bracket expression.
| Class | Matches |
|---|---|
[:alpha:] | Letters (locale-dependent) |
[:lower:] | Lowercase letters |
[:upper:] | Uppercase letters |
[:digit:] | 0–9 |
[:xdigit:] | Hex digits (0–9, a–f, A–F) |
[:alnum:] | Letters + digits |
[:space:] | Any whitespace (space, tab, CR, LF, VT, FF) |
[:blank:] | Only space and tab |
[:punct:] | Printable punctuation |
[:cntrl:] | Control characters (\0 – \037, \177) |
[:print:] | Any printable (alnum + punct + space) |
[:graph:] | Any printable except space |
# Canonical case conversion (locale-aware, beats range-based 'a-z' 'A-Z')
echo "Café Naïve" | tr '[:lower:]' '[:upper:]'
# Strip every control character (preserve newline by using -dc carefully)
printf 'visible\x07\x1bhidden\n' | tr -d '[:cntrl:]'
# Replace any whitespace run with a single space
printf 'a b\tc\nd' | tr -s '[:space:]' ' '
Output:
CAFÉ NAÏVE
visiblehidden
a b c d
Complement combined with delete
-c (complement) inverts SET1 so the operation targets everything not in the set. Pairing it with -d is the standard idiom for "keep only these characters" — extremely common when sanitising user input, building safe identifiers, or extracting a token from noisy output.
# Keep only ASCII printable bytes and newline
tr -cd '[:print:]\n' < binary.dat > sanitised.txt
# Keep only digits (handy for parsing IDs out of noisy text)
echo "Order #1234-AB-9876 placed" | tr -cd '0-9\n'
# Keep digits, letters, dashes, and dots — a URL-safe slug seed
echo "Hello, World! v2.0" | tr -cd '[:alnum:]-.\n'
# Reverse: drop letters, keep everything else
echo "Hello123" | tr -d '[:alpha:]'
Output:
12349876
HelloWorldv2.0
123
Squeeze with complement
Combining -s and -c says "squeeze runs of characters NOT in this set" — the classic recipe for tokenising arbitrary text into one word per line by treating everything non-alphabetic as a separator collapsed into a single newline.
# Tokenise: every non-letter run becomes one newline
echo "Hello, world! Goodbye---world." | tr -sc '[:alpha:]' '\n'
# Collapse repeated punctuation runs into a single character
echo "...wait!!! Why???" | tr -s '[:punct:]'
Output:
Hello
world
Goodbye
world
.wait! Why?
Translation with --truncate-set1
Same idea as -t but spelled out — useful when scripting against GNU tr where the long form aids readability. Reach for it whenever you want a strict one-to-one map and refuse to let tr pad SET2 silently.
# Strict 1-to-1: 'd' and 'e' pass through
echo "abcde" | tr --truncate-set1 'abcde' 'xyZ'
Output:
xyZde
Edge cases & pitfalls
tr works on bytes, not characters — multi-byte UTF-8 sequences are processed one byte at a time. This means tr 'é' 'E' in a UTF-8 locale will mangle output because é is two bytes (\xc3\xa9); for full Unicode case-mapping use sed, awk, or python -c. The other classic surprise: tr reads from stdin only — pass files via < redirection, never as positional arguments.
# WRONG — tr ignores the filename argument
tr 'a-z' 'A-Z' file.txt
# RIGHT — redirect file into stdin
tr 'a-z' 'A-Z' < file.txt
# RIGHT — pipe in
cat file.txt | tr 'a-z' 'A-Z'
# Multi-byte hazard: tr operates byte-by-byte
echo "café" | tr 'éè' 'ee' # corrupted output, two bytes of é mismatched
# Safer Unicode case-fold:
echo "café" | python3 -c 'import sys; print(sys.stdin.read().lower())'
Output:
HELLO WORLD
hello world
caf
café
xargs — Deep dive
The headline xargs CMD invocation hides a dozen knobs that turn shell loops into precise, parallel, NUL-safe pipelines. This section walks each knob — argument batching, replacement strings, delimiters, environment, and exit handling — with the gotchas that prevent silent data corruption.
Argument batching with -n and -L
-n N caps how many arguments go into each command invocation; -L N caps how many input lines feed each invocation (a line can contain multiple arguments). The two interact subtly: pair -n with whitespace input, -L when each input line is a logical unit (e.g., a record from a generated list).
# 2 args per call → 3 calls
echo a b c d e | xargs -n 2 echo
# Each line becomes one call regardless of arg count
printf 'a b\nc d e\nf\n' | xargs -L 1 echo
# Combine with limit on total command length
seq 1 1000 | xargs --max-chars=200 echo | wc -l
Output:
a b
c d
e
a b
c d e
f
44
Replacement strings with -I
-I REPLACE substitutes the literal REPLACE with the current input item anywhere in the command — even inside flags or quoted strings — and implies -L 1 (one item per call). This is the only way to put the argument in the middle of a command line instead of at the end, and to use the argument more than once per invocation.
# Place argument mid-command, also use it twice
cat files.txt | xargs -I FILE cp FILE FILE.bak
# Use {} as the placeholder (read like find -exec)
find . -name "*.png" | xargs -I {} convert {} -resize 50% small_{}
# Place argument inside a quoted string
cat hosts.txt | xargs -I HOST ssh HOST "echo Hello from \$(hostname)"
# Multiple references to the same arg
cat dirs.txt | xargs -I DIR sh -c 'mkdir -p DIR/in DIR/out DIR/err'
Output:
# After the cp run, every .bak twin exists alongside the original
report-2026.pdf report-2026.pdf.bak
notes.txt notes.txt.bak
budget.xlsx budget.xlsx.bak
-Iimplies-L 1, which silently disables parallel batching: passing-P 8will still spawn parallel processes but each handles exactly one item. That is usually what you want — just be aware the speedup comes from parallelism, not batching.
Parallel execution with -P
-P N runs up to N invocations of the command at the same time. Set N=0 to let xargs run as many parallel jobs as possible (limited only by system resources), or N=$(nproc) to match the CPU core count. Output from parallel jobs interleaves — pipe each command's output to its own file if order matters.
# Resize 200 images using all CPU cores
find . -maxdepth 1 -name "*.jpg" -print0 \
| xargs -0 -P "$(nproc)" -n 1 -I {} convert {} -resize 800x800 thumbs/{}
# Parallel HTTP HEAD checks (rate-limit with -P)
cat urls.txt | xargs -P 10 -n 1 curl -o /dev/null -s -w "%{http_code} %{url}\n"
# 0 = unlimited parallelism (use with care)
seq 1 100 | xargs -P 0 -n 1 -I N sh -c 'sleep 0.1; echo N done'
Output:
200 https://example.org/
404 https://example.org/missing
200 https://example.org/about
301 https://example.org/blog
500 https://example.org/api
NUL-delimited input with -0
By default xargs splits stdin on whitespace, which breaks on filenames with spaces, tabs, or newlines and creates a critical injection hazard with attacker-controlled filenames. Always pair -0 with find -print0, grep -z, fd -0, or printf '%s\0' for safe, lossless processing of any filename.
# Safe: NUL-delimited end-to-end
find . -name "*.log" -print0 | xargs -0 wc -l
# Pipe from grep with -z for NUL-delimited matches
grep -lZ 'TODO' src/ | xargs -0 sed -i 's/TODO/DONE/'
# Manually NUL-delimit a list
printf '%s\0' "file with spaces.txt" "another file.txt" | xargs -0 cat
# fd uses -0 the same way
fd -0 -e log . /var/log | xargs -0 -P 4 -n 1 gzip
Output:
142 ./app.log
58 ./error.log
37 ./access.log
237 total
Custom delimiters with -d
-d DELIM lets you split stdin on an arbitrary single character — useful when input is comma-separated or otherwise pre-formatted with a known delimiter you cannot easily convert to NUL.
# Process a comma-separated list
echo "alice,bob,carol,dave" | xargs -d ',' -n 1 echo "Hello,"
# Newline-only mode (default, but explicit can help readability)
cat lines.txt | xargs -d '\n' -I LINE echo "Saw: LINE"
# Pipe-delimited
echo "one|two|three" | xargs -d '|' -n 1 echo
Output:
Hello, alice
Hello, bob
Hello, carol
Hello, dave
one
two
three
Empty-input safety with -r
-r (--no-run-if-empty, GNU extension) prevents xargs from invoking the command at all when stdin is empty — without it, find ... | xargs rm runs rm with no arguments and prints a usage error, while find ... | xargs echo foo prints foo. Always include -r in scripts to avoid surprises.
# Safe even when no matches are found
find . -name "*.lock" -mtime +30 | xargs -r rm
# Without -r: still runs the command, producing an error message
: | xargs rm # → rm: missing operand
: | xargs -r rm # → silent, no invocation
Output:
# (no output — clean exit when there is nothing to remove)
Inspecting commands with -t and -p
-t traces every command to stderr before running it — useful for debugging or auditing what xargs does. -p is interactive: it prints each command and waits for a y/n confirmation before executing.
# Trace mode — see each invocation as it happens
ls *.tmp | xargs -t rm
# Prompt mode — confirm before each (dangerous batch deletes)
ls *.bak | xargs -p rm
# Combine -t with -P for verbose parallel output
seq 1 5 | xargs -t -P 3 -n 1 sleep
Output:
rm cache1.tmp cache2.tmp scratch.tmp
sleep 1
sleep 2
sleep 3
sleep 4
sleep 5
Argument length and --max-chars
xargs always respects the OS argument-length limit (getconf ARG_MAX, typically 128 KB to 2 MB), splitting input into multiple invocations when needed. --max-chars=N lets you lower that ceiling — handy when the downstream command has its own narrower limit, or to keep individual commands diff-friendly.
# Show the system limit
xargs --show-limits </dev/null
# Force smaller batches (each command line ≤ 500 chars)
seq 1 10000 | xargs --max-chars=500 echo | wc -l
Output:
Maximum length of command we could actually use: 2086008
Size of command buffer we are actually using: 131072
44
Exit codes
xargs's exit code reflects the worst-case outcome across all child invocations: 0 (all succeeded), 123 (one or more commands exited 1–125), 124 (a child was killed), 125 (the command itself could not be invoked), 126 (command found but not executable), 127 (command not found), 1 (other fatal xargs error). Check $? after the pipe to detect partial failures.
# Make xargs stop at the first failure with --halt
seq 1 10 | xargs -I N sh -c 'test N -lt 5 && echo ok N || exit 1'
echo "xargs exit: $?"
# GNU xargs --halt is part of GNU Parallel, not coreutils — use && / set -e instead
Output:
ok 1
ok 2
ok 3
ok 4
xargs exit: 123
Combining tr and xargs
tr and xargs complement each other on the boundary between unstructured input and command execution: tr normalises whitespace, deletes control characters, or splits a stream into one-record-per-line; xargs then turns those records into commands. The standard idiom is <source> | tr <normalise> | xargs <cmd> — but always prefer NUL-safe alternatives when filenames could contain spaces.
# Convert a comma-separated env file into shell arguments
echo "alice,bob,carol" | tr ',' '\n' | xargs -I USER useradd USER
# Normalise multi-space output into single-space, then feed as args
ps -eo comm | tr -s ' ' | sort -u | xargs -n 5 echo
# Strip CR from a Windows-generated list, then process line by line
tr -d '\r' < hosts.windows.txt | xargs -I HOST ssh HOST 'uptime'
# Build shell-safe identifiers from messy headings
echo "My Page Title!" \
| tr '[:upper:]' '[:lower:]' \
| tr -cs '[:alnum:]' '-' \
| xargs -I SLUG mkdir -p "content/SLUG"
Output:
# After the useradd loop
useradd alice
useradd bob
useradd carol
# After the ps + xargs run
bash chronyd cron dbus-daemon gvfsd
init kworker login redis-server snapd
sshd systemd systemd-udevd udevd
Alternatives and when to switch
xargs is small, ubiquitous, and POSIX, but more complex parallel pipelines often benefit from purpose-built tools. The table below maps common needs to the best tool — xargs remains the default for simple cases, but consider switching when you cross into territory it cannot handle natively.
| Need | Best tool | Why |
|---|---|---|
| Simple parallel fan-out | xargs -P | Built-in, no install, NUL-safe with -0 |
| Progress bar, job log, retries | GNU parallel | Rich features, --joblog, --retries |
| Conditional logic per item | shell for loop | Full shell syntax inside the loop body |
| Streaming N producers → M consumers | GNU parallel --pipe | Splits stdin into chunks |
| Distributed across hosts | GNU parallel --sshlogin | xargs cannot do remote |
| Async I/O (HTTP, DNS) | curl --parallel, dig +batch | Network tools have own concurrency |
# xargs version
cat urls.txt | xargs -P 8 -n 1 curl -O
# GNU parallel equivalent with progress bar and a job log
parallel --bar --joblog jobs.log --jobs 8 curl -O ::: $(cat urls.txt)
# Shell for-loop when each item needs conditional logic
for url in $(cat urls.txt); do
case "$url" in
*.pdf) curl -o "pdfs/$(basename $url)" "$url" ;;
*) curl -O "$url" ;;
esac
done
Output:
# parallel adds a progress bar on stderr
ETA: 12s Left: 4 AVG: 1.10s/it local:0/8/0%/1.0s
ETA: 9s Left: 3 AVG: 1.05s/it local:0/8/0%/1.0s
…
Quick reference
A condensed lookup of the most common flag combinations covered above. Print or pin this when wiring up a pipeline at speed.
| Goal | Command |
|---|---|
| Lowercase | tr '[:upper:]' '[:lower:]' |
| Uppercase | tr '[:lower:]' '[:upper:]' |
| Strip CR | tr -d '\r' |
| Keep only digits | tr -cd '0-9\n' |
| Collapse whitespace | tr -s '[:space:]' ' ' |
| Tokens onto lines | tr -sc '[:alpha:]' '\n' |
| Make shell-safe slug | tr '[:upper:]' '[:lower:]' | tr -cs '[:alnum:]' '-' |
NUL-safe find → command | find … -print0 | xargs -0 CMD |
| One item per call | xargs -n 1 CMD |
| Parallel, all cores | xargs -P "$(nproc)" -n 1 CMD |
| Argument in middle | xargs -I {} CMD before {} after |
| Don't run on empty input | xargs -r CMD |
| Trace each command | xargs -t CMD |
| Custom delimiter | xargs -d ',' CMD |
| Shell features inside | xargs -I {} sh -c '…{}…' |
When in doubt about whether your pipeline is safe, prefix every
xargsinvocation with-tand run on a small sample. The trace reveals exactly how arguments are batched and quoted — a five-second check that prevents most accidentalrmdisasters.