cheat sheet

systemd Unit Files

Writing, enabling, and managing systemd service, timer, and socket units.

systemd Unit Files

What it is

systemd is the init system and service manager used by default on Debian, Ubuntu, Fedora, Arch, and most other major Linux distributions, developed primarily by Red Hat and the systemd project. It manages system boot, background services (daemons), scheduled tasks (timers), and socket activation through declarative unit files written in an INI-like format. Reach for systemd unit files whenever you need to run a process reliably at boot, restart it automatically on failure, or replace a cron job with a more controllable and observable scheduled task.

Configuration

systemd reads configuration from many files spread across the filesystem. Knowing which file wins matters — the same setting in two places can quietly disagree. Unit files (*.service, *.timer, *.socket, …) live under unit-search directories and are resolved by name with strict priority; manager-level behaviour (default timeouts, log targets, kill modes) is set in system.conf/user.conf plus any drop-ins under .conf.d/. Always prefer drop-ins (*.d/override.conf) over editing the upstream file directly so package upgrades keep flowing through.

PathPurpose
/etc/systemd/system/*.{service,timer,socket,mount,path,target}local admin unit files — highest priority
/etc/systemd/system/<unit>.d/*.conflocal admin drop-ins layered on top of the base unit
/run/systemd/system/runtime-generated units (gone on reboot)
/usr/lib/systemd/system/ (or /lib/systemd/system/)distro-shipped units — never edit; override instead
~/.config/systemd/user/per-user units — systemctl --user
/etc/systemd/user/system-wide user-unit overrides
/usr/lib/systemd/user/distro-shipped user units
/etc/systemd/system.conf (+ system.conf.d/*.conf)manager defaults: DefaultTimeoutStartSec=, DefaultLimitNOFILE=, LogTarget=, CtrlAltDelBurstAction=, …
/etc/systemd/user.conf (+ user.conf.d/*.conf)same for the per-user manager
/etc/systemd/journald.conf (+ journald.conf.d/*.conf)journal storage, retention, forwarding — used by journalctl
/etc/systemd/logind.conf (+ logind.conf.d/*.conf)login/seat policy: HandleLidSwitch=, KillUserProcesses=, IdleAction=
/etc/systemd/resolved.confsystemd-resolved DNS settings
/etc/systemd/networkd.conf and /etc/systemd/network/*.{network,netdev,link}systemd-networkd config
/etc/systemd/timesyncd.confNTP servers for systemd-timesyncd
/etc/systemd/coredump.confcoredump capture policy for systemd-coredump

Drop-in precedence (later wins for the same key):

  1. Base unit in /usr/lib/systemd/system/<unit> — distro shipped.
  2. Distro drop-ins in /usr/lib/systemd/system/<unit>.d/*.conf (alphabetical).
  3. Runtime drop-ins in /run/systemd/system/<unit>.d/*.conf.
  4. Local drop-ins in /etc/systemd/system/<unit>.d/*.confwhat systemctl edit writes.
  5. A full replacement file at /etc/systemd/system/<unit> (rare — loses upstream updates).

Within one directory, drop-ins are merged in alphabetical filename order, so 50-cpu.conf overrides 10-cpu.conf for the same key. To clear an inherited list-valued directive (e.g. Environment=, ExecStart=, After=), redeclare it empty (Environment=) before adding new values — otherwise the new entry appends to the old list.

bash
# Resolved file path + every drop-in that contributed
systemctl cat myapp.service

# All effective properties after merging
systemctl show myapp.service | grep -E '^(FragmentPath|DropInPaths|UnitFileState)='

# Manager-level defaults currently in effect
systemctl show --property=DefaultTimeoutStartSec,DefaultLimitNOFILE,LogLevel

Output (systemctl show -p FragmentPath,DropInPaths myapp.service):

text
FragmentPath=/etc/systemd/system/myapp.service
DropInPaths=/etc/systemd/system/myapp.service.d/override.conf /etc/systemd/system/myapp.service.d/50-limits.conf

After editing any of these files (or any drop-in), run sudo systemctl daemon-reload so systemd re-reads them. systemctl edit does the reload for you; manual edits don't.

Service unit skeleton

A service unit has three sections: [Unit] for metadata and ordering dependencies, [Service] for the process configuration (what to run, as whom, and what to do on failure), and [Install] for the symlinks that determine which boot target activates the service. The most important [Service] fields are ExecStart, User, Restart, and Type.

ini
# /etc/systemd/system/myapp.service
[Unit]
Description=My Application
After=network.target
Wants=network.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/bin/myapp --config /etc/myapp/config.toml
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s
StandardOutput=journal
StandardError=journal
Environment="NODE_ENV=production"
EnvironmentFile=-/etc/myapp/env   # dash = ignore if missing

[Install]
WantedBy=multi-user.target

Timer unit (cron replacement)

A timer unit is paired with a same-named service unit and activates it on a schedule defined by OnCalendar (wall-clock time, cron-like) or OnBootSec/OnUnitActiveSec (relative intervals). Setting Persistent=true ensures missed runs (e.g., while the machine was off) execute on next boot — something traditional cron cannot do.

ini
# /etc/systemd/system/backup.timer
[Unit]
Description=Daily backup

[Timer]
OnCalendar=*-*-* 02:30:00
Persistent=true   # run missed timers on next boot

[Install]
WantedBy=timers.target
ini
# /etc/systemd/system/backup.service  (paired service)
[Unit]
Description=Run backup

[Service]
Type=oneshot
ExecStart=/usr/local/bin/backup.sh

Essential commands

After editing a unit file you must run systemctl daemon-reload before systemd sees the changes. enable --now combines enabling at boot with immediately starting the service; status gives a quick health summary plus the last few journal lines; journalctl -u SERVICE -f streams live logs.

bash
# Reload unit files after editing
systemctl daemon-reload

# Enable + start
systemctl enable --now myapp.service

# Status / logs
systemctl status myapp.service
journalctl -u myapp.service -f          # follow
journalctl -u myapp.service --since "1 hour ago"
journalctl -u myapp.service -n 50 --no-pager

# Restart / reload
systemctl restart myapp.service
systemctl reload myapp.service          # sends SIGHUP (if ExecReload set)

# List timers
systemctl list-timers --all

# Show unit dependencies
systemctl list-dependencies myapp.service

# Check why a unit failed
systemctl status myapp.service
journalctl -xe

Output: (none — exits 0 on success)

Service types

TypeUse case
simpleProcess is the main process; starts immediately
execLike simple, but waits until execve() succeeds before considering the unit started
forkingProcess forks; PID file required
oneshotRuns once, exits; systemd waits
notifyProcess calls sd_notify() when ready
notify-reloadLike notify but with explicit reload protocol (systemd 253+)
dbusReady when D-Bus name acquired
idleLike simple but waits for other jobs to finish

Pick simple for a foreground process that doesn't fork (most modern daemons, including anything started with npm, gunicorn, or python); pick forking only when you genuinely cannot run the daemon in the foreground; pick oneshot for scripts that complete and exit; pick notify when the process can use sd_notify(READY=1) to signal real readiness (PostgreSQL, systemd-journald, NetworkManager). The wrong type means systemd thinks the service is running before it actually is — or, worse, considers it crashed when it forked normally.

Unit file anatomy

A unit file is INI-formatted with one section per concern: [Unit] for metadata and ordering, [Service] (or [Timer], [Socket], [Path], [Mount]) for the unit-type-specific configuration, and [Install] for boot-time wiring. systemd searches unit paths in a strict priority order; the first match wins.

PathPurpose
/etc/systemd/system/local admin overrides — highest priority for system services
/run/systemd/system/runtime-generated units (lost on reboot)
/usr/lib/systemd/system/distro-shipped units — do not edit; override instead
~/.config/systemd/user/per-user services — systemctl --user
/etc/systemd/user/system-wide user-service overrides
/usr/lib/systemd/user/distro-shipped user units
bash
# Show the resolved path of any unit
systemctl cat myapp.service
systemctl show -p FragmentPath myapp.service

Output (systemctl show -p FragmentPath):

text
FragmentPath=/etc/systemd/system/myapp.service

[Unit] directives

The [Unit] block is where you express ordering (Before=, After=) and dependencies (Requires=, Wants=, Requisite=, BindsTo=). Ordering and dependency are independent: After=foo.service only controls the start order; Wants=foo.service only declares that foo should be pulled in. You almost always want both together.

DirectiveMeaning
Description=human-readable string shown by systemctl status
Documentation=URL or man: reference
After= / Before=ordering only (no dependency)
Wants=soft dependency — pull in, but don't fail if absent
Requires=hard dependency — fail to start if the other unit fails
Requisite=start only if the other unit is already active (no auto-start)
BindsTo=stronger Requires — stop this unit if the other stops
PartOf=restart/stop with the named unit (one direction)
Conflicts=mutually exclusive with the named unit
ConditionPathExists=skip activation unless the path exists
OnFailure=start the named unit if this one fails (great for alerting)
ini
[Unit]
Description=My Application
Documentation=https://example.com/docs/myapp
After=network-online.target postgresql.service
Wants=network-online.target
Requires=postgresql.service
OnFailure=myapp-alert.service
ConditionPathExists=/etc/myapp/config.toml

Use network-online.target (not network.target) when your service genuinely needs a routable network — for example, anything that contacts an external API at startup. network.target only means the network stack is loaded.

[Service] directives

The [Service] block answers what to run, as whom, where, and what to do when it fails. The Exec*= family runs commands at different lifecycle points; Restart= and RestartSec= define the auto-recovery policy; User=/Group= drop privileges; Environment= and EnvironmentFile= inject configuration.

DirectiveEffect
ExecStartPre=runs before ExecStart= (prefix - to ignore failure)
ExecStart=the main command — exactly one for Type=simple
ExecStartPost=runs after ExecStart= succeeds
ExecReload=command to run on systemctl reload (often kill -HUP $MAINPID)
ExecStop=clean shutdown command (otherwise SIGTERM is sent)
ExecStopPost=cleanup after stop
Restart=no, on-success, on-failure, on-abnormal, on-watchdog, on-abort, always
RestartSec=delay between restarts (default 100ms)
StartLimitIntervalSec= / StartLimitBurst=rate-limit restart loops
TimeoutStartSec=how long to wait for start before declaring failure
TimeoutStopSec=how long to wait for stop before SIGKILL
WorkingDirectory=cwd for Exec* commands
User= / Group=drop to this UID/GID
Environment=set variables (KEY=value, quoted)
EnvironmentFile=read variables from a file (- prefix = ignore-if-missing)
StandardOutput= / StandardError=journal (default), null, tty, file:/path, append:/path
KillMode=control-group (default), mixed, process, none
KillSignal=the signal sent on stop (default SIGTERM)
PIDFile=required for Type=forking
ini
[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=-/etc/myapp/env
Environment="NODE_ENV=production" "PORT=8080"
ExecStartPre=/usr/bin/mkdir -p /run/myapp
ExecStart=/opt/myapp/bin/myapp --config /etc/myapp/config.toml
ExecReload=/bin/kill -HUP $MAINPID
ExecStop=/bin/kill -TERM $MAINPID
Restart=on-failure
RestartSec=5s
StartLimitIntervalSec=60
StartLimitBurst=3
TimeoutStartSec=30s
StandardOutput=journal
StandardError=journal

[!WARN] Restart=always plus a fast-crashing process is a fork bomb in slow motion. Pair it with StartLimitIntervalSec + StartLimitBurst so systemd gives up after N failed restarts in a window and surfaces the failure rather than masking it.

[Install] directives

The [Install] block tells systemctl enable how to wire the unit into the boot order. The most common directive is WantedBy=multi-user.target — that creates a symlink under multi-user.target.wants/ so the unit starts at the standard multi-user boot stage. [Install] has no effect at runtime; it only governs what enable/disable does.

DirectiveEffect
WantedBy=soft "start me when this target activates"
RequiredBy=hard "fail this target if I fail"
Also=enable/disable additional units alongside this one
Alias=alternate symlink name
DefaultInstance=default instance name for template units (foo@.service)
ini
[Install]
WantedBy=multi-user.target
Alias=myapp.service

Drop-ins — override without forking the unit file

When a distro ships a service unit and you only want to tweak a directive or two, do not copy the file to /etc/systemd/system/ — write a drop-in instead. Drop-ins live in <unit>.d/ directories and contain only the keys you want to change. They're merged on top of the upstream unit, so package updates still flow through.

bash
# Open an editor on a drop-in for an existing unit
sudo systemctl edit nginx.service          # creates /etc/systemd/system/nginx.service.d/override.conf

# Edit the FULL unit (rarely what you want — loses upstream updates)
sudo systemctl edit --full nginx.service

# Show the merged result of base unit + all drop-ins
systemctl cat nginx.service

Output (systemctl cat nginx.service):

text
# /lib/systemd/system/nginx.service
[Unit]
Description=A high performance web server and a reverse proxy server
…
# /etc/systemd/system/nginx.service.d/override.conf
[Service]
LimitNOFILE=65536
Restart=always

systemctl edit writes to /etc/systemd/system/<unit>.d/override.conf and runs daemon-reload for you. To clear a directive that the base unit sets, redeclare it as empty: Environment= (with no value) wipes any inherited Environment=.

Sandboxing and hardening

Modern systemd has dozens of directives that lock down a service's view of the kernel and filesystem. Layering these on a daemon is one of the cheapest security wins available — even if the process is compromised, the blast radius is contained. Start with ProtectSystem=strict + PrivateTmp=yes + NoNewPrivileges=yes and add more as you learn the daemon's actual needs.

DirectiveEffect
NoNewPrivileges=yesprevent the process from gaining new privileges (sets PR_SET_NO_NEW_PRIVS)
ProtectSystem=strictmount /usr, /boot, /etc read-only (or full/yes/no)
ProtectHome=yeshide /home, /root, /run/user
PrivateTmp=yesgive the unit a private /tmp and /var/tmp
PrivateDevices=yesonly /dev/null, /dev/zero, /dev/random are visible
PrivateNetwork=yesno network namespace access (only loopback)
PrivateUsers=yesrun inside a user namespace
ProtectKernelTunables=yes/proc/sys, /sys read-only
ProtectKernelModules=yesblock module loading
ProtectControlGroups=yes/sys/fs/cgroup read-only
RestrictNamespaces=yesblock namespace creation
RestrictRealtime=yesblock real-time scheduling
RestrictSUIDSGID=yesblock creating setuid/setgid files
LockPersonality=yesfreeze the personality(2) syscall
MemoryDenyWriteExecute=yesblock writable+executable mappings
SystemCallFilter=@system-serviceallow only the listed syscall sets
ReadWritePaths=grant write access to specific paths despite ProtectSystem
CapabilityBoundingSet=drop capabilities (e.g. CapabilityBoundingSet= clears them all)
AmbientCapabilities=grant specific capabilities (avoids needing root)
ini
[Service]
Type=simple
User=myapp
Group=myapp
ExecStart=/opt/myapp/bin/myapp

# Sandboxing
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictNamespaces=yes
RestrictRealtime=yes
LockPersonality=yes
MemoryDenyWriteExecute=yes
SystemCallFilter=@system-service
SystemCallErrorNumber=EPERM

# This service writes here, so re-grant access
ReadWritePaths=/var/lib/myapp /var/log/myapp

# Drop all capabilities (it's a plain TCP server)
CapabilityBoundingSet=

systemd-analyze security myapp.service scores a unit's hardening on a 0–10 scale and lists every directive you haven't set. Treat anything above 5 as "needs work"; well-hardened services score under 2.

BPF-based filtering (systemd 250+)

Two sandboxing directives are implemented as eBPF programs attached to LSM hooks rather than seccomp filters, which lets them match on richer kernel data than syscall arguments alone. RestrictFileSystems= allows or denies access by filesystem type (a chmod succeeds on ext4 but is EACCES on tmpfs, say), and RestrictNetworkInterfaces= limits which network interfaces the unit can bind/send/receive on by interface name. Both require a kernel with BPF LSM (CONFIG_BPF_LSM=y, typically 5.7+) enabled at boot via lsm=...,bpf.

ini
[Service]
# Allow only ext4 and tmpfs; everything else returns EACCES at open() time
RestrictFileSystems=ext4 tmpfs

# Or invert: deny these, allow the rest
RestrictFileSystems=~proc sysfs

# Only allow traffic on a specific interface (and loopback)
RestrictNetworkInterfaces=lo eth0

What's new — systemd 256, 257, 258

Recent releases added a wave of new unit-file directives covering namespace delegation, BPF isolation, per-unit disk quotas, smarter timer behaviour, and tighter socket-activation control. The highlights below are the ones most likely to land in production unit files. Check systemd --version on the target host before relying on them — kernel and distro vintage both matter.

Sandboxing additions

DirectiveSinceWhat it does
ProtectHostname=private258Like yes, but the unit can still change the hostname inside its own UTS namespace — host hostname is unaffected. Pair with ProtectHostname=private:<name> to also set a custom hostname for the unit.
PrivateBPF=yes258Mount a private bpffs for the unit so it can't see other services' BPF programs/maps.
PrivateUsers=full258Map the entire 32-bit UID range into a private user namespace (older yes only mapped 64K).
DelegateNamespaces=258Hand specific namespace types (uts, ipc, pid, net, …) to the unit's processes for further nesting.
RestrictFileSystems=250BPF LSM filter on filesystem types (see above).
RestrictNetworkInterfaces=250 (256 added alt-name support)BPF LSM filter on interface names.
ini
[Service]
ProtectHostname=private:worker-01
PrivateBPF=yes
PrivateUsers=full
DelegateNamespaces=uts ipc

Per-unit disk quotas (systemd 258)

StateDirectoryQuota=, CacheDirectoryQuota=, and LogsDirectoryQuota= cap how much space the unit can consume in /var/lib/<unit>, /var/cache/<unit>, and /var/log/<unit> respectively. They're enforced via project quotas on xfs/ext4, so the underlying filesystem must be mounted with prjquota. systemctl status shows current usage when a quota is set.

ini
[Service]
StateDirectory=myapp
CacheDirectory=myapp
LogsDirectory=myapp
StateDirectoryQuota=2G
CacheDirectoryQuota=500M
LogsDirectoryQuota=200M

# Also new: :ro suffix grants read-only access to a directory another unit owns
StateDirectory=shared-config:ro

Timer changes

DirectiveSinceEffect
DeferReactivation=yes257If the paired service is still running when the timer next elapses, skip the firing instead of queuing another invocation. Kills the "every minute timer ends up running 5 copies of a slow job" foot-gun.
RandomizedOffsetSec=258Like RandomizedDelaySec= but the offset is stable for a given host (derived from machine-id), so the same host always fires at the same offset within the window.
ini
[Timer]
OnUnitActiveSec=5min
DeferReactivation=yes
RandomizedOffsetSec=2min

Socket-activation changes (systemd 258)

DirectiveEffect
PassPIDFD=yesSet SO_PASSPIDFD so the unit receives PID file descriptors of peers over AF_UNIX.
PassFileDescriptorsToExec=yesPass socket FDs to ExecStartPost=/ExecStopPre=/ExecStopPost= via $LISTEN_FDS.
DeferTrigger= / DeferTriggerMaxSec=Delay activating the paired service after the first packet — useful for batching.
MaxConnectionsPerSource= (256)Per-source-UID cap for AF_UNIX sockets in Accept=yes mode.

Service-manager additions

  • ExecStart=|/usr/bin/foo "$BAR" — leading | makes systemd 258 invoke the command via a shell, so pipes, redirections, and quoting work without writing /bin/sh -c '...'.
  • WantsMountsFor= (256) — like RequiresMountsFor= but creates a soft Wants= rather than a hard Requires=, so the unit still starts if the mount is unavailable.
  • MemoryZSwapWriteback= (256) — wires the kernel 6.8 memory.zswap.writeback cgroup knob into a service unit.
  • ConcurrencySoftMax= / ConcurrencyHardMax= on slice units (258) — cap how many units in a slice can run concurrently.
  • %D specifier (256) — expands to $XDG_DATA_HOME for user services, /usr/share/ for system services.
  • New "lenient" --job-mode= (258) — refuse a transaction that would stop already-running units.

sysext / confext — runtime overlays for /usr and /etc

System extensions (sysext) and configuration extensions (confext) let you layer additional content onto a running system without modifying the base image. sysext images get OverlayFS-merged into /usr and /opt; confext images merge into /etc. Both are designed for immutable base images (OSTree, image-based distros, embedded) where you still need to ship optional packages or per-fleet config without rebuilding.

bash
# Drop a signed sysext image into the well-known location and merge
sudo cp myext.raw /var/lib/extensions/
sudo systemd-sysext refresh
sudo systemd-sysext status

# Same idea for /etc overlays
sudo cp myconf.raw /var/lib/confexts/
sudo systemd-confext refresh

Output (systemd-sysext status):

text
HIERARCHY EXTENSIONS    SINCE
/opt      none          -
/usr      myext         Sun 2026-05-25 09:14:02 EDT

Units can attach their own extensions for the duration of the run via ExtensionImages= and ExtensionDirectories=:

ini
[Service]
ExtensionImages=/var/lib/extensions/debug-tools.raw
ExtensionDirectories=/var/lib/myapp/plugins

systemd 256+ supports the vpick protocol (foo.v/) so RootImage=, ExtensionImages=, etc. can point at a directory of versioned images and automatically pick the newest one — useful for blue/green updates of extensions.

mkosi + importctl + varlinkctl

These are companion tools rather than unit-file directives, but they're how modern systemd shops build and ship the images that feed RootImage=, sysext, and systemd-nspawn:

  • mkosi — declarative image builder maintained alongside systemd; produces bootable disk images, sysext images, container images, and UKIs from a mkosi.conf + package lists. Used to assemble both the base OS and any sysext/confext overlays.
  • importctl (256+) — generalises what machinectl pull-* used to do. Downloads, imports, and exports disk images, sysext, confext, and portable services; 258 adds zstd support and a --blockdev flag that attaches the downloaded image to a loop device.
  • varlinkctl — introspects and calls Varlink IPC methods exposed by systemd components (resolved, hostnamed, the service manager itself starting in 258). The 258 release added --exec (pipe the JSON reply to a command) and --push-fd= (send FDs across the call), and varlinkctl call ssh:user@host/... (256+) tunnels Varlink over SSH.
bash
# List the Varlink methods exposed by systemd-resolved
varlinkctl introspect /run/systemd/resolve/io.systemd.Resolve

# Pull a sysext image with importctl
sudo importctl pull-raw --class=sysext https://example.com/debug.raw debug

# Build a sysext with mkosi
mkosi --format=sysext --output=debug.raw build

Output: (none — exits 0 on success)

Deprecations to know

systemd 258 finalised several long-running removals — relevant when porting older units forward:

  • cgroup v1 is gone. Both the "legacy" and "hybrid" hierarchies are removed; the host kernel must be booted with systemd.unified_cgroup_hierarchy=1 (the default for years). All cpu.cfs_*, memory.limit_in_bytes, etc. v1-only directives no longer apply.
  • Minimum kernel 5.4 (recommended 5.7+ for the BPF LSM features above).
  • System V init scripts deprecated in 258, scheduled for removal in 259 — any /etc/init.d/foo shims should be ported to native .service units now.
  • D-Bus only / no GnuTLSsystemd-resolved and systemd-importd now require OpenSSL.

Timer units in depth

A timer unit activates another unit (usually a same-named .service) on a schedule. Pair foo.timer with foo.service; on enable, systemd creates a symlink from timers.target.wants/ so the timer starts at boot. The service can be a oneshot script (most common) or a long-running unit.

ini
# /etc/systemd/system/backup.timer
[Unit]
Description=Daily backup
Documentation=https://example.com/docs/backup

[Timer]
Unit=backup.service
OnCalendar=*-*-* 02:30:00
Persistent=true
RandomizedDelaySec=10min
AccuracySec=1min

[Install]
WantedBy=timers.target

Timer directives

DirectiveEffect
OnCalendar=wall-clock schedule (*-*-* 02:30, Mon..Fri 09:00, daily)
OnActiveSec=delay relative to timer activation
OnBootSec=delay relative to boot
OnStartupSec=delay relative to systemd start (different from boot in containers)
OnUnitActiveSec=interval between successive runs of the paired unit
OnUnitInactiveSec=interval after the paired unit last finished
Persistent=truecatch up missed runs (anacron-style) when the system was off
RandomizedDelaySec=jitter to avoid thundering herds across hosts
AccuracySec=window the timer can fire within (default 1min)
WakeSystem=truewake the machine from suspend to fire
Unit=which unit to start (default: same name with .service)
bash
# Test a calendar expression before saving the file
systemd-analyze calendar 'Mon..Fri 09:00'
systemd-analyze calendar '*-*-* 02:30:00'
systemd-analyze calendar 'weekly'

Output (systemd-analyze calendar 'Mon..Fri 09:00'):

text
  Original form: Mon..Fri 09:00
Normalized form: Mon..Fri *-*-* 09:00:00
    Next elapse: Mon 2026-05-25 09:00:00 EDT
       (in UTC): Mon 2026-05-25 13:00:00 UTC
       From now: 22h left

Common OnCalendar expressions

ExpressionFires
*-*-* 02:30:00every day at 02:30
Mon..Fri 09:00weekdays at 09:00
*-*-01 04:00:00the 1st of every month at 04:00
Sun *-*-* 03:00every Sunday at 03:00
*:0/15every 15 minutes
hourly / daily / weekly / monthly / yearlynamed shortcuts
*-*-* *:00:00top of every hour
2026-12-31 23:59:00once, at a specific instant

Prefer OnCalendar= to OnUnitActiveSec= whenever you want "at a wall-clock time" semantics — calendar timers run at the declared time regardless of when the machine last booted, and pair naturally with Persistent=true.

Timer inspection

bash
# List all active timers and their next/last fire times
systemctl list-timers --all

# Verify a timer's next elapse
systemctl status backup.timer

# Trigger the paired service NOW (does not affect schedule)
systemctl start backup.service

# Stream logs of the paired service across runs
journalctl -u backup.service --since today

Output (systemctl list-timers):

text
NEXT                        LEFT     LAST                        PASSED   UNIT          ACTIVATES
Sun 2026-05-25 02:30:00 EDT 16h left Sat 2026-05-24 02:30:00 EDT 7h ago   backup.timer  backup.service

See the cron cheatsheet for a head-to-head comparison of cron vs. timers and recipes for migrating existing jobs.

Socket activation

A .socket unit tells systemd to open a listening socket on the service's behalf and only start the paired service when the first connection arrives. This pattern (originally from inetd) cuts memory usage for rarely-used daemons and decouples startup order — clients can connect before the server is "up" because systemd queues the connection.

ini
# /etc/systemd/system/myapp.socket
[Unit]
Description=myapp socket

[Socket]
ListenStream=8080
Accept=no
SocketUser=myapp
SocketGroup=myapp

[Install]
WantedBy=sockets.target
ini
# /etc/systemd/system/myapp.service
[Unit]
Description=myapp
Requires=myapp.socket

[Service]
Type=simple
ExecStart=/opt/myapp/bin/myapp
User=myapp
StandardInput=socket
bash
sudo systemctl daemon-reload
sudo systemctl enable --now myapp.socket
ss -tlnp | grep 8080

Output: (ss -tlnp | grep 8080)

text
LISTEN 0  128  0.0.0.0:8080  0.0.0.0:*  users:(("systemd",pid=1,fd=42))

Accept=no (the default) hands the listening socket to a single long-lived service process — the modern style. Accept=yes forks a new instance of an @.service template unit per connection (inetd style); reserve this for legacy daemons that expect one process per client.

User services

Replacing --user everywhere flips systemd into the per-user instance that runs at login and exits at logout. User units live under ~/.config/systemd/user/ and target default.target. No root required, and the user can experiment freely without affecting the system.

bash
# Create a user service
mkdir -p ~/.config/systemd/user
cat > ~/.config/systemd/user/sync.service <<'EOF'
[Unit]
Description=Sync notes

[Service]
Type=oneshot
ExecStart=/home/alice/bin/sync-notes.sh
EOF

# Enable + start, just like system services but with --user
systemctl --user daemon-reload
systemctl --user enable --now sync.service
systemctl --user status sync.service
journalctl --user -u sync.service --since today

Output (systemctl --user status sync.service):

text
● sync.service - Sync notes
     Loaded: loaded (/home/alice/.config/systemd/user/sync.service; enabled)
     Active: inactive (dead) since Sat 2026-05-24 10:14:02 EDT; 32s ago
    Process: 4821 ExecStart=/home/alice/bin/sync-notes.sh (code=exited, status=0/SUCCESS)

User services stop at logout by default. Enable lingering with loginctl enable-linger alicedev so a user's services keep running even when no session is open — the right setting for personal automation on a headless host.

Template units — one file, many instances

A unit file whose name contains @ (e.g. myapp@.service) is a template: systemd substitutes the part between @ and .service as %i (instance name) when you start myapp@foo.service. Use templates to manage N parallel workers without copy-pasting unit files.

ini
# /etc/systemd/system/worker@.service
[Unit]
Description=Worker %i

[Service]
Type=simple
ExecStart=/opt/myapp/bin/worker --id %i --port %i
User=myapp
Restart=on-failure

[Install]
WantedBy=multi-user.target
bash
# Start three instances
sudo systemctl enable --now worker@8001.service worker@8002.service worker@8003.service

# Specifier reference for templates:
#   %i  instance name (8001)
#   %I  instance name, unescaped
#   %n  full unit name (worker@8001.service)
#   %u  configured User=
#   %h  user's home directory
#   %t  runtime dir (/run for system, /run/user/UID for user)

Output: (none — exits 0 on success)

systemd-analyze — boot timing and unit linting

systemd-analyze introspects everything systemd knows: how long boot took, which units took the longest, whether your calendar expressions are correct, and whether your service's sandboxing is solid. Run systemd-analyze blame after any boot regression to identify the culprit unit quickly.

bash
# Total boot time, broken down by phase
systemd-analyze time

# Per-unit start time, slowest first
systemd-analyze blame | head -20

# Visualise the dependency chain that determined boot duration
systemd-analyze critical-chain
systemd-analyze critical-chain myapp.service

# Verify a unit file's syntax without enabling it
systemd-analyze verify ./myapp.service

# Score a unit's sandboxing (0=worst, 10=best)
systemd-analyze security myapp.service

# Render the boot timeline as SVG
systemd-analyze plot > boot.svg

Output (systemd-analyze time):

text
Startup finished in 1.834s (kernel) + 4.213s (userspace) = 6.047s
graphical.target reached after 4.198s in userspace

Output (systemd-analyze blame excerpt):

text
2.011s NetworkManager-wait-online.service
 812ms snapd.service
 521ms apt-daily-upgrade.service
 480ms systemd-journal-flush.service
 312ms postgresql.service

Journal integration

By default, anything a service writes to stdout or stderr lands in the systemd journal, tagged with the unit name. That makes journalctl -u <unit> the canonical place to look. Override StandardOutput=/StandardError= only when you have a reason to (a daemon that already writes structured logs to its own file, for instance).

bash
# Follow a service's logs live
journalctl -u myapp.service -f

# Since boot / since a time
journalctl -u myapp.service -b
journalctl -u myapp.service --since "1 hour ago"
journalctl -u myapp.service --since "2026-05-24 09:00" --until "2026-05-24 10:00"

# Only errors and worse
journalctl -u myapp.service -p err

# JSON output for machine parsing
journalctl -u myapp.service -o json --no-pager

# Disk usage of the journal
journalctl --disk-usage

# Vacuum old entries
sudo journalctl --vacuum-time=30d
sudo journalctl --vacuum-size=500M

Output: (journalctl --disk-usage)

text
Archived and active journals take up 312.4M in the file system.

See the dedicated journalctl cheatsheet for query syntax and filters.

Common pitfalls

  1. Forgot daemon-reload — edits to a unit file have no effect until you run sudo systemctl daemon-reload. systemctl edit and package upgrades do it for you; manual edits don't.
  2. Type= doesn't match the binary — a Go or Python daemon that runs in the foreground is Type=simple, not Type=forking. If you set forking on a non-forking process, systemd thinks the service crashed instantly.
  3. network.target vs network-online.targetnetwork.target just means "network stack is loaded" and is reached very early. If your service contacts an external API at startup, depend on network-online.target and add Wants=network-online.target too.
  4. Restart loopsRestart=always without StartLimitBurst= will hammer a broken binary forever. Always cap the restart rate.
  5. Quoting in Environment=Environment=FOO=bar baz sets FOO=bar and silently ignores baz. Use Environment="FOO=bar baz" to include spaces.
  6. ExecStart= shell features don't work — there's no shell. ExecStart=/bin/sh -c '...' is the workaround when you genuinely need pipes or globbing.
  7. User= without ownership — the unit fails to start because the user can't read WorkingDirectory=. chown -R alice:alice /opt/myapp after creating the user.
  8. Editing the wrong file/usr/lib/systemd/system/ files get overwritten by package upgrades. Always override via systemctl edit (drop-ins under /etc/systemd/system/<unit>.d/).
  9. After= without Wants= or Requires= — ordering with no dependency means the unit will be ordered after the other unit if both happen to start, but the other unit won't be pulled in. Add Wants= or Requires=.
  10. Lingering not enabled for user services — user-level units die at logout unless loginctl enable-linger alicedev.

Real-world recipes

A robust long-running daemon

A production-style service: drops privileges, sandboxes itself, restarts on failure but not in a loop, and writes everything to the journal.

ini
# /etc/systemd/system/myapp.service
[Unit]
Description=My Application
Documentation=https://example.com/docs/myapp
After=network-online.target postgresql.service
Wants=network-online.target
Requires=postgresql.service
StartLimitIntervalSec=120
StartLimitBurst=4

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
EnvironmentFile=-/etc/myapp/env
ExecStartPre=/usr/bin/mkdir -p /run/myapp
ExecStart=/opt/myapp/bin/myapp --config /etc/myapp/config.toml
ExecReload=/bin/kill -HUP $MAINPID
Restart=on-failure
RestartSec=5s
TimeoutStopSec=30s

# Sandboxing
NoNewPrivileges=yes
ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes
PrivateDevices=yes
ProtectKernelTunables=yes
ProtectKernelModules=yes
ProtectControlGroups=yes
RestrictNamespaces=yes
RestrictRealtime=yes
LockPersonality=yes
ReadWritePaths=/var/lib/myapp /var/log/myapp
CapabilityBoundingSet=

# Logging
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
bash
sudo systemctl daemon-reload
sudo systemctl enable --now myapp.service
sudo systemctl status myapp.service

Output: (systemctl status myapp.service)

text
● myapp.service - My Application
     Loaded: loaded (/etc/systemd/system/myapp.service; enabled; preset: enabled)
     Active: active (running) since Sun 2026-05-25 09:14:02 EDT; 12s ago
   Main PID: 4821 (myapp)
      Tasks: 7 (limit: 4915)
     Memory: 24.3M
        CPU: 312ms
     CGroup: /system.slice/myapp.service
             └─4821 /opt/myapp/bin/myapp --config /etc/myapp/config.toml

A timer that runs every five minutes

For polling-style jobs that don't need an external scheduler. OnUnitActiveSec=5min measures from the previous run's start; pair it with OnBootSec= so the first run happens shortly after boot.

ini
# /etc/systemd/system/poll.timer
[Unit]
Description=Poll feeds every 5 minutes

[Timer]
OnBootSec=2min
OnUnitActiveSec=5min
AccuracySec=10s

[Install]
WantedBy=timers.target
ini
# /etc/systemd/system/poll.service
[Unit]
Description=Poll feeds

[Service]
Type=oneshot
ExecStart=/home/alice/bin/poll.sh
User=alicedev
bash
sudo systemctl daemon-reload
sudo systemctl enable --now poll.timer
systemctl list-timers poll.timer

Output: (systemctl list-timers poll.timer)

text
NEXT                        LEFT     LAST                        PASSED  UNIT        ACTIVATES
Sun 2026-05-25 09:19:02 EDT 4min     Sun 2026-05-25 09:14:02 EDT 32s ago poll.timer  poll.service

A one-shot triggered at boot

For tasks that need to run once during startup — flushing caches, initialising hardware, applying sysctl tweaks. Type=oneshot with RemainAfterExit=yes makes the unit show as active (exited) after the command finishes, which lets other units depend on it via After=.

ini
# /etc/systemd/system/firstrun.service
[Unit]
Description=Apply one-time first-boot configuration
After=network-online.target
Wants=network-online.target
ConditionPathExists=!/var/lib/firstrun.done

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/local/sbin/firstrun.sh
ExecStartPost=/bin/touch /var/lib/firstrun.done

[Install]
WantedBy=multi-user.target

Alert on failure

OnFailure= activates another unit whenever this one fails. A common pattern is to pair every important service with an alert@.service template that posts to Slack or fires mail.

ini
# /etc/systemd/system/alert@.service
[Unit]
Description=Notify on %i failure

[Service]
Type=oneshot
ExecStart=/usr/local/bin/notify-failure.sh %i
ini
# In the protected service:
[Unit]
OnFailure=alert@%n.service
bash
# /usr/local/bin/notify-failure.sh
#!/usr/bin/env bash
unit="$1"
status=$(systemctl status --no-pager "$unit" | head -20)
curl -fsS -X POST -H 'Content-Type: application/json' \
  -d "{\"text\":\"$unit failed on myhost\n\`\`\`$status\`\`\`\"}" \
  "$SLACK_WEBHOOK"

Output: (none — exits 0 on success)

Convert a cron job to a timer

For migrations from cron to systemd. Keep the script identical; replace the cron line with a .service + .timer pair, then disable the cron entry.

text
# Old cron entry:
# 0 3 * * *  /home/alice/bin/backup.sh

# New: ~/.config/systemd/user/backup.service + backup.timer
#   [Timer] OnCalendar=*-*-* 03:00:00
#   Persistent=true
#   RandomizedDelaySec=5min
bash
systemctl --user daemon-reload
systemctl --user enable --now backup.timer
crontab -l | grep -v 'backup.sh' | crontab -

Output: (none — exits 0 on success)

See cron for the cron-side details and a side-by-side feature comparison.

Treat unit files as code: keep them under version control, deploy them with rsync/Ansible/Nix, and let systemctl daemon-reload + systemctl restart apply the change. Don't edit production unit files by hand.

[!WARN] systemctl disable only removes the install symlinks; it does not stop the unit. Use systemctl disable --now <unit> to stop and disable in one step. Likewise enable does not start — enable --now does.

Sources