cheat sheet
logging
Configure Python's built-in logging module — loggers, handlers, formatters, dictConfig, rotation, structured records, and how it compares to loguru.
logging — Stdlib Logging
What it is
logging is Python's standard library for emitting timestamped, leveled diagnostic records from applications and libraries. It ships with the interpreter (no install needed), has been the recommended approach since PEP 282 (2002), and is what every well-behaved library uses under the hood. Reach for it when you want zero dependencies, full control, or interoperability with frameworks (Django, Flask, gunicorn, uvicorn, pytest) that already configure logging on your behalf. For greenfield apps where ergonomics matter more than stdlib purity, loguru wraps the same idea in a far smaller API.
Install
logging is part of the Python standard library and requires no installation. Verify the module loads:
python -c "import logging; print(logging.__name__, logging.getLogger().handlers)"
Output:
logging []
Mental model
logging separates four concerns: loggers generate records, handlers send records to a destination, formatters turn records into text, and filters drop records that don't match a predicate. A single record travels logger -> handler chain -> formatter -> output sink. Understanding this pipeline is the difference between "logs randomly appear or vanish" and reliable, structured output.
import logging
logger = logging.getLogger("app") # 1. logger
handler = logging.StreamHandler() # 2. handler -> stderr
fmt = logging.Formatter("%(levelname)s %(name)s: %(message)s") # 3. formatter
handler.setFormatter(fmt)
logger.addHandler(handler)
logger.setLevel(logging.INFO)
logger.info("Server started on port %d", 8080)
Output:
INFO app: Server started on port 8080
Log levels
Levels are integers with named constants. A logger emits a record only if its level is at or above the threshold set on both the logger and the destination handler. The five standard levels (plus NOTSET = 0 and the implicit floor) cover everything most apps need.
| Level | Constant | Numeric | Typical use |
|---|---|---|---|
DEBUG | logging.DEBUG | 10 | Verbose tracing only useful while debugging |
INFO | logging.INFO | 20 | Normal lifecycle events: "server started", "job complete" |
WARNING | logging.WARNING | 30 | Recoverable problem the operator should know about |
ERROR | logging.ERROR | 40 | Operation failed but the program continues |
CRITICAL | logging.CRITICAL | 50 | Program cannot continue |
import logging
logging.basicConfig(level=logging.DEBUG)
log = logging.getLogger("demo")
log.debug("connecting to %s", "db.internal")
log.info("connected")
log.warning("retry attempt %d", 3)
log.error("operation failed: %s", "timeout")
log.critical("out of disk; shutting down")
Output:
DEBUG:demo:connecting to db.internal
INFO:demo:connected
WARNING:demo:retry attempt 3
ERROR:demo:operation failed: timeout
CRITICAL:demo:out of disk; shutting down
Use lazy
%-style formatting (log.info("x=%s", val)) rather than f-strings (log.info(f"x={val}")). The lazy form skips string interpolation entirely when the level is disabled — anif log.isEnabledFor(DEBUG)is built in for free.
The getLogger(__name__) pattern
The canonical way to obtain a logger in a library or app module is logger = logging.getLogger(__name__) at the top of every file. __name__ evaluates to the dotted module path (myapp.api.routes), giving each module its own logger and forming a hierarchy that mirrors the package tree. This lets operators turn on debug logging for a single module without firehosing the whole app.
# myapp/api/routes.py
import logging
logger = logging.getLogger(__name__) # name = "myapp.api.routes"
def handle_request(req):
logger.info("handling %s", req.path)
Output: (none — exits 0 on success)
Loggers are hierarchical and share state via name (myapp.api.routes inherits from myapp.api which inherits from myapp which inherits from the root logger ""). Configuring the root logger with basicConfig automatically configures every child unless the child has handlers of its own.
import logging
logging.getLogger("myapp").setLevel(logging.DEBUG)
logging.getLogger("myapp.api").setLevel(logging.INFO) # quieter than parent
logging.getLogger("myapp.api.routes").debug("trace") # filtered out — parent INFO wins
Output: (none — record below effective level)
basicConfig — the fast path
logging.basicConfig() configures the root logger with a StreamHandler, a default format, and a chosen level. It's the right tool for scripts and small CLIs; it's the wrong tool for libraries (a library should never call basicConfig — that's the application's job).
import logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
logging.getLogger("script").info("started")
Output:
2026-05-25 14:02:31 | INFO | script | started
basicConfigis idempotent only on first call — subsequent calls in the same process are no-ops unless you passforce=True. If your tests reimport modules and callbasicConfig, surprise: only the first one wins. Useforce=Truein test fixtures.
Handlers
A handler decides where a record goes (stderr, file, syslog, HTTP endpoint, etc.). One logger can have many handlers and the same record fans out to all of them. Each handler can have its own level threshold and its own formatter.
| Handler | Module | Sends to |
|---|---|---|
StreamHandler | logging | A stream (default sys.stderr) |
FileHandler | logging | A single file (appends) |
RotatingFileHandler | logging.handlers | A file, rotates by size |
TimedRotatingFileHandler | logging.handlers | A file, rotates by time interval |
SysLogHandler | logging.handlers | Local or remote syslog |
SMTPHandler | logging.handlers | Email (one per record — use sparingly) |
QueueHandler / QueueListener | logging.handlers | Multiprocess-safe handoff |
NullHandler | logging | Discards (used by libraries) |
import logging
import sys
logger = logging.getLogger("app")
logger.setLevel(logging.DEBUG)
stream = logging.StreamHandler(sys.stderr)
stream.setLevel(logging.WARNING) # only warn+ to console
file = logging.FileHandler("/var/log/app.log")
file.setLevel(logging.DEBUG) # everything to file
fmt = logging.Formatter("%(asctime)s %(levelname)s %(message)s")
stream.setFormatter(fmt)
file.setFormatter(fmt)
logger.addHandler(stream)
logger.addHandler(file)
logger.debug("low-level detail") # file only
logger.warning("disk 80%% full") # both
Output (stderr):
2026-05-25 14:05:12,331 WARNING disk 80% full
RotatingFileHandler
RotatingFileHandler writes to a single active file and rolls it over to app.log.1, app.log.2, ... when it exceeds maxBytes. Use it for long-running daemons where unbounded log growth would fill the disk. Pair maxBytes with backupCount to cap total disk usage at roughly maxBytes * (backupCount + 1).
import logging
from logging.handlers import RotatingFileHandler
handler = RotatingFileHandler(
"/home/alice/logs/app.log",
maxBytes=10 * 1024 * 1024, # 10 MB per file
backupCount=5, # keep app.log.1 ... app.log.5
encoding="utf-8",
)
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
logger = logging.getLogger("app")
logger.setLevel(logging.INFO)
logger.addHandler(handler)
for i in range(100_000):
logger.info("event %d", i)
Output: (none — file /home/alice/logs/app.log rotates as it fills)
TimedRotatingFileHandler
TimedRotatingFileHandler rolls over at fixed time intervals (when="midnight", when="H", when="D") and stamps each rolled file with a date suffix. This is the right choice when downstream log aggregators (logrotate, Splunk forwarder, Loki promtail) expect one file per day.
import logging
from logging.handlers import TimedRotatingFileHandler
handler = TimedRotatingFileHandler(
"/home/alice/logs/app.log",
when="midnight", # roll over at 00:00 local time
interval=1,
backupCount=14, # keep two weeks
encoding="utf-8",
utc=False,
)
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
logger = logging.getLogger("app")
logger.setLevel(logging.INFO)
logger.addHandler(handler)
logger.info("daily job complete")
Output: (none — produces app.log.2026-05-24, app.log.2026-05-25, ...)
Formatters
A formatter controls how a LogRecord is rendered to text. The default style uses %-substitution against record attributes; pass style="{" for str.format or style="$" for string.Template. Common record attributes are listed below.
| Placeholder | Meaning |
|---|---|
%(asctime)s | Human-readable timestamp |
%(created)f | Unix timestamp |
%(name)s | Logger name |
%(levelname)s | DEBUG / INFO / ... |
%(levelno)d | Numeric level |
%(message)s | The formatted log message |
%(pathname)s | Full source path |
%(filename)s | Source filename |
%(module)s | Module name |
%(funcName)s | Calling function |
%(lineno)d | Source line number |
%(process)d | Process ID |
%(processName)s | Process name |
%(thread)d | Thread ID |
%(threadName)s | Thread name |
import logging
fmt = logging.Formatter(
"{asctime} [{levelname:<8}] {name}:{funcName}:{lineno} | {message}",
style="{",
datefmt="%Y-%m-%dT%H:%M:%S",
)
h = logging.StreamHandler()
h.setFormatter(fmt)
logging.basicConfig(level=logging.INFO, handlers=[h], force=True)
logging.getLogger("myapp").info("user signed in: %s", "alicedev")
Output:
2026-05-25T14:07:55 [INFO ] myapp:<module>:9 | user signed in: alicedev
dictConfig — the right way to configure
logging.config.dictConfig accepts a single nested dict describing every logger, handler, formatter, and filter in the application. Define it once at startup; never call basicConfig from inside library code. The schema is documented in the Logging Configuration reference.
import logging
import logging.config
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"default": {
"format": "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s",
"datefmt": "%Y-%m-%d %H:%M:%S",
},
"json": {
"()": "myapp.logging.JsonFormatter", # your own class
},
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "INFO",
"formatter": "default",
"stream": "ext://sys.stderr",
},
"file": {
"class": "logging.handlers.RotatingFileHandler",
"level": "DEBUG",
"formatter": "json",
"filename": "/home/alice/logs/app.log",
"maxBytes": 10485760,
"backupCount": 5,
},
},
"loggers": {
"myapp": {
"level": "DEBUG",
"handlers": ["console", "file"],
"propagate": False,
},
"uvicorn.access": {"level": "WARNING"}, # quiet noisy lib
},
"root": {
"level": "WARNING",
"handlers": ["console"],
},
}
logging.config.dictConfig(LOGGING)
logging.getLogger("myapp").info("ready")
Output:
2026-05-25 14:09:01 | INFO | myapp | ready
YAML / JSON configuration
Storing the dict above in a YAML or JSON file lets ops change logging without code changes. Load and apply at startup. Combine with environment-variable interpolation (os.environ.get) to inject log level per environment.
# /home/alice/myapp/logging.yaml
version: 1
disable_existing_loggers: false
formatters:
default:
format: "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s"
handlers:
console:
class: logging.StreamHandler
level: INFO
formatter: default
stream: ext://sys.stderr
loggers:
myapp:
level: DEBUG
handlers: [console]
propagate: false
root:
level: WARNING
handlers: [console]
Output: (none — config file)
import logging.config
import os
import yaml
with open(os.environ.get("LOG_CONFIG", "logging.yaml")) as f:
logging.config.dictConfig(yaml.safe_load(f))
logging.getLogger("myapp").info("config loaded from yaml")
Output:
2026-05-25 14:11:08 | INFO | myapp | config loaded from yaml
Structured logging via extra=
Every logger method accepts an extra= dict whose keys are promoted to record attributes. Combined with a custom formatter or a JSON formatter, this becomes the foundation for structured logging that log aggregators (Datadog, Loki, ELK) can parse and index.
import logging
logger = logging.getLogger("myapp")
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s | %(levelname)s | %(name)s | user_id=%(user_id)s | %(message)s",
)
logger.info("user signed in", extra={"user_id": 42})
logger.warning("rate limit close", extra={"user_id": 42})
Output:
2026-05-25 14:12:01,001 | INFO | myapp | user_id=42 | user signed in
2026-05-25 14:12:01,002 | WARNING | myapp | user_id=42 | rate limit close
Keys in
extramust not clash with built-inLogRecordattributes (message,asctime,levelno, etc.) — using a reserved name raisesKeyErrorat handler time.
JSON formatter (built-in approach)
Python 3.12+ ships logging.makeLogRecord and a sufficiently rich LogRecord that a tiny JSON formatter is one class. For older Pythons, the same shape works — or use the third-party python-json-logger package.
import json
import logging
class JsonFormatter(logging.Formatter):
BUILTIN = {
"name", "msg", "args", "levelname", "levelno", "pathname", "filename",
"module", "exc_info", "exc_text", "stack_info", "lineno", "funcName",
"created", "msecs", "relativeCreated", "thread", "threadName",
"processName", "process", "message", "asctime",
}
def format(self, record: logging.LogRecord) -> str:
payload = {
"ts": self.formatTime(record, "%Y-%m-%dT%H:%M:%S"),
"level": record.levelname,
"logger": record.name,
"msg": record.getMessage(),
}
for k, v in record.__dict__.items():
if k not in self.BUILTIN:
payload[k] = v
if record.exc_info:
payload["exc"] = self.formatException(record.exc_info)
return json.dumps(payload, default=str)
h = logging.StreamHandler()
h.setFormatter(JsonFormatter())
logging.basicConfig(level=logging.INFO, handlers=[h], force=True)
logging.getLogger("myapp").info("checkout", extra={"user_id": 42, "amount": 19.95})
Output:
{"ts": "2026-05-25T14:14:21", "level": "INFO", "logger": "myapp", "msg": "checkout", "user_id": 42, "amount": 19.95}
Exception logging
logger.exception(msg) is shorthand for logger.error(msg, exc_info=True) inside an except block — it logs the message plus a full traceback. Use it in every except clause where you swallow the exception; never log just str(e).
import logging
logging.basicConfig(level=logging.INFO)
log = logging.getLogger("myapp")
def risky():
return 1 / 0
try:
risky()
except ZeroDivisionError:
log.exception("risky() failed")
Output:
ERROR:myapp:risky() failed
Traceback (most recent call last):
File "/home/alice/myapp/run.py", line 9, in <module>
risky()
File "/home/alice/myapp/run.py", line 6, in risky
return 1 / 0
ZeroDivisionError: division by zero
Filters
A Filter is any object with a filter(record) -> bool method. Attach one to a handler (or logger) to drop or mutate records before they're emitted. Use cases: redact secrets, inject context (request ID, user ID), or sample noisy records.
import logging
import re
class RedactFilter(logging.Filter):
SECRET = re.compile(r"(token|password)=[^\s&]+", re.I)
def filter(self, record: logging.LogRecord) -> bool:
record.msg = self.SECRET.sub(r"\1=***", str(record.msg))
return True # keep the record
logging.basicConfig(level=logging.INFO)
log = logging.getLogger("auth")
log.addFilter(RedactFilter())
log.info("login url=/auth?token=abcdef123456&user=alicedev")
Output:
INFO:auth:login url=/auth?token=***&user=alicedev
Context propagation with LoggerAdapter
LoggerAdapter wraps a logger and merges a fixed extra dict into every record — the stdlib equivalent of loguru's logger.bind(). Use it inside request handlers, async tasks, or any unit of work where you want every log line tagged with a correlation ID.
import logging
import uuid
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s | %(levelname)s | req=%(request_id)s | %(message)s",
)
base = logging.getLogger("myapp")
def handle_request(path: str):
log = logging.LoggerAdapter(base, {"request_id": str(uuid.uuid4())[:8]})
log.info("incoming path=%s", path)
log.info("complete")
handle_request("/checkout")
Output:
2026-05-25 14:16:30 | INFO | req=a4f1c0d2 | incoming path=/checkout
2026-05-25 14:16:30 | INFO | req=a4f1c0d2 | complete
Multiprocessing — QueueHandler + QueueListener
Writing to the same file from multiple processes corrupts records (interleaved writes). The stdlib solution is QueueHandler in each worker plus a single QueueListener in the main process that drains the queue and forwards records to real handlers.
import logging
import logging.handlers
import multiprocessing as mp
def worker(q, i):
qh = logging.handlers.QueueHandler(q)
root = logging.getLogger()
root.addHandler(qh)
root.setLevel(logging.INFO)
logging.getLogger("worker").info("hello from worker %d", i)
if __name__ == "__main__":
q = mp.Queue(-1)
file_h = logging.handlers.RotatingFileHandler(
"/home/alice/logs/multi.log", maxBytes=1_000_000, backupCount=3
)
file_h.setFormatter(logging.Formatter("%(asctime)s %(processName)s %(message)s"))
listener = logging.handlers.QueueListener(q, file_h)
listener.start()
procs = [mp.Process(target=worker, args=(q, i)) for i in range(4)]
for p in procs: p.start()
for p in procs: p.join()
listener.stop()
Output (multi.log):
2026-05-25 14:17:01 Process-1 hello from worker 0
2026-05-25 14:17:01 Process-2 hello from worker 1
2026-05-25 14:17:01 Process-3 hello from worker 2
2026-05-25 14:17:01 Process-4 hello from worker 3
Library hygiene — NullHandler
Libraries should never configure logging themselves; they should attach a NullHandler to their package-level logger so users who don't configure logging don't get warnings about unconfigured loggers. This is what requests, urllib3, and most packages do.
# mylib/__init__.py
import logging
logging.getLogger(__name__).addHandler(logging.NullHandler())
Output: (none — silences "no handlers could be found" warning)
logging vs loguru
The two libraries solve the same problem from opposite directions. Pick stdlib when you need zero dependencies, framework interop, or fine-grained dictConfig. Pick loguru when ergonomics and out-of-the-box rotation matter more than dependency footprint.
| Feature | logging (stdlib) | loguru |
|---|---|---|
| Install | none | pip install loguru |
| Setup boilerplate | High (loggers + handlers + formatters) | Zero — from loguru import logger |
| Colorized output | DIY | Built-in |
| f-string-style messages | No (use %s for lazy) | Yes |
| File rotation | RotatingFileHandler config | logger.add("app.log", rotation="10 MB") |
| Compression on rotate | DIY | compression="zip" |
| Exception decorator | None | @logger.catch |
| JSON output | Custom formatter | serialize=True |
| Hierarchical loggers | Yes (dotted names) | Single logger + bind() |
| Library convention | All libs use it; expects you to wire handlers | Replaces stdlib via InterceptHandler |
Common pitfalls
basicConfigis a one-shot — second call is a no-op unlessforce=True. Don't call from libraries.logger.info(f"x={val}")is eager — string is built even at WARNING level. Uselogger.info("x=%s", val)for lazy formatting.- Mutating the root logger from a library is rude. Add a
NullHandlerand let the application configure. - Duplicate output comes from leaving
propagate=Trueon a child logger that has its own handler — the record reaches both the child handler and the root handler. Setpropagate=Falseon configured loggers. - Reserved keys in
extra=— using names likemessage,asctime,levelnoraisesKeyError. Pick distinct keys. - Multi-process logging to a single file silently corrupts records. Use
QueueHandler/QueueListener, separate files per process, or syslog. logger.error("failed: " + str(e))loses the traceback. Insideexcept, always uselogger.exception("failed").- Forgetting
disable_existing_loggers: falseindictConfigsilently disables every logger created before config (e.g. logger objects already imported byuvicorn). Set it tofalseunless you know what you're doing.
Real-world recipes
Quick script — one-line setup
A 20-line script doesn't need dictConfig. Use basicConfig with a sensible format and call it a day.
import logging
import sys
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)-8s %(message)s",
datefmt="%H:%M:%S",
stream=sys.stderr,
)
log = logging.getLogger(__name__)
log.info("download started")
log.warning("server returned 503; retrying")
log.info("download complete")
Output:
14:30:01 INFO download started
14:30:03 WARNING server returned 503; retrying
14:30:05 INFO download complete
Reusable logging.yaml loaded at app startup
A medium-sized app should externalize its logging config. Ship a default logging.yaml, let LOG_CONFIG env var override it, and call dictConfig exactly once in your entry point.
# myapp/log_setup.py
import logging
import logging.config
import os
from pathlib import Path
import yaml
DEFAULT = Path(__file__).parent / "logging.yaml"
def setup_logging() -> None:
cfg_path = Path(os.environ.get("LOG_CONFIG", DEFAULT))
with cfg_path.open() as f:
cfg = yaml.safe_load(f)
log_dir = Path(os.environ.get("LOG_DIR", "/home/alice/logs"))
log_dir.mkdir(parents=True, exist_ok=True)
for h in cfg.get("handlers", {}).values():
if "filename" in h:
h["filename"] = str(log_dir / h["filename"])
logging.config.dictConfig(cfg)
logging.getLogger(__name__).info("logging configured from %s", cfg_path)
Output:
2026-05-25 14:35:01 | INFO | myapp.log_setup | logging configured from /home/alice/myapp/logging.yaml
Per-request correlation IDs in FastAPI
Tag every log line within a single HTTP request with a request ID so logs from the same call can be filtered together. Use ContextVar for thread/async safety.
import logging
import uuid
from contextvars import ContextVar
from fastapi import FastAPI, Request
request_id_var: ContextVar[str] = ContextVar("request_id", default="-")
class RequestIdFilter(logging.Filter):
def filter(self, record: logging.LogRecord) -> bool:
record.request_id = request_id_var.get()
return True
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s | %(levelname)s | req=%(request_id)s | %(message)s",
)
for h in logging.getLogger().handlers:
h.addFilter(RequestIdFilter())
app = FastAPI()
@app.middleware("http")
async def add_request_id(request: Request, call_next):
token = request_id_var.set(uuid.uuid4().hex[:8])
try:
return await call_next(request)
finally:
request_id_var.reset(token)
@app.get("/")
async def root():
logging.getLogger("myapp").info("root handler")
return {"ok": True}
Output:
2026-05-25 14:40:11 | INFO | req=8a1f3c0d | root handler
Capturing third-party noise
Libraries log at INFO/DEBUG levels you usually don't want. Silence them per-logger by name without affecting your own logger tree.
import logging
# Your app logs DEBUG, but quiet down noisy libs
logging.basicConfig(level=logging.DEBUG)
for noisy in ("urllib3", "boto3", "botocore", "asyncio", "uvicorn.access"):
logging.getLogger(noisy).setLevel(logging.WARNING)
logging.getLogger("myapp").debug("verbose detail you want")
logging.getLogger("urllib3").info("noisy detail you don't")
Output:
DEBUG:myapp:verbose detail you want
Capturing logs in pytest
pytest captures log records via its caplog fixture so you can assert on log output without touching the global config. Records carry the structured LogRecord, not just the formatted string — assert on levelname, message, or any extra field.
import logging
def add_user(name: str, log: logging.Logger | None = None) -> None:
log = log or logging.getLogger(__name__)
log.info("adding user %s", name)
def test_add_user_logs(caplog):
with caplog.at_level(logging.INFO):
add_user("alicedev")
assert any("adding user alicedev" in r.message for r in caplog.records)
Output:
============================= test session starts ==============================
collected 1 item
test_add_user.py . [100%]
============================== 1 passed in 0.04s ===============================