tqdm

What it is

tqdm is a tiny, low-overhead Python library that wraps any iterable to display an auto-updating progress bar with ETA, throughput, and elapsed time. The name comes from the Arabic taqaddum ("progress"). It was created by Casper da Costa-Luis in 2015 and is now community-maintained. The library is the de-facto standard for loop progress in Python data pipelines, ML training scripts, and CLI tools.

Reach for tqdm when you need a drop-in progress indicator with zero configuration and minimal overhead (~60 ns / iteration). Reach for rich.progress when you also want styled tables, panels, or live displays in the same script; reach for alive-progress for fancy animated variants.

Install

pip install tqdm

Output: (none — exits 0 on success)

uv add tqdm

Output: resolved + added to pyproject.toml

poetry add tqdm

Output: updated lockfile + virtualenv install

pipx install tqdm        # global install with the standalone `tqdm` CLI on PATH

Output: installs tqdm CLI in an isolated venv, usable for shell-pipe progress (pv-style)

Versioning & Python support

• Current stable line is the 4.x series — tqdm has stayed on 4.x since 2016, with breaking changes deferred to a never-released 5.0.
• Supports Python 3.7+ on recent releases.
• Loose semver — 4.x minor releases add features and occasionally deprecate APIs with a long warning window.
• The tqdm.auto module dispatches to the right implementation (tqdm.notebook vs tqdm.std) based on the runtime; its detection logic has shifted slightly across 4.x minors.

Package metadata

• Maintainer: tqdm contributors (Casper da Costa-Luis et al.); community-driven
• Project home: github.com/tqdm/tqdm
• Docs: tqdm.github.io
• PyPI: pypi.org/project/tqdm
• License: MIT + MPL-2.0 (dual)
• Governance: Volunteer maintainer team; OpenCollective-funded
• First released: 2015
• Downloads: hundreds of millions per month — consistently in the PyPI top 30

Optional dependencies & extras

The published extras on PyPI:

• tqdm[notebook] — pulls in ipywidgets so tqdm.notebook.tqdm renders an HTML widget instead of plain text inside Jupyter
• tqdm[telegram] — pulls requests, enables tqdm.contrib.telegram to post progress to a Telegram bot
• tqdm[discord] — same idea for Discord webhooks
• tqdm[slack] — same for Slack
• tqdm[dev] — full dev/test toolchain (only useful when contributing to tqdm itself)

Core install has no required dependencies — tqdm is intentionally zero-dep on the standard library.

Common companions:

• pandas — tqdm.pandas() enables df.progress_apply(...)
• dask — built-in tqdm bridge for dask futures
• joblib — tqdm callback for Parallel
• rich — overlapping use case; alternative if you want styled tables alongside bars
• tqdm-multiprocess — community package for cleaner multi-process progress

Alternatives

Common gotchas

1. Nested bars need position=N. Inner loops printing without position= overwrite the outer bar. Pattern:

   pythoncopy

   for i in tqdm(outer, position=0):
       for j in tqdm(inner, position=1, leave=False):
           ...
   

   Output: two stacked bars; the inner one disappears when its loop ends.

2. tqdm.auto picks the wrong implementation on Windows ConHost. Detection relies on IPython.get_ipython() and TTY signals; under Windows ConHost-via-PowerShell the notebook variant occasionally wins. Force the std variant: from tqdm.std import tqdm.

3. tqdm[notebook] requires ipywidgets and a JupyterLab extension. In a fresh JupyterLab install, widgets need to be enabled before tqdm.notebook renders — otherwise you see raw IntProgress(value=0, ...) text.

4. Multi-process / multi-thread bars race over stdout. Use tqdm.contrib.concurrent.process_map / thread_map for parallel workloads, or multiprocess.Lock and pass it via tqdm(..., lock_args=(lock,)).

5. Logging conflicts. logging.info() calls during a tqdm loop scribble over the bar. Replace the logger handler with tqdm.contrib.logging.logging_redirect_tqdm() (context manager).

6. total= is mandatory for non-len() iterables. Generators have no length; without total= the bar shows iteration count but no ETA. Pass total=expected_count explicitly.

7. Default refresh rate is 10 Hz. For very fast inner loops (millions/sec), the formatting cost dominates. Pass mininterval=1.0 and miniters=1000 to cut overhead.

8. tqdm CLI piping. cat huge.csv | tqdm --bytes > out.csv adds a byte-progress meter to any pipeline — useful but underused.

Real-world recipes

Package-level recipes — integration patterns, CI handling, async usage. For the core API see the companion Python article.

Recipe 1 — ETA-aware download loop

import requests
from tqdm import tqdm

url = "https://example.com/big.bin"
with requests.get(url, stream=True) as r:
    total = int(r.headers.get("content-length", 0))
    with open("big.bin", "wb") as f, tqdm(
        total=total, unit="B", unit_scale=True, unit_divisor=1024, desc="big.bin"
    ) as bar:
        for chunk in r.iter_content(chunk_size=64 * 1024):
            f.write(chunk)
            bar.update(len(chunk))

Output: progress bar with human-readable units (MiB), throughput, and ETA derived from Content-Length. unit_divisor=1024 switches kilobytes to kibibytes (the standard for binary sizes).

Recipe 2 — async parallel scrape with tqdm.asyncio

import asyncio
import httpx
from tqdm.asyncio import tqdm_asyncio

URLS = [f"https://httpbin.org/delay/{i % 3}" for i in range(50)]

async def fetch(client, url):
    r = await client.get(url)
    return r.status_code

async def main():
    async with httpx.AsyncClient() as client:
        tasks = [fetch(client, u) for u in URLS]
        results = await tqdm_asyncio.gather(*tasks, desc="scraping")
    return results

asyncio.run(main())

Output: single progress bar tracking concurrent task completion (not start). tqdm_asyncio.gather is a drop-in for asyncio.gather.

Recipe 3 — process-pool with tqdm.contrib.concurrent.process_map

from tqdm.contrib.concurrent import process_map

def square(x: int) -> int:
    return x * x

if __name__ == "__main__":
    results = process_map(square, range(1000), max_workers=4, chunksize=10)

Output: progress bar tracks completion across 4 worker processes; chunksize batches inputs to reduce IPC overhead. The if __name__ == "__main__" guard is mandatory on macOS/Windows (spawn semantics).

Recipe 4 — CI-safe quiet mode

import os
from tqdm import tqdm

# Disable progress when running non-interactively (CI, cron, log capture)
DISABLE = not os.isatty(1) or os.environ.get("CI") == "true"

for item in tqdm(items, disable=DISABLE):
    process(item)

Output: interactive shells get the bar; CI/cron logs stay clean. The alternative mininterval=600 keeps the bar emitting one update per 10 min — useful when you want some progress signal in CI without flooding the log.

Recipe 5 — pandas .progress_apply()

import pandas as pd
from tqdm import tqdm

tqdm.pandas(desc="processing")

df = pd.DataFrame({"x": range(1_000_000)})
df["y"] = df["x"].progress_apply(lambda v: v * 2)

Output: every row goes through lambda with a single bar tracking the full apply. Identical pattern for progress_map, progress_applymap.

Recipe 6 — multi-line live status

from tqdm import tqdm
from tqdm.contrib.logging import logging_redirect_tqdm
import logging
import time

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(message)s")
log = logging.getLogger("worker")

items = list(range(20))
with logging_redirect_tqdm():
    for i in tqdm(items, desc="processing"):
        if i % 5 == 0:
            log.info("milestone at %d", i)
        time.sleep(0.05)

Output: the bar stays at the bottom of the terminal; log lines scroll above it without scribbling. logging_redirect_tqdm() is the canonical fix for "my prints are mangling my bar".

Performance tuning

Refresh-rate vs update-count

mininterval (default 0.1 s) and miniters (default dynamically tuned) gate how often the bar redraws. For very fast loops (millions/sec), the formatting cost dominates the actual work:

for i in tqdm(range(100_000_000), mininterval=1.0, miniters=10_000):
    pass

Output: bar updates once per second OR every 10k iterations, whichever comes first. The fastest possible setting.

smoothing parameter

The displayed throughput / ETA uses exponential smoothing. Default smoothing=0.3 mixes recent samples; smoothing=0 shows the lifetime average (less jumpy), smoothing=1 shows only the most recent interval (most reactive).

total= discipline

Generators have no length — without total=, the bar shows iteration count and elapsed time but no ETA. Always pass total= when known.

position and leave for nested bars

position=0 is the outermost; nested loops use position=1, position=2. leave=False removes the bar on completion — essential for nested bars so the terminal isn't littered with finished inner bars.

Drop the import in tight scripts

tqdm itself imports in ~30 ms. For very short-lived scripts, lazy-import inside a function or skip tqdm entirely for runs under a few seconds.

Version migration guide

tqdm has been on 4.x since 2016, with an unreleased 5.0 that's been on the horizon for years. The 4.x line is remarkably stable; migrations are gentle.

tqdm.auto history

The dispatcher between tqdm.notebook and tqdm.std is in tqdm.auto. Its detection logic has shifted slightly across 4.50 / 4.60 / 4.66 — under Jupyter or IPython kernels it picks notebook; under terminal, std. Force a specific implementation if auto-detection misfires:

from tqdm.std import tqdm        # always terminal
from tqdm.notebook import tqdm   # always IPython widget

Output: explicit choice; no auto-detection.

tqdm[notebook] extra

The notebook extra (added in ~4.40) pulls in ipywidgets. Older code may install ipywidgets separately — preferred to use the extra.

tqdm.contrib.concurrent evolution

process_map and thread_map were experimental for several releases before stabilising. APIs are stable now; max_workers and chunksize are the main knobs.

tqdm.contrib.logging

logging_redirect_tqdm() (context manager) was added in 4.55. Prior code used manual tqdm.write() calls instead of integrating with logging.

Pre-5.0 expectations

The community has signalled that 5.0 will be primarily a Python-floor bump and a cleanup of deprecated parameters; no major API rewrite is expected. If you pin to tqdm>=4,<5 you're safe.

Testing & CI integration

Disable in tests

Pass disable=True to suppress the bar inside test bodies; pytest captures stderr by default, and an active progress bar can interfere with assertion output.

def test_with_progress(monkeypatch):
    monkeypatch.setenv("TQDM_DISABLE", "1")    # set env var
    ...

For library code, accept disable=None (the default — auto-detects TTY) so callers can opt out:

def slow_op(items, disable=None):
    for item in tqdm(items, disable=disable):
        ...

Output: consumers can disable via slow_op(items, disable=True) for tests.

Assert on tqdm output (rare but useful)

Use tqdm(..., file=stringbuf) to redirect output to a buffer:

from io import StringIO
from tqdm import tqdm

buf = StringIO()
for _ in tqdm(range(10), file=buf, ncols=80, disable=False):
    pass
assert "100%" in buf.getvalue()

Output: captured bar string; useful for testing custom format strings or unit_scale behavior.

CI log noise

A live bar emits a carriage-return after each refresh — in a CI log viewer that doesn't interpret CR, this becomes a single very long line. Mitigations:

• ascii=True — pure-ASCII bar characters (no Unicode bar glyphs).
• mininterval=60 — one update per minute; keeps logs tractable.
• disable=True in CI; rely on log lines instead.

Ecosystem integrations

• pandas — tqdm.pandas() adds .progress_apply, .progress_map, .progress_applymap to DataFrames and Series.
• dask — tqdm.dask.TqdmCallback() hooks into Dask futures.
• joblib — wrap Parallel calls with tqdm_joblib context (community recipe).
• asyncio — tqdm.asyncio.tqdm_asyncio.gather drops in for asyncio.gather.
• concurrent.futures — tqdm.contrib.concurrent.process_map / thread_map wrap ProcessPoolExecutor / ThreadPoolExecutor.
• ipywidgets — required for tqdm.notebook widget bars.
• rich.progress — alternative; not a tqdm extension but the most common replacement for richer styling.
• telegram / discord / slack — tqdm.contrib.{telegram,discord,slack} post progress to a chat webhook for long-running jobs you walk away from.
• keras / pytorch — both ecosystems have first-party progress bars built on tqdm or compatible APIs; you can usually swap in your own tqdm instance.
• huggingface datasets / transformers — uses tqdm under the hood; HF_HUB_DISABLE_PROGRESS_BARS=1 env var to silence.

Troubleshooting common errors

Bar duplicates / "rebounds" on each iteration

Two tqdm instances are writing to the same TTY without position=. Set position=0 on outer and position=1, leave=False on inner; or use tqdm.contrib.concurrent.process_map which handles positioning internally.

Bar is invisible in a terminal but other output appears

stderr is being captured / redirected. tqdm writes to stderr by default; pass file=sys.stdout if you want it on stdout, or check whether your runner swallows stderr.

TypeError: object of type 'generator' has no len()

A generator was passed without total=. tqdm can iterate but has no length — pass total=expected_count to get an ETA. If unknown, the bar will still show iteration count + elapsed.

Bar updates lag in a long-running async loop

tqdm.asyncio.tqdm_asyncio.gather only ticks on task completion, not start. For finer-grained updates, manually wrap each task with tqdm.update(1) calls.

Widget renders as plain text in Jupyter

tqdm.notebook requires ipywidgets and an enabled JupyterLab extension. Install: pip install "tqdm[notebook]" ipywidgets and reload the lab page.

Bar collides with logger output

Use from tqdm.contrib.logging import logging_redirect_tqdm and wrap the loop in with logging_redirect_tqdm(): — log lines scroll above the bar instead of overwriting it.

Compatibility matrix

Environment compatibility:

When NOT to use this

• Silent batch jobs. A cron job or systemd unit writing to a log file gains nothing from a bar that becomes garbled CR-output. Use periodic logging.info calls instead.
• High-throughput inner loops (>10M iter/sec). The formatting cost — even with aggressive mininterval — adds measurable overhead. Bench it; if tqdm is >5% of runtime, drop it.
• Long-running daemon processes. A bar with total=None and elapsed time isn't meaningful. Use a metrics emitter (Prometheus, StatsD) for "how long has this been running".
• Sub-second tasks. A bar for a 200 ms task is visual noise. Skip it.
• Distributed-system progress. A single tqdm bar can't see across machines. Aggregate via a centralised counter (Redis, a metrics backend) and visualise in Grafana.
• Strict structured logs. Mixing CR-rewritten progress bars with JSON-per-line logs breaks the line-oriented contract. Use a separate progress channel or disable tqdm in production.

See also

• Python: tqdm — iterable, manual, pandas, async, Jupyter
• Packages: pip-rich — alternative with styled tables and panels
• Packages: pip-jupyter — host for tqdm.notebook widget mode