Python · Testing with pytest · Intermediate

Test Coverage

5 tasks

Measure which lines of code are exercised by your tests with pytest-cov.

Test Coverage with pytest-cov

#
## The question coverage answers You have a test suite — but how much of your actual code does it exercise? A function with three branches might be tested only on the happy path, leaving two branches completely untouched. **Coverage** measures which lines (and which branches) were actually executed when your tests ran. Coverage is a diagnostic tool: it tells you where your tests aren't looking. It does not tell you whether your tests are *good*. ## pytest-cov: coverage integrated into pytest `pytest-cov` is a plugin that wraps `coverage.py` and runs it alongside your tests: ```bash pip install pytest-cov ``` Run tests with coverage measurement: ```bash pytest --cov=src tests/ ``` `--cov=src` specifies the package or directory to measure. At the end of the test run, you get a coverage report automatically. ## Reading the terminal report ``` ---------- coverage: platform linux, python 3.11 ---------- Name Stmts Miss Cover ------------------------------------------ src/products.py 42 3 93% src/auth.py 28 8 71% src/utils.py 15 0 100% ------------------------------------------ TOTAL 85 11 87% ``` - **Stmts** — total executable statements in the file - **Miss** — statements that were never reached during the test run - **Cover** — percentage of Stmts that were executed To see *which* lines were missed, add `--cov-report=term-missing`: ```bash pytest --cov=src --cov-report=term-missing tests/ ``` Output includes a `Missing` column: ``` src/auth.py 28 8 71% 45-52, 78 ``` Lines 45–52 and 78 were never executed. Open the file — they might be an error-handling block that needs a test for "what if the database is down." ## HTML report: the best view ```bash pytest --cov=src --cov-report=html tests/ ``` Generates `htmlcov/index.html`. Open it in a browser: executed lines are green, unexecuted are red. Click any file to see exactly which lines are uncovered in context. ## Line coverage vs branch coverage **Line coverage** records whether a line ran at all. But one `if` statement contains two paths: ```python def discount(price, is_member): if is_member: # ← ran return price * 0.9 # ← ran (True branch) return price # ← NEVER ran (False branch) ``` If tests only call `discount(100, True)`, line coverage reports 100% — every line ran. But the `False` path was never tested. **Branch coverage** catches this. Enable it: ```bash pytest --cov=src --cov-branch tests/ ``` The report now tracks each branch of every `if`, `for`, `while`, `try/except`, and conditional expression. A branch that never runs appears as a gap. ## Why 100% line coverage is not the goal You can reach 100% line coverage with tests that assert nothing. You can have 75% coverage with tests that catch every real bug. The number is a hint, not a target. The right way to use coverage: - **Find gaps** — which non-trivial code paths have no tests? Are they important? - **Set a minimum floor** — `--cov-fail-under=80` in CI prevents accidental regressions (coverage dropping from 85% to 60% after a refactor is a red flag) - **Identify dead code** — a line that's *always* uncovered might be unreachable and safe to delete Don't chase the number. Chase meaningful tests. ## Excluding code from coverage Not everything is worth measuring. Config files, migration stubs, debug utilities — exclude them: **Via `.coveragerc`:** ```ini [run] omit = tests/* setup.py */migrations/* [report] exclude_lines = if __name__ == .__main__.: raise NotImplementedError ``` **Inline with pragma:** ```python if __name__ == '__main__': # pragma: no cover main() ``` The pragma comment suppresses that line from all coverage reports. Use sparingly — only for code that genuinely cannot be tested meaningfully.

Coverage in Practice

#
## The code under test ```python # src/pricing.py def calculate_price(base_price, is_member, coupon_code=None): price = base_price if is_member: price *= 0.9 # 10% member discount if coupon_code == 'SAVE20': price *= 0.8 # additional 20% off elif coupon_code is not None: raise ValueError(f'Unknown coupon: {coupon_code}') return round(price, 2) ``` ## Partial test suite (intentionally incomplete) ```python # tests/test_pricing.py from src.pricing import calculate_price def test_no_discount(): assert calculate_price(100, is_member=False) == 100.0 def test_member_discount(): assert calculate_price(100, is_member=True) == 90.0 ``` ## Running coverage and reading the report ```bash pytest --cov=src --cov-report=term-missing tests/ ``` ``` Name Stmts Miss Cover Missing ------------------------------------------------- src/pricing.py 9 2 78% 10, 12 ``` 78% — the coupon code branches were never tested. Lines 10 and 12 are the `price *= 0.8` block and the `raise ValueError` block. ## Adding branch coverage ```bash pytest --cov=src --cov-branch --cov-report=term-missing tests/ ``` ``` Name Stmts Miss Branch BrPart Cover src/pricing.py 9 2 6 2 64% ``` Lower still — the untested branches inside the coupon conditions count against coverage separately. ## Filling the gaps ```python import pytest def test_coupon_save20(): assert calculate_price(100, is_member=False, coupon_code='SAVE20') == 80.0 def test_member_and_coupon(): assert calculate_price(100, is_member=True, coupon_code='SAVE20') == pytest.approx(72.0) # float-safe comparison def test_invalid_coupon(): with pytest.raises(ValueError, match='Unknown coupon: INVALID'): calculate_price(100, is_member=False, coupon_code='INVALID') ``` After adding these, coverage reaches 100% for this file. ## HTML report ```bash pytest --cov=src --cov-report=html tests/ # then open htmlcov/index.html in a browser ``` Red lines are uncovered; green lines ran. Click through files to see exactly what was missed. ## .coveragerc: exclude files and lines ```ini # .coveragerc [run] omit = tests/* setup.py [report] exclude_lines = if __name__ == .__main__.: raise NotImplementedError \.\.\. [html] directory = htmlcov ``` ## Enforce a minimum in CI ```bash pytest --cov=src --cov-fail-under=80 tests/ ``` Exits with code 2 if total coverage drops below 80%. CI treats this as a test failure. This catches "someone added code and forgot to write tests" automatically. ## pragma: no cover ```python def _debug_dump(obj): # pragma: no cover '''Dev utility — not meaningful to test.''' import pprint pprint.pprint(vars(obj)) ``` The function won't appear in coverage stats. Use this rarely — only for code that genuinely can't be unit tested (debug utilities, `__main__` blocks, abstract stubs). ## Combine flags for a full CI command ```bash pytest --cov=src --cov-branch --cov-report=term-missing --cov-fail-under=75 tests/ ```

pytest-cov Quick Reference

#
**Install:** ```bash pip install pytest-cov ``` **CLI flags:** ```bash pytest --cov=src # measure src/ package pytest --cov=src --cov-branch # include branch coverage pytest --cov=src --cov-report=term-missing # show missing line numbers pytest --cov=src --cov-report=html # HTML report → htmlcov/ pytest --cov=src --cov-fail-under=80 # fail if coverage < 80% ``` **Combine:** ```bash pytest --cov=src --cov-branch --cov-report=term-missing --cov-fail-under=75 tests/ ``` **.coveragerc:** ```ini [run] omit = tests/* setup.py */migrations/* [report] exclude_lines = if __name__ == .__main__.: raise NotImplementedError [html] directory = htmlcov ``` **Inline exclusion:** ```python def unreachable(): # pragma: no cover ... ``` **Coverage concepts:** | Term | Meaning | |------|---------| | Stmts | Total executable statements | | Miss | Statements never executed | | Branch | Total decision branches (if/else, try/except…) | | BrPart | Branches only partially covered | | Cover % | (Stmts − Miss) / Stmts | **Set persistent options in pytest.ini:** ```ini [pytest] addopts = --cov=src --cov-report=term-missing ```
01

Run Coverage and Read the Report

#

Create src/calculator.py with three functions: add(a, b), divide(a, b) (raises ZeroDivisionError when b == 0), and absolute_value(x). Create tests/test_calc.py with a single test that only tests add(). Run "pytest --cov=src --cov-report=term-missing tests/" and read the output. Identify which functions are not covered and which specific line numbers are listed in the Missing column.

# src/calculator.py

def add(a, b):
    return a + b

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('cannot divide by zero')
    return a / b

def absolute_value(x):
    if x < 0:
        return -x
    return x


# tests/test_calc.py
from src.calculator import add

def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0

# Run: pytest --cov=src --cov-report=term-missing tests/
# What is the coverage percentage?
# Which lines appear in the Missing column?
Solution
# src/calculator.py

def add(a, b):
    return a + b

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('cannot divide by zero')
    return a / b

def absolute_value(x):
    if x < 0:
        return -x
    return x


# tests/test_calc.py
from src.calculator import add

def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0

# Expected report (approximate — exact line numbers depend on blank lines in your file):
# Name               Stmts   Miss  Cover   Missing
# src/calculator.py     <N>   <Miss>  ~40%   <lines in divide() and absolute_value()>
#
# The Missing column will show the body lines of divide() and absolute_value().
# Open the file and verify: the listed lines should be inside those two functions.
# Coverage: ~40% — only add() is exercised.
02

Generate HTML Report and Write a Missing Test

#

Using src/calculator.py from exercise 1: run "pytest --cov=src --cov-report=html tests/" to generate htmlcov/index.html. Open it in a browser (or use --cov-report=term-missing to find the lines). Write tests for divide() and absolute_value() — including the error path and both branches of the if statement. Run coverage again and confirm 100% is reached.

# tests/test_calc.py  (extend from exercise 1)
from src.calculator import add, divide, absolute_value
import pytest


def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0


def test_divide():
    # cover both the normal path and the ZeroDivisionError path
    pass


def test_absolute_value():
    # cover both the negative-input and positive-input branches
    pass

# Run: pytest --cov=src --cov-report=html tests/
# Open htmlcov/index.html — red lines are uncovered
Solution
# tests/test_calc.py
from src.calculator import add, divide, absolute_value
import pytest


def test_add():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0


def test_divide_normal():
    assert divide(10, 2) == 5.0


def test_divide_by_zero():
    with pytest.raises(ZeroDivisionError):
        divide(10, 0)


def test_absolute_value_negative():
    assert absolute_value(-5) == 5


def test_absolute_value_positive():
    assert absolute_value(3) == 3

# After adding these tests:
# src/calculator.py    9    0   100%
# All red lines should be green in the HTML report.
03

Enforce a Coverage Minimum with --cov-fail-under

#

Using the partial test suite from exercise 1 (only test_add): run "pytest --cov=src --cov-fail-under=80 tests/" and observe the exit code — it should be 2, not 0. Note the failure message in the output. Then add the full tests from exercise 2 and run again — it should pass with exit code 0. Add the flag to pytest.ini via addopts to make it permanent.

# Run 1 — with only test_add (expect exit code 2, coverage ~33%):
# pytest --cov=src --cov-fail-under=80 tests/

# Run 2 — after adding all tests (expect exit code 0, coverage 100%):
# pytest --cov=src --cov-fail-under=80 tests/

# To persist the minimum, add to pytest.ini:
# [pytest]
# addopts = --cov=src --cov-report=term-missing --cov-fail-under=80
Solution
# pytest.ini
# [pytest]
# addopts = --cov=src --cov-report=term-missing --cov-fail-under=80

# Run 1 output (partial tests):
# FAIL Required test coverage of 80% not reached. Total coverage: 33.33%
# (exit code 2)

# Run 2 output (full test suite):
# Required test coverage of 80% reached. Total coverage: 100.00%
# (exit code 0)

# addopts applies to every pytest run automatically — both local and CI
# use the same settings without anyone remembering the flags.
04

Create a .coveragerc to Exclude Files

#

Create a .coveragerc file that excludes tests/ and setup.py from measurement, and excludes lines matching "if __name__ == .__main__.:". Create setup.py with a few lines and src/main.py with a run() function and an if __name__ == "__main__": block. Run coverage and confirm those files and lines do not appear in the report.

# .coveragerc  (create at project root)
[run]
omit =
    # add: tests/*
    # add: setup.py

[report]
exclude_lines =
    # add the __main__ guard pattern


# setup.py
from setuptools import setup
setup(name='myproject', version='0.1')


# src/main.py
def run():
    print('running')

if __name__ == '__main__':
    run()

# Run: pytest --cov=src tests/
# setup.py and the __main__ line should not appear in the report
Solution
# .coveragerc
[run]
omit =
    tests/*
    setup.py

[report]
exclude_lines =
    if __name__ == .__main__.:
    raise NotImplementedError

[html]
directory = htmlcov


# setup.py
from setuptools import setup
setup(name='myproject', version='0.1')


# src/main.py
def run():
    print('running')

if __name__ == '__main__':
    run()

# Run: pytest --cov=src tests/
# setup.py will not appear; the __main__ line is excluded from stats
05

Exclude a Function with pragma: no cover

#

Add a debug_print(a, b) function to src/calculator.py that prints both arguments (two print statements). Add "# pragma: no cover" to the function definition line. Run "pytest --cov=src --cov-report=term-missing tests/" with the full test suite from exercise 2. Confirm the debug_print lines do not appear in the report. Then remove the pragma and run again — verify the lines show up as missing.

# src/calculator.py  (add to the existing file)

def add(a, b):
    return a + b

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('cannot divide by zero')
    return a / b

def absolute_value(x):
    if x < 0:
        return -x
    return x

def debug_print(a, b):  # add pragma: no cover here
    print(f'debug: a={a}, b={b}')
    print(f'debug: sum={a + b}')

# Run: pytest --cov=src --cov-report=term-missing tests/
# With pragma: debug_print lines should NOT be listed under Missing
# Without pragma: they appear as uncovered
Solution
# src/calculator.py

def add(a, b):
    return a + b

def divide(a, b):
    if b == 0:
        raise ZeroDivisionError('cannot divide by zero')
    return a / b

def absolute_value(x):
    if x < 0:
        return -x
    return x

def debug_print(a, b):  # pragma: no cover
    print(f'debug: a={a}, b={b}')
    print(f'debug: sum={a + b}')

# Run: pytest --cov=src --cov-report=term-missing tests/
# With pragma: 100% coverage, debug_print not in stats
# Without pragma: coverage drops and lines 17-18 appear as Missing