Python · Testing with pytest · Expert

Large Test Suite Architecture

5 tasks

Organize large test suites for speed, isolation, and maintainability.

Quick topic start and explanations before exercises (exercises below):

Structuring a large test suite

#
A single `tests/` folder with 20 files is manageable. At 200 files across 5 modules, it becomes unclear where to put new tests, which tests are safe to run quickly, and why a previously passing test suddenly fails on a fresh branch. Test architecture is the set of decisions that keep your suite navigable, trustworthy, and fast as it grows. ## The three-layer model Tests fall naturally into three categories based on what they touch: **Unit tests** test one function or class in isolation. They have no I/O — no database, no network, no filesystem. They run in milliseconds. A 500-test unit suite typically completes in under two seconds. **Integration tests** test how components work together: a view + database, a service + cache, a parser + file reader. They set up real infrastructure (a test database, a running server, a temporary directory) and are slower — often 100–500ms per test. **End-to-end tests (e2e)** drive the application from the outside, usually via a browser (Selenium, Playwright) or a real HTTP client against a running server. They are the slowest and most brittle, but catch regressions that unit and integration tests cannot. The standard directory layout reflects this split: ``` tests/ ├── conftest.py # shared fixtures for the whole suite ├── unit/ │ ├── conftest.py # fixtures only available to unit tests │ ├── test_models.py │ ├── test_services.py │ └── test_utils.py ├── integration/ │ ├── conftest.py # fixtures only available to integration tests │ ├── test_api.py │ └── test_db.py └── e2e/ ├── conftest.py └── test_checkout_flow.py ``` ## conftest.py hierarchy pytest looks for `conftest.py` files starting from the test file's directory and walking up to the `rootdir`. A fixture defined in `tests/conftest.py` is available to every test in the suite. A fixture defined in `tests/unit/conftest.py` is available only to tests under `tests/unit/`. This layering lets you avoid a common anti-pattern: one giant `conftest.py` at the root that defines everything. When unit tests import integration fixtures (database sessions, HTTP clients), they inadvertently inherit setup costs — and the line between "fast" and "slow" tests disappears. Rule of thumb: **a fixture should be defined in the highest conftest where it is needed, and no higher.** A database fixture used only by integration tests belongs in `tests/integration/conftest.py`, not in the root. ## Test isolation principles **No shared mutable state.** If test A modifies a global object (a module-level list, a class variable, a singleton) and test B reads that object, test B's result depends on whether test A ran first. This is a hidden ordering dependency. Fixes: use fresh instances per test (function-scope fixtures), or use `monkeypatch` to restore state after each test. **No order dependency.** Your suite must produce the same results whether pytest runs tests alphabetically, in reverse, or in a random order (`pytest-randomly`). If changing order changes results, a test is not actually isolated. Run `pytest --randomly-seed=last` to reproduce a specific order, or `pytest -p randomly --randomly-seed=0` to enable randomisation. **No inter-test communication.** Tests must not pass data to each other through shared variables. If you find yourself writing `test_02_login_after_register`, the setup logic belongs in a fixture, not spread across test functions with numbering. ## Performance strategy The goal is a feedback loop fast enough that developers run the full unit suite before every commit, and run integration tests in CI on every push. Typical targets: - **Unit suite**: under 5 seconds locally, always run. - **Integration suite**: under 2 minutes in CI, run on push. - **E2e suite**: run nightly or on release branches. To run subsets, combine pytest's path targeting and mark filtering: ```bash # Unit tests only (fast) pytest tests/unit/ # Integration tests only pytest tests/integration/ # Everything except slow tests pytest -m "not slow" # Only the tests for a specific module pytest tests/unit/test_services.py ``` ## Registering custom marks pytest warns when a test uses an unregistered mark. Register marks in `pytest.ini`: ```ini [pytest] markers = unit: fast tests with no I/O integration: tests that use the database or network slow: tests that take more than 1 second smoke: a minimal subset run before deployment ``` `--strict-markers` (also in `addopts`) causes pytest to fail immediately if a test uses an unregistered mark — catching typos like `@pytest.mark.integraion` before they silently become no-ops. ## Common anti-patterns **God conftest.** One `conftest.py` with 50 fixtures, imported implicitly by every test. Hard to navigate, hides coupling between test layers. **Tests that clean up after themselves (instead of using fixtures).** If setup runs in the test body and teardown is in a `try/finally`, the fixture system is being bypassed. When pytest's fixture teardown runs is predictable and correct; manual `try/finally` inside tests is error-prone and hides intent. **Fixture overuse.** Not everything needs a fixture. A simple `user = User(name="Alice")` in the test body is clearer than a `alice_user` fixture that does the same thing and is referenced from 15 tests. Use fixtures for setup that is genuinely reused or involves teardown. **Implicitly ordered tests.** Naming tests `test_01_`, `test_02_` to control order is a smell. If order matters, the tests share state they should not. Extract the shared state into a fixture or a helper function called by each test independently.

Test architecture in a mid-size Django project

#
This walkthrough shows what a well-structured test suite looks like for a Django e-commerce project: a product catalogue, user accounts, an orders service, and a public API. The goal is a suite where you can run fast feedback in 3 seconds and the full suite in CI in under 90 seconds. ## Directory tree ``` my_shop/ ├── src/ │ ├── catalogue/ │ ├── orders/ │ └── accounts/ ├── tests/ │ ├── conftest.py │ ├── unit/ │ │ ├── conftest.py │ │ ├── catalogue/ │ │ │ ├── test_models.py │ │ │ └── test_services.py │ │ ├── orders/ │ │ │ ├── test_pricing.py │ │ │ └── test_validators.py │ │ └── accounts/ │ │ └── test_password_policy.py │ ├── integration/ │ │ ├── conftest.py │ │ ├── test_order_flow.py │ │ ├── test_api_products.py │ │ └── test_api_checkout.py │ └── e2e/ │ ├── conftest.py │ └── test_checkout_browser.py ├── pytest.ini └── Makefile ``` The source tree mirrors the test tree under `tests/unit/`. This makes it easy to find the tests for a module: `catalogue/models.py` → `tests/unit/catalogue/test_models.py`. ## Root conftest.py `tests/conftest.py` holds only what every test needs: ```python # tests/conftest.py import pytest # django_db_setup is provided by pytest-django automatically. # Override it only if you need custom DB creation logic (rarely needed). @pytest.fixture def settings_override(settings): # Override settings safely -- changes revert after each test. settings.EMAIL_BACKEND = 'django.core.mail.backends.locmem.EmailBackend' return settings ``` Keep this file small. A rule of thumb: if you add a fixture here and 90% of tests never use it, move it down to the appropriate subdirectory conftest. ## Unit conftest.py ```python # tests/unit/conftest.py import pytest from catalogue.models import Product from accounts.models import User @pytest.fixture def product(): # An in-memory Product -- no database needed. return Product(name="Widget", price=9_99, stock=10) @pytest.fixture def user(): return User(email="[email protected]", is_active=True) ``` These fixtures build objects in memory without touching the database. They run at function scope (the default), so each test gets a fresh object. ## Integration conftest.py ```python # tests/integration/conftest.py import pytest @pytest.fixture(scope='session') def api_base_url(live_server): # Full URL of the Django test server. return live_server.url @pytest.fixture def auth_client(client, django_user_model): # A logged-in Django test client. user = django_user_model.objects.create_user( username='testuser', password='testpass' ) client.force_login(user) return client @pytest.fixture def seeded_catalogue(db): # Three products in the database. from catalogue.models import Product Product.objects.bulk_create([ Product(name='Widget', price=999, stock=10), Product(name='Gadget', price=1999, stock=5), Product(name='Doohickey', price=499, stock=0), ]) ``` The `db` fixture (from pytest-django) marks these tests as database tests. Notice the `scope='session'` on `api_base_url` — starting the live server is expensive, so it is shared across the entire test session. ## Marks and pytest.ini ```ini [pytest] addopts = -v --tb=short --strict-markers testpaths = tests markers = unit: fast tests with no I/O integration: tests that use the database or network slow: tests that take more than 1 second smoke: minimal subset run before deployment ``` Apply marks at the module level to avoid repeating them on every function: ```python # tests/integration/test_order_flow.py import pytest pytestmark = [pytest.mark.integration] @pytest.mark.django_db def test_order_created_on_checkout(auth_client, seeded_catalogue): resp = auth_client.post('/api/orders/', {'product_id': 1, 'quantity': 2}) assert resp.status_code == 201 @pytest.mark.django_db @pytest.mark.slow def test_order_email_sent(auth_client, seeded_catalogue, mailoutbox): auth_client.post('/api/orders/', {'product_id': 1, 'quantity': 1}) assert len(mailoutbox) == 1 assert 'Order confirmation' in mailoutbox[0].subject ``` `pytestmark` is a module-level list. Every test in the file inherits all marks in the list. Adding `pytest.mark.slow` to individual heavy tests lets you exclude them from the fast integration pass. ## Makefile targets ```makefile .PHONY: test test-unit test-integration test-fast test-ci test-unit: pytest tests/unit/ -q test-integration: pytest tests/integration/ -q test-fast: pytest tests/unit/ tests/integration/ -m "not slow" -q test: pytest -q test-ci: pytest --cov=src --cov-report=term-missing --cov-fail-under=80 -q ``` `-q` (quiet) suppresses individual test names and prints only the summary — useful for fast feedback loops. ## Spotting and fixing a test ordering dependency Here is a real pattern that creates ordering dependencies: ```python # BROKEN — tests share state through a module-level list # (Product and client are imported from elsewhere; focus is on _created_ids) _created_ids = [] def test_create_product(): product = Product.objects.create(name='Widget', price=999) _created_ids.append(product.id) def test_created_product_appears_in_list(): # Only passes if test_create_product ran first resp = client.get('/api/products/') assert any(p['id'] in _created_ids for p in resp.json()) ``` The fix is to move setup into a fixture, making each test self-contained: ```python # FIXED — each test is independent @pytest.fixture def widget(db): return Product.objects.create(name='Widget', price=999) @pytest.mark.django_db def test_create_product(db): product = Product.objects.create(name='Widget', price=999) assert product.id is not None @pytest.mark.django_db def test_created_product_appears_in_list(widget, client): resp = client.get('/api/products/') ids = [p['id'] for p in resp.json()] assert widget.id in ids ``` Now each test sets up what it needs and does not depend on another test having run. The database is rolled back between tests by pytest-django's transactional fixtures. To detect ordering dependencies proactively, install `pytest-randomly` and run: ```bash pip install pytest-randomly pytest --randomly-seed=random ``` If your suite fails with one seed but passes with another, you have a hidden ordering dependency.

Test architecture reference card

#
## Standard directory layout ``` tests/ ├── conftest.py ← fixtures for all tests ├── unit/ │ ├── conftest.py ← fixtures for unit tests only │ └── module_name/ │ └── test_*.py ├── integration/ │ ├── conftest.py ← fixtures for integration tests only │ └── test_*.py └── e2e/ ├── conftest.py └── test_*.py ``` ## conftest.py scope rules | Where defined | Available to | |---|---| | `tests/conftest.py` | All tests | | `tests/unit/conftest.py` | Tests under `tests/unit/` only | | `tests/integration/conftest.py` | Tests under `tests/integration/` only | Fixtures defined lower in the tree **override** same-named fixtures defined higher up. ## pytest.ini marks setup ```ini [pytest] addopts = -v --tb=short --strict-markers testpaths = tests markers = unit: fast tests, no I/O integration: uses database or network slow: takes more than 1 second smoke: minimal pre-deploy check ``` ## Running subsets ```bash pytest tests/unit/ # unit suite only pytest tests/integration/ # integration suite only pytest -m "not slow" # exclude slow tests pytest -m "smoke" # smoke tests only pytest -m "integration and not slow" # combine marks pytest tests/unit/test_models.py # single file pytest tests/unit/test_models.py::test_price_is_positive # single test ``` ## Module-level mark ```python # Applies to every test in the file pytestmark = [pytest.mark.integration, pytest.mark.django_db] ``` ## Detecting ordering dependencies ```bash pip install pytest-randomly pytest --randomly-seed=random # random order each run pytest --randomly-seed=last # reproduce last order pytest -p no:randomly # disable randomisation ``` ## Makefile targets ```makefile test-unit: pytest tests/unit/ -q test-integration: pytest tests/integration/ -q test-fast: pytest tests/unit/ tests/integration/ -m "not slow" -q test-ci: pytest --cov=src --cov-fail-under=80 -q ``` ## Common anti-patterns | Anti-pattern | Symptom | Fix | |---|---|---| | God conftest | Root conftest has 50+ fixtures | Move fixtures to subdirectory conftest | | Numbered tests | `test_01_`, `test_02_` | Extract shared setup into a fixture | | Shared mutable state | Tests fail in random order | Use function-scope fixtures, not module globals | | Fixture overuse | `alice_user` fixture used in 1 test | Inline the object creation in the test body | | Mixed layers | Unit tests import `db` fixture | Keep unit conftest free of I/O fixtures | | Manual teardown in test body | `try/finally` inside test functions | Use `yield` fixtures — they run even on failure |

Exercises:

01

Reorganize a flat test directory into unit/ and integration/

#

You have a flat `tests/` directory with these files: ``` tests/ ├── test_models.py ← tests Product and Order models in memory (no DB) ├── test_services.py ← tests pure calculation functions (no DB) ├── test_api.py ← tests API endpoints with a real database └── test_db_queries.py ← tests database query methods (needs DB) ``` Reorganize this into the following structure. Write the new file paths and a `pytest.ini` that sets `testpaths = tests` so pytest finds tests automatically. Target structure: ``` tests/ ├── conftest.py ← empty for now (just create the file) ├── unit/ │ ├── conftest.py ← empty for now │ ├── test_models.py │ └── test_services.py └── integration/ ├── conftest.py ← empty for now ├── test_api.py └── test_db_queries.py ``` Also: what command runs only the integration tests after the reorganization? 

# List the new file paths (one per line):
# tests/conftest.py
# ...


# pytest.ini content:
# [pytest]
# testpaths = ...


# Command to run only integration tests:
# pytest ...
Solution 
# New file paths after reorganization:
# tests/conftest.py
# tests/unit/conftest.py
# tests/unit/test_models.py
# tests/unit/test_services.py
# tests/integration/conftest.py
# tests/integration/test_api.py
# tests/integration/test_db_queries.py

# pytest.ini (create this file at the project root):
# [pytest]
# testpaths = tests

# Command to run only integration tests:
# pytest tests/integration/
02

Register marks and apply them to tests

#

You have this `pytest.ini` and two test files. Your task is to: 1. Add `addopts = --strict-markers` to `pytest.ini` 2. Register three marks: `unit`, `integration`, `slow` 3. Apply marks to the test files using `pytestmark` (module-level) ```ini # pytest.ini — current state [pytest] testpaths = tests ``` ```python # tests/unit/test_pricing.py def test_discount_applied(): assert apply_discount(100, 0.1) == 90 def test_negative_discount_raises(): with pytest.raises(ValueError): apply_discount(100, -0.1) ``` ```python # tests/integration/test_orders.py def test_order_saved_to_db(db): order = Order.objects.create(total=100) assert Order.objects.count() == 1 def test_large_order_sends_email(db, mailoutbox): # takes 2 seconds Order.objects.create(total=10000) assert len(mailoutbox) == 1 ``` Write the updated `pytest.ini` and both test files with `pytestmark` added. Mark `test_large_order_sends_email` as both `integration` and `slow`. 

# pytest.ini
[pytest]
testpaths = tests
# add addopts and markers here


# tests/unit/test_pricing.py
import pytest
# add pytestmark here

def test_discount_applied():
    assert apply_discount(100, 0.1) == 90

def test_negative_discount_raises():
    with pytest.raises(ValueError):
        apply_discount(100, -0.1)


# tests/integration/test_orders.py
import pytest
# add pytestmark here

def test_order_saved_to_db(db):
    order = Order.objects.create(total=100)
    assert Order.objects.count() == 1

def test_large_order_sends_email(db, mailoutbox):
    Order.objects.create(total=10000)
    assert len(mailoutbox) == 1
Solution 
# pytest.ini
[pytest]
testpaths = tests
addopts = --strict-markers
markers =
    unit: fast tests with no I/O
    integration: tests that use the database or network
    slow: tests that take more than 1 second


# tests/unit/test_pricing.py
import pytest

pytestmark = [pytest.mark.unit]

def test_discount_applied():
    assert apply_discount(100, 0.1) == 90

def test_negative_discount_raises():
    with pytest.raises(ValueError):
        apply_discount(100, -0.1)


# tests/integration/test_orders.py
import pytest

pytestmark = [pytest.mark.integration]

def test_order_saved_to_db(db):
    order = Order.objects.create(total=100)
    assert Order.objects.count() == 1

@pytest.mark.slow
def test_large_order_sends_email(db, mailoutbox):
    Order.objects.create(total=10000)
    assert len(mailoutbox) == 1
03

Build a conftest.py hierarchy with scoped fixtures

#

Create three `conftest.py` files for a test suite with this layout: ``` tests/ ├── conftest.py ├── unit/ │ └── conftest.py └── integration/ └── conftest.py ``` Requirements: - `tests/conftest.py`: define a **session-scoped** fixture `app_config` that returns a dict `{"env": "test", "debug": False}`. This fixture should be available to all tests. - `tests/unit/conftest.py`: define a **function-scoped** fixture `calculator` that returns a new `Calculator()` instance. Available only to unit tests. - `tests/integration/conftest.py`: define a **function-scoped** fixture `db_session` that prints `"opening db"` before yielding the string `"db_connection"` and prints `"closing db"` after. Available only to integration tests. Also write a test function in `tests/unit/test_calc.py` that uses both `app_config` and `calculator`, and a test function in `tests/integration/test_db.py` that uses both `app_config` and `db_session`. 

# tests/conftest.py
import pytest

# session-scoped app_config fixture here


# tests/unit/conftest.py
import pytest

# function-scoped calculator fixture here


# tests/integration/conftest.py
import pytest

# function-scoped db_session fixture here


# tests/unit/test_calc.py
# use app_config and calculator


# tests/integration/test_db.py
# use app_config and db_session
Solution 
# tests/conftest.py
import pytest

@pytest.fixture(scope='session')
def app_config():
    return {"env": "test", "debug": False}


# tests/unit/conftest.py
import pytest

class Calculator:
    def add(self, a, b):
        return a + b

@pytest.fixture
def calculator():
    return Calculator()


# tests/integration/conftest.py
import pytest

@pytest.fixture
def db_session():
    print("opening db")
    yield "db_connection"
    print("closing db")


# tests/unit/test_calc.py
def test_add(app_config, calculator):
    assert app_config["env"] == "test"
    assert calculator.add(2, 3) == 5


# tests/integration/test_db.py
def test_connect(app_config, db_session):
    assert app_config["env"] == "test"
    assert db_session == "db_connection"
04

Write a Makefile with test targets

#

Write a `Makefile` for a project with this test structure: ``` tests/ ├── unit/ └── integration/ ``` And this `pytest.ini`: ```ini [pytest] testpaths = tests addopts = --strict-markers markers = unit: fast tests integration: database tests slow: tests over 1 second ``` The Makefile must have these four targets: | Target | What it runs | |---|---| | `make test-unit` | Unit tests only, quiet output | | `make test-integration` | Integration tests only, quiet output | | `make test-fast` | Unit + integration, excluding slow tests, quiet output | | `make test-ci` | Full suite with coverage (src/), fail under 80%, quiet output | All targets should be declared as `.PHONY`. 

# Makefile
.PHONY: ...

test-unit:
	...

test-integration:
	...

test-fast:
	...

test-ci:
	...
Solution 
# Makefile
.PHONY: test-unit test-integration test-fast test-ci

test-unit:
	pytest tests/unit/ -q

test-integration:
	pytest tests/integration/ -q

test-fast:
	pytest tests/unit/ tests/integration/ -m "not slow" -q

test-ci:
	pytest --cov=src --cov-report=term-missing --cov-fail-under=80 -q
05

Find and fix a test ordering dependency

#

The following test file passes when tests run in the default order, but fails when tests run in a different order. Imagine running them in reverse: ``` test_empty_cart_total → test_remove_item_from_cart → test_cart_total → test_add_item_to_cart ``` Find the ordering dependency and fix it so the tests pass in any order. **Tip:** Install `pytest-randomly` and run `pytest --randomly-seed=0` to verify your fix works regardless of order. ```python # test_cart.py import pytest _cart = [] # shared module-level state def test_add_item_to_cart(): _cart.append({"id": 1, "name": "Widget", "qty": 2}) assert len(_cart) == 1 def test_cart_total(): total = sum(item["qty"] * 10 for item in _cart) assert total == 20 # 2 × £10 def test_remove_item_from_cart(): _cart.clear() assert len(_cart) == 0 def test_empty_cart_total(): total = sum(item["qty"] * 10 for item in _cart) assert total == 0 ``` **Part 1 — Identify:** Which test fails first when run in reverse order? Why? **Part 2 — Fix:** Rewrite the file so all four tests are independent. Use a pytest fixture instead of the module-level `_cart`. 

# Part 1 — Which test fails first in reverse order, and why?
# (write your answer as a comment)
#
# Test that fails first: ...
# Reason: ...


# Part 2 — Fixed test_cart.py
import pytest

# define a cart fixture here


def test_add_item_to_cart(cart):
    pass  # implement


def test_cart_total(cart):
    pass  # implement


def test_remove_item_from_cart(cart):
    pass  # implement


def test_empty_cart_total(cart):
    pass  # implement
Solution 
# Part 1 — Which test fails first in reverse order?
#
# Test that fails first: test_cart_total
# Reason: In reverse order the execution is:
#   test_empty_cart_total → test_remove_item_from_cart → test_cart_total → test_add_item_to_cart
# test_empty_cart_total runs first; _cart is already empty, total == 0 — accidentally PASSES.
# test_remove_item_from_cart clears an already-empty list — PASSES.
# test_cart_total runs next; _cart is still empty, so total == 0, not 20 — FAIL.
#
# The root cause: _cart is a module-level list shared across all tests.
# Tests communicate through it implicitly, so results depend on run order.


# Part 2 — Fixed test_cart.py
import pytest


@pytest.fixture
def cart():
    return []


def test_add_item_to_cart(cart):
    cart.append({"id": 1, "name": "Widget", "qty": 2})
    assert len(cart) == 1


def test_cart_total(cart):
    cart.append({"id": 1, "name": "Widget", "qty": 2})
    total = sum(item["qty"] * 10 for item in cart)
    assert total == 20


def test_remove_item_from_cart(cart):
    cart.append({"id": 1, "name": "Widget", "qty": 2})
    cart.clear()
    assert len(cart) == 0


def test_empty_cart_total(cart):
    total = sum(item["qty"] * 10 for item in cart)
    assert total == 0