When tests only run on a developer's laptop, they stop doing their real job: catching regressions before code reaches production. Continuous Integration (CI) is the practice of running your full test suite automatically on every push or pull request, on a clean machine that knows nothing about your local setup. The two tools you will encounter most often are **GitHub Actions** (built into GitHub, free for public repos) and **tox** (a Python-specific test automation tool).
## Why CI is not just "pytest on a server"
Running pytest in CI introduces constraints that do not exist locally:
- **No pre-installed packages.** The CI runner starts from a base OS image. Your workflow must install every dependency.
- **No `.env` files.** Secrets and environment variables must be provided through the CI system's secrets store.
- **Non-zero exit code matters.** pytest exits with code `1` when any test fails. CI platforms treat a non-zero exit as pipeline failure — the pull request cannot be merged.
- **Reproducibility.** A test that passes locally but fails in CI almost always means the test depends on something in your environment (a globally installed package, a local file, a running service).
## GitHub Actions basics
GitHub Actions runs workflows defined as YAML files in `.github/workflows/`. A workflow triggers on events (push, pull request, schedule) and runs a sequence of steps inside a runner — a temporary virtual machine.
The minimal structure for a pytest workflow:
```yaml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install -r requirements.txt
- name: Run tests
run: pytest
```
Each `uses:` line pulls in a pre-built action. `actions/checkout@v4` clones your repository into the runner. `actions/setup-python@v5` installs the requested Python version and adds it to `PATH`.
## Caching pip downloads
Installing packages from scratch on every run is slow. GitHub Actions lets you cache the pip download cache between runs:
```yaml
- name: Cache pip
uses: actions/cache@v4
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
```
The `key` includes a hash of `requirements.txt`, so the cache is invalidated when dependencies change. `restore-keys` provides a fallback that matches the OS even when the exact key misses — it restores a partial cache that is still faster than starting from nothing.
## Failing the pipeline on coverage drop
Coverage measures what percentage of your code is executed by tests. Adding `--cov` to pytest (with `pytest-cov` installed) produces a coverage report. Adding `--cov-fail-under=80` causes pytest to exit with code `2` if coverage drops below 80%, which fails the CI pipeline:
```yaml
- name: Run tests with coverage
run: pytest --cov=src --cov-fail-under=80 --cov-report=term-missing --cov-report=xml
```
`--cov-report=xml` writes `coverage.xml` in Cobertura format — many CI integrations and PR review tools can parse this file to show coverage diffs inline. If you do not need the XML file, you can omit this flag.
`--cov=src` tells pytest-cov which directory to measure (your source code, not the tests themselves). `--cov-report=term-missing` prints a table showing which lines are not covered.
The threshold is a policy decision. 80% is a common starting point; 100% is often impractical and counterproductive (you end up writing tests that test nothing meaningful just to hit the number).
## Testing across Python versions with tox
`tox` is a tool that creates isolated virtual environments and runs your test suite inside each one. It is most useful for library authors who need to verify compatibility with multiple Python versions, but it is also common in application projects that need to support different environments.
A minimal `tox.ini`:
```ini
[tox]
envlist = py311, py312
[testenv]
deps = -r requirements.txt
commands = pytest {posargs}
```
Running `tox` locally creates `.tox/py311/` and `.tox/py312/` virtual environments, installs dependencies in each, and runs `pytest` inside each. `{posargs}` passes any arguments you add after `tox --` directly to pytest (`tox -- -k test_login`).
In GitHub Actions, you can use a **matrix** to run tox across versions:
```yaml
strategy:
matrix:
python-version: ["3.11", "3.12"]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- run: pip install tox
- run: tox -e py${{ matrix.python-version | replace('.', '') }}
```
This runs two parallel jobs — one for each version — and marks the workflow as failed if either fails.
## Configuring pytest for CI
You can keep CI-specific pytest defaults in `pytest.ini` (or `pyproject.toml`) so every run — local and CI — uses the same settings:
```ini
[pytest]
addopts = -v --tb=short
testpaths = tests
```
`--tb=short` gives enough traceback to diagnose failures without filling the log with noise. `-v` (verbose) prints each test name as it runs, which helps when reading CI logs to see which test failed.
## Test result reports
Some CI systems and GitHub integrations can render test results as a structured report rather than raw terminal output. pytest can produce a JUnit XML file:
```
pytest --junitxml=reports/test-results.xml
```
GitHub Actions can then upload this as an artifact:
```yaml
- name: Upload test results
uses: actions/upload-artifact@v4
with:
name: test-results
path: reports/test-results.xml
```
This does not change whether the pipeline passes or fails — it just makes the results accessible for download after the run, useful when debugging intermittent failures.
Create a `.github/workflows/tests.yml` file for a project with this structure:
```
my_project/
├── src/
│ └── app.py
├── tests/
│ └── test_app.py
└── requirements-dev.txt # contains: pytest
```
Requirements for the workflow:
1. Trigger on `push` and `pull_request` to the `main` branch
2. Run on `ubuntu-latest` with Python 3.12
3. Check out the code
4. Install dependencies from `requirements-dev.txt`
5. Run `pytest`
Write the complete YAML content for the workflow file.
# Write the content of .github/workflows/tests.yml below.
# This is a YAML file, not Python — fill in each section.
# name: ...
# on:
# ...
# jobs:
# test:
# runs-on: ...
# steps:
# - ...
You have this GitHub Actions workflow:
```yaml
name: Tests
on:
push:
branches: [main]
pull_request:
branches: [main]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install dependencies
run: pip install -r requirements-dev.txt
- name: Run tests
run: pytest
```
And `requirements-dev.txt`:
```
pytest
```
Make two changes:
1. Add `pytest-cov` to `requirements-dev.txt`
2. Update the `Run tests` step to:
- Measure coverage for the `src/` directory
- Print uncovered lines to terminal
- Fail the pipeline if coverage drops below 80%
Write the updated `requirements-dev.txt` and the updated `Run tests` step.
# requirements-dev.txt
pytest
# add pytest-cov here
# Updated 'Run tests' step (just the step, not the full workflow):
#
# - name: Run tests
# run: pytest ...
Solution
# requirements-dev.txt
pytest
pytest-cov
# Updated step in .github/workflows/tests.yml:
#
# - name: Run tests with coverage
# run: pytest --cov=src --cov-report=term-missing --cov-fail-under=80
Write a `tox.ini` for a project where:
- Tests should run on Python 3.11 and 3.12
- Dependencies are in `requirements.txt` and `requirements-dev.txt`
- The test command is `pytest` with any arguments the user passes through
Also write the command to:
1. Run tox for all environments
2. Run tox for Python 3.12 only
3. Pass `-v` to pytest through tox
# tox.ini
[tox]
envlist = ...
[testenv]
deps =
...
commands =
...
# Commands (write as comments):
# 1. Run all environments:
# tox ...
#
# 2. Run Python 3.12 only:
# tox ...
#
# 3. Pass -v to pytest:
# tox ...
You have this workflow step that runs tests with coverage:
```yaml
- name: Run tests with coverage
run: pytest --cov=src --cov-report=term-missing --cov-fail-under=80 --cov-report=xml
```
Add a step after it that:
1. Uploads `coverage.xml` as a GitHub Actions artifact named `coverage-report`
2. Runs even if the previous step fails (e.g., when the coverage threshold is not met)
Write only the new step in YAML.
# Write the two steps to add after the 'Run tests with coverage' step:
# - name: Upload coverage report
# uses: ...
# with:
# ...
# if: ...
A team has a `pytest.ini` that is empty and a CI workflow where the test step is:
```yaml
- name: Run tests
run: pytest -v --tb=short --strict-markers tests/
```
The problem: developers running `pytest` locally get different output than CI (no `-v`, full tracebacks), and the `tests/` path has to be repeated in both the workflow file and any local scripts.
Fix this by writing a `pytest.ini` that:
1. Sets `addopts` so that `-v --tb=short --strict-markers` are always applied
2. Sets `testpaths` so pytest always looks in `tests/` by default
3. Registers three custom marks to avoid `PytestUnknownMarkWarning`: `unit`, `integration`, `slow`
After this change, the CI step should just be `pytest` with no extra arguments.
Write the complete `pytest.ini` and the simplified CI step.
# pytest.ini
[pytest]
addopts = ...
testpaths = ...
markers =
...
# Simplified CI step (write as a comment):
# - name: Run tests
# run: ...
Solution
# pytest.ini
[pytest]
addopts = -v --tb=short --strict-markers
testpaths = tests
markers =
unit: fast unit tests with no I/O
integration: tests that hit the database or network
slow: tests that take more than 1 second
# Simplified CI step:
# - name: Run tests
# run: pytest
No split tab
Cookie preferences
We use necessary cookies to run the site. With your permission, we can also save your site preferences and use analytics and advertising cookies to understand usage and support the project.
Open tools in tabs.Exercises, IDE tools, and trainers stay available as site tabs.
Switch without losing context.Move between explanations, code, and utilities while keeping your place.
Use the sidebar as your map.The left panels hold navigation, settings, files, libraries, and tool controls.
PythonJavaScriptSQLite
One IDE, three practical modes
Python in the browser.Run small scripts, try libraries, and practice API requests without installing anything.
JavaScript for quick experiments.Test browser-friendly code and compare ideas next to your learning materials.
SQLite for data practice.Open the database explorer to inspect tables, write queries, and learn SQL workflows locally.
TopicIDE
Work side by side with split tabs
Keep instructions visible.Open an exercise or reference page beside the IDE instead of jumping back and forth.
Compare tools while you learn.Place regex checks, explanations, and code experiments next to each other when the task needs it.
Close the split when you are done.The workspace returns to a single focused tab, and your open site tabs remain available.
Code Typing Trainer
Or plain text
This trainer is designed for a physical keyboard.Open this section on a laptop or desktop with a wide screen. Touch typing practice will not work correctly on a phone.
Speed: 0 chars/min
0 words/min
Best speed (60s): 0 chars/min
0 words/min
Errors: 0
Total time: 0.0 s
To practice touch typing, avoid looking at your physical keyboard.