Skip to main content

Code Coverage Gates in GitHub Actions for Python

Code coverage measures the percentage of your codebase executed by tests. A coverage gate is an automated check that blocks merging pull requests if test coverage falls below a threshold (e.g., 80%). This prevents shipping untested code while encouraging developers to write meaningful tests.

Coverage gates work by running pytest-cov to generate coverage reports, uploading them to a service like Codecov, and failing the workflow if coverage is insufficient. GitHub shows the gate status as a required check on pull requests.

Running pytest-cov for Local Coverage​

Before configuring CI, measure coverage locally using pytest-cov:

pip install pytest-cov
pytest --cov=src --cov-report=html --cov-report=term-missing tests/

The --cov=src flag measures coverage for the src/ directory. --cov-report=html generates an HTML report in htmlcov/, and --cov-report=term-missing prints coverage to the terminal, highlighting uncovered lines.

The output shows coverage percentage (lines executed divided by lines in code) and which lines weren't executed, helping you identify untested code.

Configuring Coverage in Your Workflow​

Add pytest-cov to your workflow and generate XML reports for upload:

name: Coverage Gate

on: [push, pull_request]

jobs:
  coverage:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with:
        python-version: '3.11'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
        pip install pytest-cov
    - name: Run tests with coverage
      run: pytest tests/ --cov=src --cov-report=xml --cov-report=term-missing
    - name: Check coverage threshold
      run: |
        COVERAGE=$(coverage report | grep TOTAL | awk '{print $4}' | sed 's/%//')
        echo "Coverage: $COVERAGE%"
        if (( $(echo "$COVERAGE < 80" | bc -l) )); then
          echo "Coverage is below 80%"
          exit 1
        fi

The --cov-report=xml flag generates coverage.xml for upload. The coverage check script extracts the coverage percentage and fails the job if it's below 80%.

Using Codecov for Coverage Reports​

Codecov is a hosted service that tracks coverage history and integrates with GitHub. It provides better UX than raw CI checks: visual diffs, per-commit tracking, and pull request comments.

Upload coverage to Codecov in your workflow:

- name: Run tests with coverage
  run: pytest tests/ --cov=src --cov-report=xml
- name: Upload to Codecov
  uses: codecov/codecov-action@v3
  with:
    file: ./coverage.xml
    flags: unittests
    name: codecov-umbrella
    fail_ci_if_error: true

The fail_ci_if_error: true flag causes the upload step to fail if Codecov is unreachable or rejects the report (though this is rare).

Codecov automatically compares the current PR's coverage to the base branch's coverage. If coverage drops (e.g., new untested code), Codecov posts a comment on the pull request showing the delta. If coverage meets your threshold, the check passes.

Setting Coverage Requirements in GitHub​

Configure coverage requirements as a branch protection rule:

  1. Go to Settings > Branches > Branch protection rules > main
  2. Under Require status checks to pass before merging, enable Codecov (or whatever coverage service you use)
  3. Optionally, set Require branches to be up to date before merging

Now, pull requests cannot merge until coverage checks pass.

Handling Uncovered Code​

Mark code that shouldn't be covered (debug utilities, platform-specific code) with pragma comments:

def debug_print_internal_state():  # pragma: no cover
    """Print internal state for debugging. Not tested in CI."""
    print(internal_state)

if sys.platform == "win32":  # pragma: no cover
    # Windows-specific setup; tested on Windows runners, ignored here.
    setup_windows()

The # pragma: no cover comment tells coverage tools to ignore that line. Use sparingly; every pragma reduces overall coverage and should be justified.

Coverage for Different Test Levels​

Unit tests typically achieve 80–90% coverage. Integration tests add another 5–10%. Full end-to-end coverage is rarely practical. Configure your coverage threshold to match your test strategy:

- name: Run unit tests
  run: pytest tests/unit --cov=src --cov-report=xml:coverage-unit.xml
- name: Run integration tests
  run: pytest tests/integration --cov=src --cov-append --cov-report=xml
- name: Check coverage
  run: coverage report --fail-under=80

The --cov-append flag adds integration test coverage to the unit test coverage. --fail-under=80 fails if total coverage is below 80%.

Excluded Files and Branches​

Coverage gates on main are strict (e.g., 85% minimum). On feature branches, use lower thresholds or exclude certain files:

- name: Check coverage (feature branch)
  if: github.ref != 'refs/heads/main'
  run: coverage report --fail-under=70
- name: Check coverage (main branch)
  if: github.ref == 'refs/heads/main'
  run: coverage report --fail-under=85

This enforces stricter coverage on the main branch while allowing feature work to proceed at lower coverage.

Key Takeaways​

  • Use pytest-cov to measure test coverage; --cov=src measures the source directory.
  • Generate coverage.xml for upload; --cov-report=term-missing shows uncovered lines.
  • Upload coverage to Codecov for history tracking and pull request comments.
  • Set coverage gates as branch protection rules to block merges below thresholds.
  • Use # pragma: no cover to exclude debug code, platform-specific code, or untestable code.
  • Enforce stricter thresholds on main branch; allow lower coverage on feature branches.

Frequently Asked Questions​

What is a reasonable code coverage threshold?​

80% is a practical standard for most projects. It balances test effort against bug prevention. Below 60%, you risk significant untested code. Above 95%, diminishing returns; tests become brittle and cover edge cases that rarely occur in production.

Why does my coverage percentage drop when I add a new feature?​

New features start with zero coverage. When you add 100 lines of untested code, overall coverage drops. Write tests for the feature to bring coverage back up. This is by design—the metric encourages test-driven development.

How do I exclude generated code from coverage?​

Use .coveragerc (for coverage.py):

[run]
omit =
    */migrations/*
    */generated/*
    setup.py

[report]
exclude_lines =
    pragma: no cover
    def __repr__
    if __name__ == .__main__.:

This excludes migration files, generated code, and common non-critical code from coverage.

Can I have different coverage thresholds for different modules?​

Yes, with more complex .coveragerc configuration. However, this is often a sign of uneven test investment. Consider raising overall coverage instead of creating exceptions.

Does coverage guarantee bug-free code?​

No. High coverage means you're testing the code paths, but the tests themselves might be wrong (wrong assertions) or incomplete (missing edge cases). Coverage is a metric of test investment, not correctness.

Further Reading​