Skip to main content

Matrix Builds for Python: Test Multiple Versions

A matrix strategy in GitHub Actions runs the same workflow across multiple configurations—Python versions, operating systems, or dependency versions—in parallel. This catches version-specific bugs (e.g., deprecated APIs in Python 3.13) and OS-specific issues without duplicating workflow code.

Testing against Python 3.9, 3.11, and 3.13 simultaneously ensures your package works for users on any of those versions. If a test fails only on Python 3.13, the matrix immediately highlights the issue.

Defining a Matrix Strategy​

A matrix is a YAML dictionary in your job that generates multiple job runs from a single definition. Here's a basic example:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['3.9', '3.11', '3.13']
    steps:
    - uses: actions/checkout@v4
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run tests
      run: pytest tests/

The strategy.matrix.python-version array defines three configurations. GitHub Actions spawns three jobs automatically, one for each version. The ${{ matrix.python-version }} syntax accesses the current version in each job.

When you push this workflow, GitHub creates three parallel jobs: test (3.9), test (3.11), and test (3.13). All run simultaneously, completing in roughly the time of a single job (plus overhead).

Multi-Dimensional Matrices​

Combine multiple variables to test across OS, Python version, and dependency versions:

strategy:
  matrix:
    os: [ubuntu-latest, macos-latest, windows-latest]
    python-version: ['3.9', '3.11', '3.13']
    include:
      - os: ubuntu-latest
        python-version: '3.13-dev'

This creates 3 × 3 = 9 jobs automatically. The include: keyword adds custom configurations that don't fit the matrix (e.g., testing a development Python version only on Linux).

Use exclude: to skip certain combinations:

exclude:
  - os: windows-latest
    python-version: '3.9'

This skips testing Python 3.9 on Windows (perhaps due to a known compatibility issue).

Environment Variables and Conditional Steps​

Access matrix values in steps and conditionals:

- name: Run tests (Python ${{ matrix.python-version }} on ${{ runner.os }})
  run: pytest tests/ -v
- name: Upload coverage only for Python 3.11 on Ubuntu
  if: |
    matrix.python-version == '3.11' &&
    runner.os == 'Linux'
  run: bash <(curl -s https://codecov.io/bash)

The if: conditional runs a step only for specific matrix configurations. This avoids uploading coverage reports from every job, focusing on a single canonical version.

Failing Fast with fail-fast​

By default, if one job in a matrix fails, other jobs continue running. Set fail-fast: false to run all jobs even if one fails, useful for detecting multiple issues at once:

strategy:
  fail-fast: false
  matrix:
    python-version: ['3.9', '3.11', '3.13']

Set fail-fast: true (the default) if you want to cancel remaining jobs as soon as one fails, saving runner minutes and providing immediate feedback.

Handling Version-Specific Imports​

Some packages drop support for old Python versions. Handle imports conditionally:

import sys

if sys.version_info >= (3, 10):
    from graphlib import TopologicalSorter
else:
    from toposort import TopologicalSorter

In your workflow, test that the conditional import works:

- name: Verify conditional imports
  run: python -c "import mypackage.utils; print('Imports OK')"

This catches import errors for versions you're testing without creating separate test files.

Common Matrix Configurations​

Testing a Python package across versions and OSs:

strategy:
  matrix:
    os: [ubuntu-latest, windows-latest]
    python-version: ['3.9', '3.11', '3.13']
    exclude:
      - os: windows-latest
        python-version: '3.9'

Testing against dependency versions:

strategy:
  matrix:
    python-version: ['3.11']
    dependency-version: [minimal, latest]
steps:
  - name: Install minimal dependencies
    if: matrix.dependency-version == 'minimal'
    run: pip install -e .[minimal]
  - name: Install latest dependencies
    if: matrix.dependency-version == 'latest'
    run: pip install -e .

Key Takeaways​

  • A matrix strategy automatically generates multiple jobs from a single definition, testing different configurations in parallel.
  • Define matrix variables (Python version, OS, dependency version) and GitHub spawns a job for each combination.
  • Use ${{ matrix.<variable> }} to access matrix values in steps.
  • Use include: to add custom configurations and exclude: to skip combinations.
  • Set fail-fast: false to run all jobs even if one fails; fail-fast: true to cancel remaining jobs immediately.
  • Use conditional steps (if: matrix.python-version == '3.11') to run certain steps only for specific configurations.

Frequently Asked Questions​

Why does my matrix job have a different name?​

GitHub appends matrix values to the job name in parentheses, e.g., test (3.11, ubuntu-latest). You can customize this with name: Test (${{ matrix.python-version }}).

How many matrix jobs can I run?​

GitHub limits you to 256 jobs per workflow run. For example, 4 Python versions × 4 OSs × 16 dependency versions = 256 jobs. Larger matrices are usually unnecessary; focus on the most important configurations.

Can I run matrix jobs sequentially instead of in parallel?​

Not directly through matrix settings. Instead, create a dependency between jobs using needs: to serialize them. However, this defeats the purpose of the matrix (speed) and wastes runner minutes.

What if a job fails only on one matrix configuration?​

GitHub highlights which configuration failed. Fix the issue for that version (conditional import, version-specific API, etc.) or update requirements to drop support for that version.

Can I reuse a matrix across multiple workflows?​

Matrix definitions are specific to each workflow. To standardize matrices across workflows, use a shell script or GitHub Actions composite action to generate the matrix dynamically.

Further Reading​