Skip to main content

How to Run Python Tests in GitHub Actions

Running Python tests in GitHub Actions ensures every commit and pull request is verified against your test suite before merging. GitHub Actions integrates seamlessly with pytest, unittest, and other Python test runners, automatically collecting test results and surfacing failures in pull request checks.

In this guide, you'll learn how to configure your workflow to discover and run tests, parallelize test execution to reduce feedback latency, generate XML reports for test result summaries, and fail the workflow when tests fail. You'll move beyond basic test execution to professional-grade test automation.

Setting Up pytest in Your Workflow​

pytest is the most popular Python testing framework because it requires minimal boilerplate, discovers tests automatically, and integrates deeply with CI/CD systems. Add pytest to your requirements.txt (or requirements-dev.txt):

pytest>=7.0
pytest-cov>=4.0
pytest-xdist>=3.0

Then update your workflow to install and run pytest:

name: Python Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with:
        python-version: '3.11'
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements.txt
    - name: Run tests with pytest
      run: pytest tests/ -v --tb=short

The -v flag prints verbose output (one line per test). The --tb=short flag limits traceback verbosity so you can spot assertion failures quickly. pytest automatically discovers test files matching test_*.py or *_test.py in the tests/ directory.

Generating Test Reports for GitHub​

GitHub Actions can parse JUnit XML test reports and display test results directly in the pull request checks interface. This visual feedback accelerates debugging without requiring developers to click into the Actions tab.

Generate JUnit XML and upload it to GitHub:

- name: Run tests and generate report
  run: pytest tests/ -v --junit-xml=test-results.xml --tb=short
- name: Upload test results
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: test-results
    path: test-results.xml
- name: Publish test results
  if: always()
  uses: EnricoMi/publish-unit-test-result-action@v2
  with:
    files: test-results.xml
    check_name: pytest Results

The if: always() condition ensures the upload step runs even if tests fail. The publish-unit-test-result-action is a popular community action that displays test results in the GitHub UI, including pass/fail counts and detailed failure messages.

Parallelizing Tests for Speed​

Running tests sequentially is slow. pytest-xdist enables parallel test execution across multiple CPU cores or worker processes, reducing total test time by 40–70% on multi-core runners.

Install pytest-xdist, then use the -n flag to specify the number of workers:

- name: Run tests in parallel
  run: pytest tests/ -v --tb=short -n auto

The auto value detects the number of CPU cores available and spawns that many worker processes. You can also hardcode a value like -n 4 for four workers. pytest-xdist distributes tests across workers, so individual tests still run in isolation.

Handling Fixtures and Shared State​

pytest fixtures that manage database connections, temporary files, or API mocks must be thread-safe when running tests in parallel. If your tests use fixtures that hold state, pytest-xdist can scope them appropriately.

For example, fixture-based database setup:

import pytest

@pytest.fixture(scope="function")
def db():
    """Create a fresh database for each test."""
    database = setup_test_db()
    yield database
    teardown_test_db(database)

def test_user_creation(db):
    """Test that users can be created."""
    user = db.create_user(name="Alice")
    assert user.id is not None

Each test gets a fresh database fixture, so parallel runs don't interfere. pytest-xdist respects fixture scopes and distributes tests safely.

Running Tests with unittest​

If your project uses unittest instead of pytest, run tests using Python's built-in test discovery:

- name: Run tests with unittest
  run: python -m unittest discover -s tests/ -p 'test_*.py' -v

This discovers all test files under tests/ matching the pattern test_*.py. The -v flag prints test names and results. unittest is slower and less flexible than pytest, so consider migrating to pytest for larger projects.

Setting Exit Codes and Conditional Steps​

By default, pytest and unittest return exit code 0 if all tests pass and non-zero (typically 1) if any test fails. GitHub Actions fails the workflow step if the exit code is non-zero, automatically failing the entire job.

You can make workflows conditional on test success:

- name: Run tests
  id: tests
  run: pytest tests/ -v
- name: Upload coverage only on success
  if: steps.tests.outcome == 'success'
  run: curl https://codecov.io/upload -F coverage

The id: field labels the step so later steps can reference its outcome.

Key Takeaways​

  • Use pytest with -v --tb=short for clear, concise test output in workflows.
  • Generate JUnit XML reports with --junit-xml=test-results.xml and publish them to GitHub using publish-unit-test-result-action.
  • Parallelize tests with pytest -n auto to reduce workflow time by 40–70%.
  • pytest fixtures with scope="function" are safe for parallel execution.
  • Use unittest discover if your project relies on the standard library test runner.
  • Set if: always() on artifact upload steps to ensure they run regardless of test outcome.

Frequently Asked Questions​

Why do some tests pass locally but fail in the workflow?​

Environment differences are common. The workflow runs on a clean virtual machine without your local configuration or cached dependencies. Ensure requirements.txt specifies exact versions (not wildcards like pytest>=7.0). Use absolute imports, not relative imports, so tests find your package regardless of the working directory.

How do I run only a specific test file or test function in the workflow?​

Pass the file path or function name to pytest: pytest tests/test_auth.py::test_login_success -v. Use conditionals to run different tests based on the branch: if: github.ref == 'refs/heads/main'.

Can I mark tests as expected failures without failing the workflow?​

Yes, use pytest's @pytest.mark.xfail decorator to mark tests that are expected to fail. pytest counts them separately (xfailed) and does not fail the workflow: @pytest.mark.xfail(reason="Bug #1234").

How do I skip tests in the workflow?​

Use @pytest.mark.skip(reason="Not ready") or @pytest.mark.skipif(condition, reason="..."). Skipped tests do not cause the workflow to fail.

Does GitHub provide a test result dashboard?​

GitHub's Actions tab shows test runs, but for deeper analytics (trends, flaky test detection, historical comparisons), use third-party tools like Codecov, BuildKite, or Datadog. Many integrate via GitHub Actions via community actions.

Further Reading​