Skip to main content

GitHub Actions Caching: Speed Up Python Workflows

Every workflow that installs Python dependencies waits for pip to download and build packages. On a fresh runner, installing a large dependency tree (NumPy, pandas, Django, FastAPI) can take 2–5 minutes. Caching stores the downloaded and compiled packages across workflow runs, reducing installation time to seconds.

GitHub's actions/cache action preserves directories between runs using a cache key derived from file contents. When you update requirements.txt, the cache invalidates automatically, ensuring fresh installations while avoiding stale packages.

Caching Python Dependencies with pip​

The simplest caching strategy targets pip's cache directory (~/.cache/pip/):

name: Build with Caching

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-python@v5
      with:
        python-version: '3.11'
    - name: Cache pip dependencies
      uses: actions/cache@v4
      with:
        path: ~/.cache/pip
        key: ${{ runner.os }}-pip-${{ hashFiles('**/requirements.txt') }}
        restore-keys: |
          ${{ runner.os }}-pip-
    - name: Install dependencies
      run: pip install -r requirements.txt
    - name: Run tests
      run: pytest tests/

The cache key combines the OS name (${{ runner.os }}) with a hash of requirements.txt. When requirements.txt changes, the key changes, and pip installs fresh packages. On subsequent pushes with the same dependencies, the cache hits and pip skips downloads.

The restore-keys: fallback pattern allows partial cache hits. If no exact key matches but a cache exists for ubuntu-pip-, GitHub restores it. This reduces misses when dependencies are added/removed.

Caching Virtual Environments​

Caching ~/.cache/pip/ is fast for pip downloads, but doesn't cache compiled wheels. For even faster runs, cache the entire virtual environment:

- name: Cache virtual environment
  uses: actions/cache@v4
  with:
    path: venv/
    key: ${{ runner.os }}-venv-${{ hashFiles('**/requirements.txt') }}
    restore-keys: |
      ${{ runner.os }}-venv-
- name: Install dependencies
  run: |
    python -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate
    pip install -r requirements.txt

This caches the entire virtual environment directory. On cache hits, activating the cached venv skips pip entirely, reducing installation from 30 seconds to under 1 second.

Note: Virtual environment caches are OS-specific. A venv created on Ubuntu cannot run on macOS or Windows (binary incompatibilities). Always include ${{ runner.os }} in the cache key to isolate caches by OS.

Cache Limits and Eviction​

GitHub allows up to 5 GB of cache per repository. When the total size exceeds 5 GB, GitHub evicts the least recently used caches. For most Python projects, pip caches are 200–500 MB, so limits rarely matter. If you hit the limit, you have 100 different Python versions cached; consider reducing matrix breadth or using selective caching.

View cache usage in your repository's Actions > Caches tab on GitHub. You can manually delete caches there if needed.

Caching Poetry and Pipenv Lock Files​

For projects using poetry or pipenv, cache the lock file hash instead:

Poetry:

- name: Cache Poetry virtual environment
  uses: actions/cache@v4
  with:
    path: |
      ~/.cache/pypoetry
      ~/.config/pypoetry
    key: ${{ runner.os }}-poetry-${{ hashFiles('**/poetry.lock') }}
    restore-keys: |
      ${{ runner.os }}-poetry-

Pipenv:

- name: Cache Pipenv virtual environment
  uses: actions/cache@v4
  with:
    path: ~/.local/share/virtualenvs
    key: ${{ runner.os }}-pipenv-${{ hashFiles('**/Pipfile.lock') }}
    restore-keys: |
      ${{ runner.os }}-pipenv-

These strategies hash lock files (which change only when dependencies change), ensuring deterministic caching.

Measuring Cache Impact​

GitHub shows cache hit/miss statistics in workflow logs:

Cache hit: true

or

Cache hit: false

Track these metrics over time. A healthy project should see 80%+ cache hits on main branch runs. Low hit rates indicate frequent dependency changes or an overly specific cache key.

Compare workflow runtime with and without caching:

Without caching: 2m 45s (1m 50s for pip, 55s for tests) With caching (cold): 2m 50s (first run, cache miss) With caching (warm): 1m 5s (cache hit, saves 1m 45s)

Caching saves the most on large dependency trees (data science, ML) and least on minimal dependencies (CLI tools).

Key Takeaways​

  • Cache ~/.cache/pip to cache downloaded packages; cache the venv directory to cache compiled wheels.
  • Use hashFiles('**/requirements.txt') as the cache key to invalidate cache when dependencies change.
  • Always include ${{ runner.os }} in the key to isolate caches by operating system.
  • Virtual environment caches are fast but OS-specific; pip caches are slower but OS-independent.
  • GitHub allows 5 GB cache per repository; evicts least recently used caches beyond the limit.
  • Monitor cache hit rates; healthy projects should see 80%+ hits on default branch runs.

Frequently Asked Questions​

Why is my cache not being used?​

Check the cache key. If you change requirements.txt, the hash changes and the old cache is ignored (correct behavior). If the key never matches, ensure it references the right file. Use hashFiles() to hash dependency files; manually constructing keys is error-prone.

Can I cache system-level dependencies installed with apt?​

GitHub Actions doesn't cache apt packages across runs. If your project needs system libraries (libpq for psycopg2, libssl for cryptography), either include them in a Docker container or accept the 10–30 second install overhead.

What's the difference between key and restore-keys?​

key is the exact cache to restore; if it doesn't exist, the cache misses. restore-keys: are fallback patterns; if key misses, GitHub searches for any cache matching the patterns in order. This is useful for handling version or file changes gracefully.

Should I cache my entire project directory?​

No. Cache only external dependencies (pip, npm, poetry), not your code. Caching code can hide build artifacts and make debugging harder.

How do I manually clear the cache?​

Go to your repository's Settings > Actions > Caches and click the trash icon next to the cache you want to delete. Alternatively, use the GitHub API.

Further Reading​