4. Dependency Caching

Overview

Every time your workflow runs, the runner downloads and installs all your dependencies from scratch. For a simple Python project, this takes 10-15 seconds. For an AI project with torch, transformers, langchain, and openai, it can take 2-3 minutes. Multiply that by 3 matrix testing jobs and you have 6-9 minutes just installing dependencies — before a single test runs.

Dependency caching solves this. The first time, the workflow downloads and installs everything normally. At the end, it saves the dependencies in a cache. The next time, instead of downloading from PyPI, it restores the cache in seconds. The result: pip install goes from 2 minutes to 5-10 seconds.

For AI projects, caching is not an optional optimization — it's a necessity. AI dependencies are enormous: torch weighs ~2 GB, transformers ~500 MB. Without caching, every push downloads all of that. With caching, you only download it when requirements.txt changes.

The problem: Visualized

This is what a workflow looks like without caching vs with caching for a typical AI project:

Without cache (every run):
├── Checkout code                    2s
├── Setup Python                     5s
├── pip install -r requirements.txt  147s  ← 2.5 minutes every time
├── Run pytest                       12s
└── Total                            166s  (2 min 46s)

With cache (from the second run onward):
├── Checkout code                    2s
├── Setup Python + restore cache     7s   ← Cache hit!
├── pip install -r requirements.txt  3s   ← Everything is already installed
├── Run pytest                       12s
└── Total                            24s   (24 seconds!)

From 2 minutes 46 seconds to 24 seconds. An 85% reduction in CI time.

Option 1: actions/setup-python with cache: "pip" (The simplest)

The easiest way to enable caching is to add one line to the setup-python step:

# .github/workflows/test.yml
name: Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      # checkout MUST come before setup-python — cache key is built from requirements.txt hash
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Python with cache
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          # One line turns 2-min installs into 5-sec cache restores — key auto-invalidates when requirements.txt changes
          cache: "pip"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests
        run: pytest tests/ -v --tb=short

What does cache: "pip" do?

1. Before pip install: It looks for a cache based on the hash of requirements.txt
2. If it finds a cache (hit): It restores the downloaded packages → pip install is almost instant
3. If it doesn't find one (miss): pip install downloads everything normally
4. After the job: It saves the cache for next time

What exactly does it cache?

cache: "pip" caches pip's cache directory (~/.cache/pip on Linux), not the full virtualenv. This means pip install still runs, but instead of downloading packages from the internet, it takes them from the local cache.

Output in the logs

First time (cache miss):

Run actions/setup-python@v5
  Cache not found for input keys: setup-python-Linux-...pip-abc123def456

Run pip install -r requirements.txt
  Downloading langchain-0.3.14-py3-none-any.whl (1.2 MB)
  ...

Post job cleanup:
  Cache saved with key: setup-python-Linux-...pip-abc123def456

Second time (cache hit):

Run actions/setup-python@v5
  Cache restored from key: setup-python-Linux-...pip-abc123def456

Run pip install -r requirements.txt
  Requirement already satisfied: langchain==0.3.14
  ...

The difference is dramatic: "Downloading" vs "Requirement already satisfied".

Option 2: actions/cache directly (More control)

For more control over what gets cached, use actions/cache directly:

      - name: Cache pip dependencies
        uses: actions/cache@v4
        with:
          path: ~/.cache/pip
          # hashFiles auto-invalidates cache when deps change — no manual cache-busting needed
          key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
          # Fallback: partial match restores most packages, pip only downloads what changed
          restore-keys: |
            ${{ runner.os }}-pip-

The three parameters of actions/cache

path — Which directory to cache

key — Unique identifier of the cache

key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}

When requirements.txt changes, the hash changes, and the cache is invalidated automatically.

restore-keys — Fallback when there is no exact match

restore-keys: |
  ${{ runner.os }}-pip-

If there is no cache with the exact key, it looks for the most recent cache whose key starts with Linux-pip-. It's a partial fallback: pip install only downloads what changed.

The cache decision flow

Is there a cache with the exact key "Linux-pip-abc123"?
├── YES → Cache hit → Restores ~/.cache/pip → pip install is instant
│
└── NO → Is there a cache with the prefix "Linux-pip-"?
    ├── YES → Partial hit → Restores partial cache → pip downloads only what's new
    └── NO → Cache miss → pip downloads everything → Saves a new cache at the end

hashFiles(): Why it works

The hashFiles() function computes a SHA-256 hash of a file's contents. If the file doesn't change, the hash doesn't change, and the cache hit is exact.

# requirements.txt (version 1)
langchain==0.3.14     → Hash: a1b2c3d4e5f6...

# requirements.txt (version 2 — langchain updated)
langchain==0.3.15     → Hash: f6e5d4c3b2a1... (different)

A change in a single line produces a completely different hash. The previous cache no longer matches.

Multiple dependency files

# For requirements.txt + requirements-dev.txt:
key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }}

# For pyproject.toml:
key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml') }}

Caching with matrix testing

When you combine caching with matrix testing, each combination gets its own cache:

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4

      - name: Setup Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: "pip"

      - run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - run: pytest tests/ -v --tb=short

Caches generated

Cache keys:
├── setup-python-Linux-python-3.10-pip-abc123
├── setup-python-Linux-python-3.11-pip-abc123
└── setup-python-Linux-python-3.12-pip-abc123

Each Python version has its own cache because the wheels can differ between versions.

Result with matrix + caching

First run (cache miss):    ~148s per job
Second run (cache hit):    ~24s per job   ← 84% faster

Before / After: The full comparison

A typical AI project

# requirements.txt
langchain==0.3.14
langchain-openai==0.3.1
openai==1.58.1
pydantic==2.10.4
tiktoken==0.8.0
faiss-cpu==1.9.0
chromadb==0.5.23
pytest==8.3.4
pytest-cov==6.0.0

For projects with torch (~2 GB), the difference is even more brutal: pip install goes from ~185s to ~6s (-97%).

Cache invalidation: When the cache is stale

The cache is invalidated automatically when requirements.txt changes. But there are special situations:

An updated transitive dependency

If langchain==0.3.14 didn't change but one of its internal dependencies published a security fix, the hash is the same and the cache serves the old version.

Solution: Pin critical versions or invalidate manually with a "salt":

key: ${{ runner.os }}-pip-v2-${{ hashFiles('requirements.txt') }}

Change v2 to v3 when you need to invalidate.

Cache limits in GitHub Actions

GitHub deletes caches that aren't used for more than 7 days.

Risk: Unpinned dependencies

# BAD — unpinned versions
langchain>=0.3.0
openai>=1.50.0

With unpinned versions, the cache can serve an old version while your users install the new one. Always pin your versions:

langchain==0.3.14
openai==1.58.1

Best practices for AI projects

1. Separate heavy dependencies from light ones

# requirements.txt (core — light)
langchain==0.3.14
openai==1.58.1
pydantic==2.10.4

# requirements-ml.txt (heavy — only for ML features)
-r requirements.txt
torch==2.5.1
transformers==4.47.1

2. Use CPU-only torch in CI

# requirements-ci.txt
--extra-index-url https://download.pytorch.org/whl/cpu
torch==2.5.1+cpu
transformers==4.47.1

The CPU version of torch weighs ~200 MB instead of ~2 GB. In CI you don't need a GPU.

3. Don't cache what you don't need to test

If your CI only runs unit tests (which use mocks), you don't need to install torch:

# requirements-test.txt (only for CI unit tests)
langchain==0.3.14
openai==1.58.1
pytest==8.3.4
pytest-cov==6.0.0

Fewer dependencies = smaller cache = faster restore.

      - name: Cache virtualenv
        uses: actions/cache@v4
        id: cache-venv
        with:
          path: .venv
          key: ${{ runner.os }}-venv-${{ hashFiles('requirements.txt') }}

      - name: Create venv and install deps
        if: steps.cache-venv.outputs.cache-hit != 'true'
        run: |
          python -m venv .venv
          source .venv/bin/activate
          pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests
        run: |
          source .venv/bin/activate
          pytest tests/ -v --tb=short

Comparison: setup-python cache vs actions/cache

Recommendation: Start with cache: "pip" in setup-python. Only use actions/cache if you need to cache the full venv or you have a complex dependency structure.

Recommended complete workflow

Putting matrix testing + caching together:

# .github/workflows/test.yml
name: Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Setup Python ${{ matrix.python-version }}
        uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          # Each matrix combination gets its own cache — wheels differ between Python versions
          cache: "pip"

      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - name: Run tests
        run: pytest tests/ -v --tb=short
        env:
          PYTHONPATH: ${{ github.workspace }}

Troubleshooting

"Cache miss on every run even though I didn't change requirements.txt"

Cause: The cache key includes something that changes on every run.

# BAD — github.sha changes on every commit
key: ${{ runner.os }}-pip-${{ github.sha }}

# GOOD — only changes when requirements.txt changes
key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}

"Cache hit but pip install is still slow"

Cause: The pip cache stores the downloaded packages (wheels), but pip install still verifies each package. If the virtualenv isn't cached, pip reinstalls from the local cache.

Solution: For most projects, 3-5 seconds of verification is acceptable. If you need more speed, cache the full venv (see best practices).

"Error: Unable to reserve cache"

Cause: The repo hit the 10 GB cache limit.

Solution: Clean caches manually:

gh cache list
gh cache delete <cache-key>

Exercises

Exercise 1: Enable basic caching

Your current workflow has no caching. Add caching in the simplest way possible:

# .github/workflows/test.yml
name: Tests
on: push

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v

name: Tests
on: push

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v

Exercise 2: Cache with actions/cache for more control

Configure caching using actions/cache directly. The project has two dependency files: requirements.txt and requirements-dev.txt.

name: Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Cache pip dependencies
        uses: actions/cache@v4
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('requirements*.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-

      - run: |
          python -m pip install --upgrade pip
          pip install -r requirements-dev.txt

      - run: pytest tests/ -v --tb=short

Exercise 3: Caching with matrix testing

Combine caching with a matrix of Python 3.10, 3.11, and 3.12.

name: Tests
on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: ${{ matrix.python-version }}
          cache: "pip"

      - run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt

      - run: pytest tests/ -v --tb=short

setup-python-Linux-python-3.10-pip-abc123
setup-python-Linux-python-3.11-pip-abc123
setup-python-Linux-python-3.12-pip-abc123

Exercise 4: Diagnose why the cache isn't working

This workflow should have caching but every run shows "Cache miss". Why?

name: Tests
on: push

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - uses: actions/checkout@v4
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v

name: Tests
on: push

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
          cache: "pip"
      - run: pip install -r requirements.txt
      - run: pytest tests/ -v

Summary

• ✅ Without a cache, pip install downloads everything from scratch on every run (2-3 min for AI projects)
• ✅ cache: "pip" in setup-python is the simplest approach — one line of YAML
• ✅ actions/cache directly gives you control over path, key, and restore-keys
• ✅ hashFiles('requirements.txt') invalidates the cache automatically when the deps change
• ✅ restore-keys allows a partial cache when the exact key doesn't exist
• ✅ Each matrix combination (OS + Python version) has its own cache
• ✅ Pinned dependencies avoid the risk of a stale cache
• ✅ Heavy AI projects benefit enormously: from ~3 minutes to ~20 seconds
• ✅ checkout must come before setup-python when you use cache: "pip"

Additional resources

1. Caching dependencies - GitHub Actions - Official caching guide
2. actions/cache - Complete documentation of the action
3. actions/setup-python - Caching - Caching built into setup-python
4. Cache limits and eviction - Limits and expiration policies
5. PyTorch CPU wheels - Installing torch without a GPU for CI