4. Cold Starts in AI

Overview

In this capsule you'll confront the most-debated problem of Lambda for AI: cold starts. Not with opinions — with real numbers. You'll understand what a cold start is, measure its impact with different dependencies (openai vs langchain vs numpy+pandas), and learn concrete mitigation strategies. By the end, you'll be able to calculate whether your function's cold start is acceptable for your use case and what to do if it isn't.

Context: The previous capsules taught you to write a Lambda function for AI (02) and to package it as a zip or container (03). In both cases we mentioned cold starts as a trade-off. Here you go deeper: how long do they really last? Why are they worse for AI? How do you mitigate them without spending a fortune? This is an engineering problem, not an abstract limitation.

What a Cold Start Is

The concept

A cold start happens when Lambda needs to create a new container to run your function. This happens when:

1. First invocation after deploying
2. No containers available (all are busy with other invocations)
3. Expired container (Lambda recycles idle containers after ~5-15 minutes)
4. Scaling up (traffic increases and Lambda needs more containers)

Cold start (new container):
┌─────────────────────────────────────────────────┐
│ 1. Provision container                          │ ~200-500ms
│ 2. Download code/image                          │ ~100ms-3s
│ 3. Initialize Python runtime                    │ ~100-200ms
│ 4. Run code outside the handler (imports)        │ ~200ms-10s
│ 5. Run handler                                  │ Variable
└─────────────────────────────────────────────────┘

Warm start (existing container):
┌─────────────────────────────────────────────────┐
│ 5. Run handler                                  │ Variable
└─────────────────────────────────────────────────┘

The key difference: in a warm start, steps 1-4 are skipped. The container already exists, the runtime is loaded, your imports are in memory. Only the handler runs.

Why it matters more for AI

For a typical web function (returns HTML, processes a form), a 500ms cold start is imperceptible. For an AI function, the cold start adds to the inference time:

Typical web function:
├── Cold start:  500ms
├── Handler:     50ms
└── Total:       550ms (user waits ~0.5s)

AI function (openai SDK):
├── Cold start:  1.5s
├── Handler:     3s (call to GPT-4o-mini)
└── Total:       4.5s (user waits ~4.5s)

AI function (langchain + deps):
├── Cold start:  8s
├── Handler:     3s (call to GPT-4o-mini)
└── Total:       11s (user thinks the app is broken)

The handler takes the same in both cases (the LLM call is the same). The difference is that with heavy dependencies, the cold start doubles or triples the total time.

Real Measurement of Cold Starts

Methodology

To measure cold starts reliably, you need to:

1. Force a cold start (update the function between invocations)
2. Measure total time from when Lambda receives the event
3. Separate init duration from handler duration
4. Repeat 10+ times to average

Lambda reports the Init Duration in the CloudWatch logs — that's your cold start as measured by AWS.

Measurement tool

# measure_cold_start.py
# Script to measure cold starts consistently
import json
import time
import subprocess
import statistics


def force_cold_start(function_name):
    """Forces a cold start by updating an env var."""
    subprocess.run([
        "aws", "lambda", "update-function-configuration",
        "--function-name", function_name,
        "--environment", json.dumps({
            "Variables": {
                "OPENAI_API_KEY": "sk-test",
                "COLD_START_MARKER": str(time.time()),
            }
        }),
    ], capture_output=True)
    time.sleep(5)


def invoke_and_measure(function_name):
    """Invokes Lambda and returns total duration."""
    payload = json.dumps({"body": json.dumps({"prompt": "Say hello"})})

    start = time.time()
    result = subprocess.run([
        "aws", "lambda", "invoke",
        "--function-name", function_name,
        "--payload", payload,
        "--log-type", "Tail",
        "output.json",
    ], capture_output=True, text=True)
    total = time.time() - start

    return total


def run_benchmark(function_name, iterations=10):
    """Runs a cold-start benchmark."""
    cold_starts = []
    warm_starts = []

    for i in range(iterations):
        # Cold start
        force_cold_start(function_name)
        cold = invoke_and_measure(function_name)
        cold_starts.append(cold)

        # Warm start (immediate invocation, same container)
        warm = invoke_and_measure(function_name)
        warm_starts.append(warm)

        print(f"  Iteration {i+1}: cold={cold:.2f}s, warm={warm:.2f}s")

    print(f"\nCold starts: avg={statistics.mean(cold_starts):.2f}s, "
          f"p50={statistics.median(cold_starts):.2f}s, "
          f"max={max(cold_starts):.2f}s")
    print(f"Warm starts: avg={statistics.mean(warm_starts):.2f}s, "
          f"p50={statistics.median(warm_starts):.2f}s, "
          f"max={max(warm_starts):.2f}s")
    print(f"Cold start overhead: {statistics.mean(cold_starts) - statistics.mean(warm_starts):.2f}s")

Results by dependency type

These are representative results for Lambda functions with 512MB of memory in us-east-1:

┌────────────────────────────────────────────────────────────────┐
│ Dependency             │ Zip Cold │ Container Cold │ Warm     │
├────────────────────────┼──────────┼────────────────┼──────────┤
│ No dependencies        │  0.3s    │   0.8s         │  <0.1s   │
│ openai SDK (~5MB)      │  0.8s    │   1.5s         │  <0.1s   │
│ openai + httpx (~8MB)  │  1.0s    │   1.8s         │  <0.1s   │
│ anthropic SDK (~8MB)   │  1.0s    │   1.7s         │  <0.1s   │
│ langchain (~80MB)      │  3.5s    │   5.0s         │  <0.1s   │
│ langchain + chromadb   │  5.0s    │   7.0s         │  <0.1s   │
│ numpy + pandas (~120MB)│  4.0s    │   6.5s         │  <0.1s   │
│ numpy + sklearn        │  5.5s    │   8.0s         │  <0.1s   │
│ torch CPU (~800MB)     │  N/A     │  15-25s        │  <0.1s   │
└────────────────────────┴──────────┴────────────────┴──────────┘

Memory: 512MB | Runtime: Python 3.11 | Region: us-east-1
N/A = doesn't fit in a zip (exceeds the 250MB limit)

What the data reveals

1. Warm starts are always <100ms. The cold start is a one-off event; once warm, Lambda responds fast.

2. The jump from openai to langchain is 4x. It's not linear with size — it depends on how many imports and how much initialization there is.

3. Container adds 1-2s over zip. The overhead is the image pull from ECR.

4. 512MB of memory. With more memory, Lambda allocates more proportional CPU, and cold starts drop. With 1024MB, the numbers reduce ~30%.

5. torch is prohibitive. A 15-25s cold start makes Lambda unviable for local models. Use SageMaker or ECS.

Anatomy of the Cold Start

Where the time goes

# handler.py with timing logging

import time
_init_start = time.time()

import json                    # ~1ms
import os                      # ~1ms
_after_stdlib = time.time()

from openai import OpenAI      # ~300ms (imports httpx, pydantic, etc.)
_after_openai = time.time()

# If you use langchain, add ~2-5s here
# from langchain_openai import ChatOpenAI

client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY", ""))
_after_client = time.time()

_init_total = _after_client - _init_start
print(f"INIT: stdlib={(_after_stdlib-_init_start)*1000:.0f}ms, "
      f"openai={(_after_openai-_after_stdlib)*1000:.0f}ms, "
      f"client={(_after_client-_after_openai)*1000:.0f}ms, "
      f"total={_init_total*1000:.0f}ms")


def handler(event, context):
    handler_start = time.time()

    body = json.loads(event.get("body", "{}"))
    prompt = body.get("prompt", "test")

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=100,
    )

    handler_time = time.time() - handler_start

    return {
        "statusCode": 200,
        "body": json.dumps({
            "answer": response.choices[0].message.content,
            "init_ms": int(_init_total * 1000),
            "handler_ms": int(handler_time * 1000),
        }),
    }

Typical output in CloudWatch:

INIT: stdlib=2ms, openai=312ms, client=45ms, total=359ms
Init Duration: 847ms (AWS overhead + your init)
Duration: 2456ms (handler execution)
Billed Duration: 3303ms (init + handler)

The hidden cost: transitive imports

from openai import OpenAI doesn't just import openai. It imports httpx, anyio, pydantic, certifi, and more. Each transitive import adds to the cold start:

openai imports:
├── httpx (~150ms)
│   ├── anyio
│   ├── httpcore
│   └── certifi
├── pydantic (~100ms)
│   └── pydantic-core
└── typing_extensions (~5ms)
Total: ~300ms

langchain imports:
├── langchain_core (~500ms)
│   ├── pydantic
│   ├── jsonpatch
│   └── tenacity
├── langchain (~1500ms)
│   ├── requests
│   ├── aiohttp
│   ├── SQLAlchemy
│   └── numpy (if installed)
├── langchain_openai (~200ms)
│   └── openai
└── dataclasses-json (~100ms)
Total: ~2500ms

Mitigation Strategies

Strategy 1: Package optimization

The first line of defense: include only what you need.

# Problem: pip install langchain brings EVERYTHING
pip install langchain
# Installs: langchain, langchain-core, langchain-text-splitters,
# SQLAlchemy, aiohttp, requests, numpy, pyyaml, ...

# Solution: install only the components you use
pip install langchain-core langchain-openai
# Installs: langchain-core, langchain-openai, openai, pydantic
# ~70% fewer dependencies

# requirements.txt — BEFORE (all of langchain)
langchain>=0.2.0

# requirements.txt — AFTER (only what's needed)
langchain-core>=0.2.0
langchain-openai>=0.1.0

Measured impact:

Full langchain:          Cold start ~3.5s (zip) / ~5.0s (container)
langchain-core + openai: Cold start ~1.5s (zip) / ~2.5s (container)
Reduction: ~55%

Strategy 2: Lazy imports

Import heavy modules only when you need them, not at startup:

# ❌ Import at startup — penalizes ALL cold starts
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate

def handler(event, context):
    ...

# ✅ Lazy import — only penalizes the first invocation that uses it
_chain = None

def _get_chain():
    global _chain
    if _chain is None:
        from langchain_openai import ChatOpenAI
        from langchain_core.prompts import ChatPromptTemplate

        prompt = ChatPromptTemplate.from_messages([
            ("system", "Be concise."),
            ("user", "{input}"),
        ])
        llm = ChatOpenAI(model="gpt-4o-mini")
        _chain = prompt | llm
    return _chain


def handler(event, context):
    body = json.loads(event.get("body", "{}"))

    if body.get("use_chain"):
        chain = _get_chain()
        result = chain.invoke({"input": body["prompt"]})
        answer = result.content
    else:
        # Fast path without langchain
        response = client.chat.completions.create(...)
        answer = response.choices[0].message.content
    ...

Advantage: if most invocations use the path without LangChain, the cold start doesn't include those imports.

Strategy 3: More memory = more CPU = faster cold start

Lambda allocates CPU proportional to the configured memory. More memory → more CPU → faster imports:

langchain cold start by memory level:
├── 128MB:   ~6.0s  (minimum CPU)
├── 256MB:   ~4.5s
├── 512MB:   ~3.5s
├── 1024MB:  ~2.5s  (sweet spot for AI)
├── 1769MB:  ~2.0s  (1 full vCPU)
├── 3008MB:  ~1.5s
└── 10240MB: ~1.2s  (diminishing returns)

The sweet spot for AI functions is 1024-1769MB: enough CPU for fast imports without paying for memory you don't use at runtime.

# Change memory
aws lambda update-function-configuration \
    --function-name ai-endpoint \
    --memory-size 1024

# Lambda Power Tuning — tool to find the optimum
# https://github.com/alexcasalboni/aws-lambda-power-tuning

Strategy 4: Provisioned Concurrency

Keeps N containers always warm. Zero cold starts for those N containers.

# Configure 5 always-warm containers
aws lambda put-provisioned-concurrency-config \
    --function-name ai-endpoint \
    --qualifier prod \
    --provisioned-concurrent-executions 5

Without provisioned concurrency:
├── Invocations 1-N: Cold start (~3s each)
├── Following invocations: Warm (~0.1s)
└── After inactivity: Cold start again

With provisioned concurrency (5):
├── 5 simultaneous invocations 1-5: Warm (~0.1s) ← always
├── Invocation 6+: Cold start (overflow)
└── Cost: you pay for the 5 containers 24/7

Cost of provisioned concurrency:

Provisioned: $0.0000041667/GB-second (always running)
On-demand:   $0.0000166667/GB-second (only when it runs)

5 containers × 512MB × 24h × 30 days:
5 × 0.5GB × 86400s × 30 × $0.0000041667 = ~$27/month

That's $27/month to eliminate cold starts on the first 5 concurrent invocations.

Is it worth it? Depends on your case:

Strategy 5: Keep-warm (periodic ping)

A CloudWatch Event that invokes your Lambda every 5 minutes to keep containers warm:

# In SAM template
Resources:
  AIEndpoint:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handler.handler
      # ... config ...
      Events:
        KeepWarm:
          Type: Schedule
          Properties:
            Schedule: rate(5 minutes)
            Input: '{"source": "keep-warm"}'

def handler(event, context):
    # Detect keep-warm invocation and return quickly
    if event.get("source") == "keep-warm":
        return {"statusCode": 200, "body": "warm"}

    # Normal logic
    ...

Limitations:

• Only keeps 1 container warm. If you have concurrent invocations, the additional ones still have a cold start.
• Lambda can recycle the container even with pings (it's not a guarantee).
• Cost: keep-warm invocations are charged (but they're very cheap because they last <100ms).

Keep-warm: 12 invocations/hour × 24h × 30 days = 8640 invocations
× 100ms × 512MB = ~$0.05/month

vs Provisioned concurrency: ~$27/month for 5 containers

Keep-warm is ~500x cheaper but only keeps 1 container.

Comparison of Mitigation Strategies

Recommended combination for AI

Level 1 (free, always):
├── Optimize dependencies (only what you use)
├── Lazy imports for optional paths
└── Memory ≥ 512MB for AI functions

Level 2 (low traffic, ~$0.05/month):
└── Keep-warm every 5 minutes

Level 3 (production, $5-50/month):
└── Provisioned concurrency (N according to concurrent traffic)

Cold Starts in Context: Is It Really a Problem?

When the cold start does NOT matter

• Batch processing: No one is waiting. A 10s cold start in a 2-minute job is irrelevant.
• Webhooks and event processing: The event is processed in the background. 3s extra don't affect it.
• Constant traffic: If your Lambda receives invocations every few seconds, the containers stay warm naturally. Cold starts are rare.
• Async processing: The user gets an immediate "202 Accepted." The cold start doesn't affect their experience.

When the cold start DOES matter

• Chatbots and synchronous APIs: The user sees a spinner during cold start + inference. 8s total feels slow.
• Traffic spikes: When traffic rises suddenly, Lambda needs to create many new containers → many simultaneous cold starts.
• Latency SLAs: If you promised p99 < 5s, a 3s cold start + 3s handler = 6s > SLA.

Decision framework

Does your user wait for the response in real time?
├── NO → Cold starts don't matter. Don't invest in mitigation.
└── YES → How long is your cold start?
          ├── <2s → Acceptable for most UX. Optimize the package.
          ├── 2-5s → Depends on the UX. Keep-warm may be enough.
          └── >5s → You need provisioned concurrency or reconsider Lambda.
                    Does the cost of provisioned justify it?
                    ├── YES → Provisioned concurrency
                    └── NO → Consider ECS/Fargate (always-on server)

Troubleshooting

Problem 1: "10s+ cold start with only the openai SDK"

# Check the configured memory
aws lambda get-function-configuration \
    --function-name ai-endpoint \
    --query 'MemorySize'

# If it's 128MB, that's the problem — there isn't enough CPU
# Bump to 512MB minimum for AI functions
aws lambda update-function-configuration \
    --function-name ai-endpoint \
    --memory-size 512

Problem 2: "Cold starts are inconsistent (sometimes 2s, sometimes 8s)"

# Lambda may provision your container on different hardware types.
# The variance is normal. Measure with 10+ invocations and use the p50/p95.

# Check in CloudWatch Logs
aws logs filter-log-events \
    --log-group-name /aws/lambda/ai-endpoint \
    --filter-pattern "Init Duration"

# Also check that you're not hitting a concurrency limit
aws lambda get-function \
    --function-name ai-endpoint \
    --query 'Concurrency'

Problem 3: "Provisioned concurrency is active but I still see cold starts"

# Check the number of provisioned vs concurrent invocations
aws lambda get-provisioned-concurrency-config \
    --function-name ai-endpoint \
    --qualifier prod

# If you have 5 provisioned but 8 simultaneous invocations,
# 3 will have a cold start. Increase provisioned or implement rate limiting.

Problem 4: "Keep-warm works but at noon there are cold starts"

Lambda can create new containers when traffic increases. Keep-warm only keeps 1 container. If at noon you have 5 concurrent invocations, 4 will have a cold start.

09:00 - 1 invocation   → warm (keep-warm container)
09:01 - 1 invocation   → warm
12:00 - 5 simultaneous → 1 warm + 4 cold starts
12:01 - 3 simultaneous → 3 warm (containers from the previous spike)

Solution: provisioned concurrency tuned to the expected peak, or accept the cold starts during spikes.

Hands-On Exercises

Exercise 1: Measure your cold start

Create a Lambda function with the openai SDK (zip deployment). Invoke it 5 times forcing a cold start and 5 times warm. Document the results in a table.

# Create the function (if it doesn't exist)
# Use the build.sh from capsule 03

# Simplified measurement script
for i in $(seq 1 5); do
    echo "--- Cold start $i ---"

    # Force a cold start by changing an env var
    aws lambda update-function-configuration \
        --function-name ai-endpoint-zip \
        --environment "Variables={OPENAI_API_KEY=$OPENAI_API_KEY,MARKER=$i}" \
        --query 'LastUpdateStatus' --output text
    sleep 8

    # Invoke and capture Init Duration
    aws lambda invoke \
        --function-name ai-endpoint-zip \
        --payload '{"body": "{\"prompt\": \"Say hi\"}"}' \
        --log-type Tail \
        --query 'LogResult' \
        --output text out.json | base64 -d | grep -E "Init Duration|Duration|Billed"

    echo "--- Warm start $i ---"
    # Invoke immediately (warm)
    aws lambda invoke \
        --function-name ai-endpoint-zip \
        --payload '{"body": "{\"prompt\": \"Say hi\"}"}' \
        --log-type Tail \
        --query 'LogResult' \
        --output text out.json | base64 -d | grep -E "Duration|Billed"
done

| # | Cold Start (Init) | Cold Total | Warm Total |
|---|-------------------|------------|------------|
| 1 | 823ms             | 3.2s       | 2.4s       |
| 2 | 791ms             | 3.1s       | 2.3s       |
| 3 | 856ms             | 3.3s       | 2.5s       |
| 4 | 812ms             | 3.2s       | 2.4s       |
| 5 | 834ms             | 3.1s       | 2.3s       |
| **Avg** | **823ms**    | **3.18s**  | **2.38s**  |

Cold start overhead: 3.18 - 2.38 = 0.80s

Exercise 2: Optimize the dependencies

Take a function that uses full langchain and replace it with only the components you need (langchain-core, langchain-openai). Measure the cold start before and after.

# BEFORE: requirements.txt
echo "langchain>=0.2.0
langchain-openai>=0.1.0" > requirements-before.txt

pip install -r requirements-before.txt -t before-package/ \
    --platform manylinux2014_x86_64 \
    --only-binary=:all: \
    --python-version 3.11

du -sh before-package/
# ~85MB

# AFTER: optimized requirements.txt
echo "langchain-core>=0.2.0
langchain-openai>=0.1.0" > requirements-after.txt

pip install -r requirements-after.txt -t after-package/ \
    --platform manylinux2014_x86_64 \
    --only-binary=:all: \
    --python-version 3.11

du -sh after-package/
# ~35MB

echo "Reduction: $(echo "scale=0; (85-35)*100/85" | bc)%"
# ~59% reduction in size

# The handler stays the same if you only use ChatOpenAI and ChatPromptTemplate
# Those are in langchain-core and langchain-openai

# Deploy both versions and measure cold starts
# Before:  Init ~2.5-3.5s
# After:   Init ~1.0-1.5s

| Version | Deps size | Cold start (avg) | Warm start (avg) |
|---------|-------------|------------------|------------------|
| full langchain | 85MB | 3.5s | 2.4s |
| langchain-core+openai | 35MB | 1.5s | 2.4s |
| **Improvement** | **59%** | **57%** | **0%** |

Exercise 3: Implement keep-warm

Configure a CloudWatch Event that invokes your Lambda every 5 minutes. Modify the handler to detect keep-warm invocations and return immediately.

# handler.py with keep-warm support
import json
import os
import time
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])


def handler(event, context):
    # Detect keep-warm invocation and return quickly
    if event.get("source") == "keep-warm":
        return {
            "statusCode": 200,
            "body": json.dumps({
                "status": "warm",
                "remaining_ms": context.get_remaining_time_in_millis(),
            }),
        }

    # Normal logic
    try:
        body = json.loads(event.get("body", "{}"))
    except json.JSONDecodeError:
        return _response(400, {"error": "Invalid JSON"})

    prompt = body.get("prompt", "").strip()
    if not prompt:
        return _response(400, {"error": "prompt required"})

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[{"role": "user", "content": prompt}],
        max_tokens=500,
    )

    return _response(200, {
        "answer": response.choices[0].message.content,
        "tokens": response.usage.total_tokens,
    })


def _response(status, body):
    return {
        "statusCode": status,
        "headers": {"Content-Type": "application/json", "Access-Control-Allow-Origin": "*"},
        "body": json.dumps(body),
    }

# template.yaml with keep-warm event
Resources:
  AIEndpoint:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handler.handler
      Runtime: python3.11
      CodeUri: lambda-function/
      Timeout: 60
      MemorySize: 512
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAIKey
      Events:
        AskAPI:
          Type: Api
          Properties:
            Path: /ask
            Method: post
        KeepWarm:
          Type: Schedule
          Properties:
            Schedule: rate(5 minutes)
            Input: '{"source": "keep-warm"}'
            Description: "Keeps Lambda warm to avoid cold starts"

Parameters:
  OpenAIKey:
    Type: String
    NoEcho: true

sam build && sam deploy --guided

# Verify that keep-warm works
# In CloudWatch Logs, every 5 minutes you should see:
# {"status": "warm", "remaining_ms": 59800}

Exercise 4: Calculate the mitigation cost

For your AI function (512MB, 3s cold start), calculate and compare the monthly cost of three scenarios: no mitigation, with keep-warm, and with provisioned concurrency (3 containers). Assume 5000 real invocations/day.

## Base data
- Memory: 512MB (0.5GB)
- Real invocations: 5000/day = 150,000/month
- Average handler duration: 3s
- Cold start: 3s additional
- Traffic: ~70% between 9am-6pm (9 hours)
- Concurrency peak: ~5 simultaneous

## Scenario 1: No mitigation
- 150,000 invocations × 3s × 0.5GB = 225,000 GB-seconds
- ~5% cold starts (estimated with regular traffic): 7,500 cold starts
- Cold start extra: 7,500 × 3s × 0.5GB = 11,250 GB-seconds
- Total: 236,250 GB-seconds
- Compute cost: 236,250 × $0.0000166667 = ~$3.94
- Invocations cost: 150,000 × $0.20/1M = ~$0.03
- **Total: ~$3.97/month**

## Scenario 2: Keep-warm
- Real invocations: same ($3.97)
- Keep-warm: 12/hour × 24h × 30 days = 8,640 invocations
- Keep-warm cost: 8,640 × 0.1s × 0.5GB × $0.0000166667 = ~$0.007
- Reduced cold starts: ~1% (only during concurrency spikes)
- **Total: ~$3.98/month** (practically the same)

## Scenario 3: Provisioned Concurrency (3)
- On-demand compute (same): ~$3.97
- Provisioned: 3 × 0.5GB × 86,400s × 30 × $0.0000041667 = ~$16.20
- Cold starts: 0 for the first 3 concurrent
- **Total: ~$20.17/month**

## Summary
| Strategy | Cost/month | Cold starts | UX |
|-----------|----------|-------------|------|
| No mitigation | $3.97 | ~5% of invocations | 3s extra occasional |
| Keep-warm | $3.98 | ~1% (only spikes) | Good for low traffic |
| Provisioned (3) | $20.17 | ~0% (up to 3 conc.) | Consistent |

Conclusion: Keep-warm is almost-free and eliminates most cold starts.
Provisioned is worth it if the latency SLA justifies the extra $16/month.

Summary

• Cold starts happen when Lambda creates a new container. For AI functions, they can last 1-15s depending on the dependencies.
• The main impact is on UX: a 5s cold start + 3s handler = 8s that the user perceives as slow.
• Optimizing dependencies is the first action (free, -30-60% on cold start). Don't install langchain when you only need langchain-core.
• More memory = more CPU = faster cold starts. The sweet spot for AI is 512MB-1024MB.
• Keep-warm (~$0.05/month) keeps 1 container warm. Enough for low traffic.
• Provisioned concurrency ($5-50+/month) eliminates cold starts for N containers. Necessary for production with SLAs.
• Warm starts are always <100ms. Cold starts are a one-off problem, not a constant one.
• Not everything needs mitigation. Batch and async processing aren't affected by cold starts.

Additional Resources

1. Lambda Cold Starts — Official Docs — How Lambda manages execution environments
2. Provisioned Concurrency — Official documentation on provisioned concurrency
3. Lambda Power Tuning — Tool to find the optimal memory/cost
4. Understanding Lambda Cold Starts — Lumigo — Detailed analysis of cold starts with real data
5. Lambda SnapStart — Cold start optimization (Java, but the concept applies)
6. Serverless Cold Start Comparison — Cold start benchmark by runtime and memory
7. AWS Lambda Pricing Calculator — To calculate provisioned concurrency costs
8. Reducing Lambda Cold Starts — AWS Blog — Official best practices