Module 3: Serverless & Lambda for AI

8. Project: Lambda AI Endpoint

Project description

This is the integrator project for Module 3. You'll build a complete Lambda AI Endpoint: a Lambda function that receives a prompt via API Gateway, invokes GPT-4o-mini, and returns a structured response. It includes a health endpoint, timeout and memory configuration optimized for AI, cold-start handling, CORS, API-key authentication, and cost estimation. By the end, you'll have a production-ready serverless endpoint.

Why it matters: This Lambda is the central artifact of Phase 1. In Module 4, you'll deploy it on LocalStack — running this same code locally without an AWS account or cost. In Module 5, you'll integrate it with S3 to persist prompts and responses. And in Module 8 (Integrator Project), it will be part of your system deployed in production. Building it well here means the next modules are built on a solid foundation.

Project objective

Produce a functional Lambda AI Endpoint that:

Receives a prompt via POST /ask with API Gateway (HTTP API v2)
Invokes GPT-4o-mini with optimized timeout and memory
Returns a structured response with metadata (tokens, duration, cold start)
Has a GET /health endpoint that verifies configuration
Handles errors gracefully (timeout, LLM errors, invalid input)
Includes CORS for invocation from browsers
Has API-key authentication (configurable)
Structured logging for CloudWatch Insights

Module Recap

Capsule	Concept	Where you use it in the project
02	Lambda fundamentals	Handler, event, context
03	Container vs zip, cold starts	Package optimization, IS_COLD_START
04	Environment variables, secrets	API keys, config via env vars
05	Timeout and memory	768MB, 60s timeout, client timeout
06	API Gateway	HTTP API v2, CORS, auth, routes
07	Cost estimation	Cost estimation of the endpoint

Technical Specifications

Architecture

                         ┌─────────────────────┐
    POST /ask ──────────→│                     │
    GET /health ────────→│  API Gateway        │
                         │  (HTTP API v2)      │
                         │  CORS + Throttling  │
                         └──────────┬──────────┘
                                    │
                         ┌──────────┴──────────┐
                         │                     │
                    ┌────┴────┐          ┌─────┴────┐
                    │  Ask    │          │  Health   │
                    │ Function│          │ Function  │
                    │ 768MB   │          │ 128MB     │
                    │ 60s     │          │ 5s        │
                    └────┬────┘          └──────────┘
                         │
                    ┌────┴────┐
                    │ OpenAI  │
                    │  API    │
                    └─────────┘

Required endpoints

POST /ask        → Receives prompt, invokes LLM, returns response
GET  /health     → Endpoint status and configuration

Request/Response format

# POST /ask
# Request:
{
  "prompt": "What is serverless?",
  "max_tokens": 500,
  "system_prompt": "Respond concisely and helpfully."  # Optional
}

# Response (200):
{
  "answer": "Serverless is an execution model...",
  "model": "gpt-4o-mini",
  "tokens_used": 142,
  "duration_ms": 2345,
  "cold_start": false
}

# Response (400):
{
  "error": "prompt is required"
}

# Response (502):
{
  "error": "LLM call failed: timeout"
}

# GET /health
# Response (200):
{
  "status": "healthy",
  "checks": {
    "lambda": "up",
    "openai_key": "configured"
  },
  "config": {
    "model": "gpt-4o-mini",
    "memory_mb": "768",
    "timeout_s": "60",
    "region": "us-east-1"
  }
}

Lambda configuration

Parameter	Ask Function	Health Function
Runtime	Python 3.11	Python 3.11
Architecture	arm64	arm64
Memory	768 MB	128 MB
Timeout	60s	5s
Handler	handler.handler	health.handler

Required files

lambda-ai-endpoint/
├── src/
│   ├── handler.py          # POST /ask handler
│   ├── health.py           # GET /health handler
│   └── requirements.txt    # Dependencies (openai only)
├── template.yaml           # SAM template
├── samconfig.toml          # SAM deploy config
├── env.json                # Local variables for sam local
├── events/
│   ├── ask.json            # Test event for /ask
│   └── health.json         # Test event for /health
├── tests/
│   └── test_handler.py     # Unit tests
└── README.md               # Endpoint documentation

Complete Code

src/handler.py

import json
import logging
import os
import time
from openai import OpenAI

logger = logging.getLogger()
logger.setLevel(os.environ.get("LOG_LEVEL", "INFO"))

LAMBDA_TIMEOUT = int(os.environ.get("LAMBDA_TIMEOUT", "60"))
MODEL_NAME = os.environ.get("MODEL_NAME", "gpt-4o-mini")
MAX_PROMPT_LENGTH = int(os.environ.get("MAX_PROMPT_LENGTH", "10000"))
MAX_TOKENS_LIMIT = int(os.environ.get("MAX_TOKENS_LIMIT", "2000"))
DEFAULT_MAX_TOKENS = int(os.environ.get("DEFAULT_MAX_TOKENS", "500"))
DEFAULT_SYSTEM_PROMPT = os.environ.get(
    "DEFAULT_SYSTEM_PROMPT",
    "Respond concisely and helpfully."
)
MIN_REMAINING_MS = 10000

client = OpenAI(
    api_key=os.environ.get("OPENAI_API_KEY", ""),
    timeout=LAMBDA_TIMEOUT - 10,
    max_retries=1,
)

CORS_HEADERS = {
    "Content-Type": "application/json",
    "Access-Control-Allow-Origin": os.environ.get("ALLOWED_ORIGIN", "*"),
    "Access-Control-Allow-Methods": "POST, OPTIONS",
    "Access-Control-Allow-Headers": "Content-Type, X-Api-Key",
}

IS_COLD_START = True


def _response(status_code, body):
    """Builds an HTTP response with CORS headers."""
    return {
        "statusCode": status_code,
        "headers": CORS_HEADERS,
        "body": json.dumps(body) if isinstance(body, dict) else body,
    }


def _validate_request(body):
    """Validates the request body and returns errors or None."""
    prompt = body.get("prompt", "")
    if not isinstance(prompt, str) or not prompt.strip():
        return "prompt is required and must be a non-empty string"

    if len(prompt) > MAX_PROMPT_LENGTH:
        return f"prompt too long: {len(prompt)} chars (max {MAX_PROMPT_LENGTH})"

    max_tokens = body.get("max_tokens", DEFAULT_MAX_TOKENS)
    if not isinstance(max_tokens, int) or max_tokens < 1 or max_tokens > MAX_TOKENS_LIMIT:
        return f"max_tokens must be integer between 1 and {MAX_TOKENS_LIMIT}"

    return None


def _check_auth(event):
    """Verifies the API key if configured."""
    api_keys_raw = os.environ.get("API_KEYS", "")
    if not api_keys_raw:
        return True

    valid_keys = set(api_keys_raw.split(","))
    request_key = event.get("headers", {}).get("x-api-key", "")
    return request_key in valid_keys


def handler(event, context):
    global IS_COLD_START
    was_cold = IS_COLD_START
    IS_COLD_START = False

    start_time = time.time()

    method = event.get("requestContext", {}).get("http", {}).get("method", "")
    if method == "OPTIONS":
        return _response(200, "")

    if not _check_auth(event):
        return _response(401, {"error": "Unauthorized: invalid or missing API key"})

    remaining_ms = context.get_remaining_time_in_millis()
    if remaining_ms < MIN_REMAINING_MS:
        logger.warning(f"Insufficient time: {remaining_ms}ms remaining")
        return _response(408, {
            "error": "Insufficient time remaining for LLM call",
            "remaining_ms": remaining_ms,
        })

    try:
        body = json.loads(event.get("body", "{}"))
    except (json.JSONDecodeError, TypeError):
        return _response(400, {"error": "Invalid JSON body"})

    validation_error = _validate_request(body)
    if validation_error:
        return _response(400, {"error": validation_error})

    prompt = body["prompt"].strip()
    max_tokens = min(body.get("max_tokens", DEFAULT_MAX_TOKENS), MAX_TOKENS_LIMIT)
    system_prompt = body.get("system_prompt", DEFAULT_SYSTEM_PROMPT)

    model = MODEL_NAME
    if remaining_ms < 20000:
        model = "gpt-4o-mini"
        max_tokens = min(max_tokens, 200)

    try:
        llm_start = time.time()
        response = client.chat.completions.create(
            model=model,
            messages=[
                {"role": "system", "content": system_prompt},
                {"role": "user", "content": prompt},
            ],
            max_tokens=max_tokens,
        )
        llm_ms = round((time.time() - llm_start) * 1000)

    except Exception as e:
        error_ms = round((time.time() - start_time) * 1000)
        logger.error(json.dumps({
            "event": "llm_error",
            "error": str(e),
            "duration_ms": error_ms,
            "cold_start": was_cold,
        }))
        return _response(502, {"error": f"LLM call failed: {str(e)}"})

    total_ms = round((time.time() - start_time) * 1000)
    answer = response.choices[0].message.content
    tokens = response.usage.total_tokens

    logger.info(json.dumps({
        "event": "ask_success",
        "cold_start": was_cold,
        "llm_duration_ms": llm_ms,
        "total_duration_ms": total_ms,
        "overhead_ms": total_ms - llm_ms,
        "tokens_used": tokens,
        "model": model,
        "prompt_length": len(prompt),
        "remaining_ms": context.get_remaining_time_in_millis(),
    }))

    return _response(200, {
        "answer": answer,
        "model": response.model,
        "tokens_used": tokens,
        "duration_ms": total_ms,
        "cold_start": was_cold,
    })

src/health.py

import json
import os
import time


def handler(event, context):
    start = time.time()

    checks = {"lambda": "up"}

    openai_key = os.environ.get("OPENAI_API_KEY", "")
    if not openai_key:
        checks["openai_key"] = "missing"
    elif not openai_key.startswith("sk-"):
        checks["openai_key"] = "invalid_format"
    else:
        checks["openai_key"] = "configured"

    overall = "healthy" if checks["openai_key"] == "configured" else "degraded"
    status_code = 200 if overall == "healthy" else 503

    return {
        "statusCode": status_code,
        "headers": {
            "Content-Type": "application/json",
            "Access-Control-Allow-Origin": os.environ.get("ALLOWED_ORIGIN", "*"),
        },
        "body": json.dumps({
            "status": overall,
            "checks": checks,
            "config": {
                "model": os.environ.get("MODEL_NAME", "gpt-4o-mini"),
                "memory_mb": os.environ.get("AWS_LAMBDA_FUNCTION_MEMORY_SIZE", "unknown"),
                "timeout_s": os.environ.get("LAMBDA_TIMEOUT", "60"),
                "region": os.environ.get("AWS_REGION", "unknown"),
                "function": os.environ.get("AWS_LAMBDA_FUNCTION_NAME", "unknown"),
                "architecture": "arm64",
            },
            "latency_ms": round((time.time() - start) * 1000, 1),
        }),
    }

src/requirements.txt

openai>=1.0.0

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Lambda AI Endpoint — Module 3 Project

Globals:
  Function:
    Runtime: python3.11
    Architectures:
      - arm64

Parameters:
  OpenAiApiKey:
    Type: String
    NoEcho: true
    Description: OpenAI API key
  AllowedOrigin:
    Type: String
    Default: "*"
    Description: CORS allowed origin
  ApiKeys:
    Type: String
    Default: ""
    Description: Comma-separated valid API keys (empty = no auth)
  ModelName:
    Type: String
    Default: "gpt-4o-mini"
    AllowedValues: ["gpt-4o-mini", "gpt-4o"]

Resources:
  # --- API Gateway ---
  AiApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      StageName: prod
      Description: AI Endpoint HTTP API
      CorsConfiguration:
        AllowOrigins:
          - !Ref AllowedOrigin
        AllowMethods:
          - POST
          - GET
          - OPTIONS
        AllowHeaders:
          - Content-Type
          - X-Api-Key
        MaxAge: 3600
      RouteSettings:
        "POST /ask":
          ThrottlingBurstLimit: 50
          ThrottlingRateLimit: 100
        "GET /health":
          ThrottlingBurstLimit: 200
          ThrottlingRateLimit: 500

  # --- Ask Function ---
  AskFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: ai-endpoint-ask
      Handler: handler.handler
      CodeUri: ./src/
      MemorySize: 768
      Timeout: 60
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAiApiKey
          MODEL_NAME: !Ref ModelName
          LAMBDA_TIMEOUT: "60"
          ALLOWED_ORIGIN: !Ref AllowedOrigin
          API_KEYS: !Ref ApiKeys
          LOG_LEVEL: INFO
          DEFAULT_MAX_TOKENS: "500"
          MAX_TOKENS_LIMIT: "2000"
          MAX_PROMPT_LENGTH: "10000"
          DEFAULT_SYSTEM_PROMPT: "Respond concisely and helpfully."
      Events:
        AskRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /ask
            Method: POST

  # --- Health Function ---
  HealthFunction:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: ai-endpoint-health
      Handler: health.handler
      CodeUri: ./src/
      MemorySize: 128
      Timeout: 5
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAiApiKey
          MODEL_NAME: !Ref ModelName
          LAMBDA_TIMEOUT: "60"
          ALLOWED_ORIGIN: !Ref AllowedOrigin
      Events:
        HealthRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /health
            Method: GET

Outputs:
  ApiUrl:
    Description: Base URL of the API Gateway
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod"
  AskEndpoint:
    Description: POST /ask endpoint
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod/ask"
  HealthEndpoint:
    Description: GET /health endpoint
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod/health"
  AskFunctionArn:
    Description: ARN of the Ask function
    Value: !GetAtt AskFunction.Arn

samconfig.toml

version = 0.1

[default.deploy.parameters]
stack_name = "ai-endpoint"
resolve_s3 = true
s3_prefix = "ai-endpoint"
region = "us-east-1"
confirm_changeset = true
capabilities = "CAPABILITY_IAM"
parameter_overrides = "ModelName=gpt-4o-mini AllowedOrigin=*"

env.json (for sam local)

{
  "AskFunction": {
    "OPENAI_API_KEY": "sk-proj-your-key-here",
    "MODEL_NAME": "gpt-4o-mini",
    "LAMBDA_TIMEOUT": "60",
    "ALLOWED_ORIGIN": "*",
    "API_KEYS": "",
    "LOG_LEVEL": "DEBUG",
    "DEFAULT_MAX_TOKENS": "500",
    "MAX_TOKENS_LIMIT": "2000",
    "MAX_PROMPT_LENGTH": "10000",
    "DEFAULT_SYSTEM_PROMPT": "Respond concisely and helpfully."
  },
  "HealthFunction": {
    "OPENAI_API_KEY": "sk-proj-your-key-here",
    "MODEL_NAME": "gpt-4o-mini",
    "LAMBDA_TIMEOUT": "60",
    "ALLOWED_ORIGIN": "*"
  }
}

events/ask.json

{
  "version": "2.0",
  "routeKey": "POST /ask",
  "rawPath": "/prod/ask",
  "headers": {
    "content-type": "application/json",
    "x-api-key": ""
  },
  "requestContext": {
    "http": {
      "method": "POST",
      "path": "/prod/ask",
      "sourceIp": "127.0.0.1"
    },
    "time": "08/Mar/2026:12:00:00 +0000",
    "requestId": "test-request-001"
  },
  "body": "{\"prompt\": \"What is serverless computing? Explain in 3 points.\", \"max_tokens\": 300}",
  "isBase64Encoded": false
}

events/health.json

{
  "version": "2.0",
  "routeKey": "GET /health",
  "rawPath": "/prod/health",
  "headers": {},
  "requestContext": {
    "http": {
      "method": "GET",
      "path": "/prod/health",
      "sourceIp": "127.0.0.1"
    },
    "time": "08/Mar/2026:12:00:00 +0000",
    "requestId": "test-request-002"
  },
  "isBase64Encoded": false
}

tests/test_handler.py

import json
import os
import pytest

os.environ["OPENAI_API_KEY"] = "sk-test-fake-key-for-testing"
os.environ["MODEL_NAME"] = "gpt-4o-mini"
os.environ["LAMBDA_TIMEOUT"] = "60"
os.environ["ALLOWED_ORIGIN"] = "*"
os.environ["API_KEYS"] = ""

from src.handler import _validate_request, _check_auth, _response


class TestValidation:
    def test_empty_prompt_rejected(self):
        error = _validate_request({"prompt": ""})
        assert error is not None
        assert "required" in error

    def test_missing_prompt_rejected(self):
        error = _validate_request({})
        assert error is not None

    def test_long_prompt_rejected(self):
        error = _validate_request({"prompt": "x" * 10001})
        assert error is not None
        assert "too long" in error

    def test_valid_prompt_accepted(self):
        error = _validate_request({"prompt": "Hello"})
        assert error is None

    def test_max_tokens_validation(self):
        error = _validate_request({"prompt": "Hello", "max_tokens": 5000})
        assert error is not None
        assert "max_tokens" in error

    def test_valid_max_tokens(self):
        error = _validate_request({"prompt": "Hello", "max_tokens": 500})
        assert error is None


class TestAuth:
    def test_no_auth_configured(self):
        os.environ["API_KEYS"] = ""
        assert _check_auth({"headers": {}}) is True

    def test_valid_key(self):
        os.environ["API_KEYS"] = "key1,key2"
        assert _check_auth({"headers": {"x-api-key": "key1"}}) is True

    def test_invalid_key(self):
        os.environ["API_KEYS"] = "key1,key2"
        assert _check_auth({"headers": {"x-api-key": "wrong"}}) is False

    def test_missing_key(self):
        os.environ["API_KEYS"] = "key1"
        assert _check_auth({"headers": {}}) is False


class TestResponse:
    def test_response_format(self):
        resp = _response(200, {"answer": "test"})
        assert resp["statusCode"] == 200
        assert "Content-Type" in resp["headers"]
        assert "Access-Control-Allow-Origin" in resp["headers"]
        body = json.loads(resp["body"])
        assert body["answer"] == "test"

    def test_cors_headers_present(self):
        resp = _response(200, {"ok": True})
        assert "Access-Control-Allow-Origin" in resp["headers"]
        assert "Access-Control-Allow-Methods" in resp["headers"]

Step by Step to Build

1. Create the project structure (2 min)

mkdir -p lambda-ai-endpoint/src
mkdir -p lambda-ai-endpoint/events
mkdir -p lambda-ai-endpoint/tests
cd lambda-ai-endpoint

2. Create the code files (10 min)

Copy the files from the previous sections:

# Create handler.py, health.py, requirements.txt in src/
# Create template.yaml, samconfig.toml, env.json in the root
# Create ask.json, health.json in events/
# Create test_handler.py in tests/

3. Install dependencies (2 min)

# For local development
pip install openai pytest

# SAM CLI (if you don't have it)
pip install aws-sam-cli

4. Configure the environment (2 min)

# Edit env.json with your real OPENAI_API_KEY
# This file is for `sam local invoke` — do NOT push it to Git
echo "env.json" >> .gitignore

5. Unit tests (3 min)

# Run the validation and auth tests
cd lambda-ai-endpoint
PYTHONPATH=. pytest tests/test_handler.py -v

# Expected result:
# test_empty_prompt_rejected PASSED
# test_missing_prompt_rejected PASSED
# test_long_prompt_rejected PASSED
# test_valid_prompt_accepted PASSED
# test_max_tokens_validation PASSED
# test_valid_max_tokens PASSED
# test_no_auth_configured PASSED
# test_valid_key PASSED
# test_invalid_key PASSED
# test_missing_key PASSED
# test_response_format PASSED
# test_cors_headers_present PASSED

6. Local test with SAM (5 min)

# Individual test of the handler
sam local invoke AskFunction \
  --event events/ask.json \
  --env-vars env.json

# Test the health check
sam local invoke HealthFunction \
  --event events/health.json \
  --env-vars env.json

# Test with local API Gateway
sam local start-api --env-vars env.json
# → API running at http://127.0.0.1:3000

# In another terminal:
curl http://127.0.0.1:3000/health
curl -X POST http://127.0.0.1:3000/ask \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What is serverless?", "max_tokens": 200}'

7. Preview with LocalStack (5 min)

In Module 4 you'll do this in detail, but you can try it now as a preview:

# Start LocalStack
docker run -d --name localstack \
  -p 4566:4566 \
  -e SERVICES=lambda,apigateway \
  localstack/localstack

# Wait for it to start
sleep 10
curl http://localhost:4566/_localstack/health

# Deploy the Lambda on LocalStack
samlocal deploy \
  --stack-name ai-endpoint \
  --resolve-s3 \
  --parameter-overrides \
    OpenAiApiKey=sk-proj-your-key \
    AllowedOrigin=* \
  --no-confirm-changeset

# samlocal is SAM configured to use LocalStack
# If you don't have it: pip install aws-sam-cli-local

8. Build and package (3 min)

# Build (installs dependencies in a container)
sam build

# Verify the build is correct
ls .aws-sam/build/AskFunction/
# handler.py  health.py  requirements.txt  openai/  ...

9. Deploy config (only if you have an AWS account)

# If you have an AWS account and want to deploy:
sam deploy --guided

# It will ask you:
# Stack Name: ai-endpoint
# AWS Region: us-east-1
# Parameter OpenAiApiKey: sk-proj-...
# Parameter AllowedOrigin: *
# Parameter ApiKeys: (enter for no auth)
# Parameter ModelName: gpt-4o-mini
# Confirm changes? [y/N]: y

# If you do NOT have an AWS account: use LocalStack (Module 4)

10. Verify the deployment (3 min)

# Get the API URL
aws cloudformation describe-stacks \
  --stack-name ai-endpoint \
  --query 'Stacks[0].Outputs'

# Or if you deployed with sam deploy:
# The URL appears in the Outputs at the end of the deploy

# Test health
curl https://YOUR-API-ID.execute-api.us-east-1.amazonaws.com/prod/health

# Test ask
curl -X POST https://YOUR-API-ID.execute-api.us-east-1.amazonaws.com/prod/ask \
  -H "Content-Type: application/json" \
  -d '{"prompt": "What is Lambda?", "max_tokens": 200}'

Cold Start Mitigation Applied

What we already did in the code

# 1. Imports outside the handler (they run at init, only once)
import json, logging, os, time
from openai import OpenAI

# 2. Client initialized outside the handler (reused between invocations)
client = OpenAI(api_key=os.environ.get("OPENAI_API_KEY", ""))

# 3. Global variables computed once
CORS_HEADERS = {...}
IS_COLD_START = True

# 4. Only the necessary dependency in requirements.txt (openai)
# We don't include langchain, pandas, numpy — they increase cold start

arm64 for a shorter cold start

# template.yaml — arm64 reduces cold start and cost
Globals:
  Function:
    Architectures:
      - arm64  # ~5-10% less cold start + 20% less cost

Provisioned Concurrency (optional)

# Only if cold starts are unacceptable for your case
# Add this to AskFunction in template.yaml:

  AskFunction:
    Properties:
      ProvisionedConcurrencyConfig:
        ProvisionedConcurrentExecutions: 2
      # Keeps 2 instances warm 24/7
      # Additional cost: ~$16/month (2 × 768MB × 30 days)

Timeout and Memory Configured for AI

Justification of the chosen values

Ask Function: 768MB, 60s
├── Memory: 768MB
│   ├── Enough CPU for TLS handshake and JSON parsing
│   ├── Doesn't waste on CPU it doesn't use (I/O bound)
│   └── Cold start: ~2-3s (vs ~8s with 128MB)
├── Timeout: 60s
│   ├── OpenAI client timeout: 50s (60 - 10 buffer)
│   ├── Allows responses of up to ~2000 tokens in gpt-4o-mini
│   └── API Gateway limit is 29s → Lambda has margin
└── arm64: 20% cheaper, equivalent performance

Health Function: 128MB, 5s
├── Memory: 128MB (doesn't do heavy work)
├── Timeout: 5s (just reads env vars and returns JSON)
└── Lowest possible cost

The API Gateway constraint: 29 seconds

API Gateway timeout: 29s (hard limit, not configurable)
Lambda timeout: 60s (what we configured)

Why do we configure Lambda to 60s if API Gateway cuts off at 29s?

1. For direct invocation (without API Gateway), the 60s timeout applies
2. For async invocations, Lambda can run the full 60s
3. The OpenAI client timeout is 50s as a safety net
4. In practice, gpt-4o-mini responds in 1-8s → well within the limit

If an invocation takes >29s via API Gateway:
├── API Gateway returns 504 to the client
├── Lambda KEEPS executing until its timeout or until it finishes
└── The response is lost (Lambda doesn't know Gateway already cut off)

Mitigation: the handler checks remaining_ms and uses a fast model
if little time is left.

Cost Estimation for This Endpoint

Scenario: development/learning endpoint

Parameters:
├── 50 invocations/day (manual testing + demos)
├── 768MB, 4s avg (gpt-4o-mini responds fast)
├── arm64

Lambda:
├── 1,500 inv/month (free tier: 1M free) → $0.00
├── 4,500 GB-s (free tier: 400K free) → $0.00
└── Total Lambda: $0.00

OpenAI:
├── 1,500 × 100 input tokens × $0.15/1M = $0.02
├── 1,500 × 300 output tokens × $0.60/1M = $0.27
└── Total OpenAI: $0.29/month

Others: API Gateway $0.00, CloudWatch $0.50

═══════════════════════════════════
TOTAL: ~$0.79/month
═══════════════════════════════════

Scenario: light production endpoint

Parameters:
├── 5,000 invocations/day
├── 768MB, 5s avg
├── arm64

Lambda:
├── 150,000 inv/month → requests: $0.00 (free tier)
├── 562,500 GB-s → billable: 162,500 × $0.0000133334 = $2.17
└── Total Lambda: $2.17

OpenAI:
├── 150,000 × 100 × $0.15/1M = $2.25
├── 150,000 × 300 × $0.60/1M = $27.00
└── Total OpenAI: $29.25/month

Others: API Gateway $0.15, CloudWatch $1.50

═══════════════════════════════════
TOTAL: ~$33.07/month
├── Lambda + infra: $3.82 (12%)
├── OpenAI: $29.25 (88%)
═══════════════════════════════════

Completeness Checklist

Code and structure

src/handler.py implements POST /ask with complete validation
src/health.py implements GET /health with configuration checks
src/requirements.txt contains only openai>=1.0.0
template.yaml defines both functions with API Gateway HTTP API v2
samconfig.toml configures deploy parameters
env.json exists with local variables (NOT in Git)
events/ask.json and events/health.json exist for testing

Functionality

POST /ask receives a prompt and returns the LLM response
POST /ask validates input (empty prompt, max_tokens out of range, long prompt)
POST /ask returns metadata: model, tokens_used, duration_ms, cold_start
GET /health returns Lambda status and configuration
CORS headers present in all responses
API key auth works when API_KEYS is configured
No auth when API_KEYS is empty (development)
LLM errors return 502 with a descriptive message
LLM timeout returns an error before Lambda dies

Configuration

Ask Function: 768MB memory, 60s timeout, arm64
Health Function: 128MB memory, 5s timeout, arm64
OpenAI client timeout = Lambda timeout - 10s
API Gateway throttling configured (50 burst, 100 rate for /ask)
CORS configured for a configurable AllowedOrigin
IS_COLD_START tracking implemented
Structured logging (JSON) for CloudWatch Insights

Testing

Unit tests pass: validation, auth, response format
sam local invoke works for both functions
sam local start-api starts and responds correctly
(Optional) Deploy to AWS works without errors
(Optional) LocalStack preview works

Optimization

Only openai as a dependency (minimum cold start)
Client and imports outside the handler (reused between invocations)
arm64 configured (20% cheaper)
Handler adapts model/tokens if little time is left (remaining_ms)

Project Troubleshooting

"sam local invoke fails with ModuleNotFoundError: openai"

# sam local invoke runs in a Docker container
# It needs to build first to install dependencies
sam build
sam local invoke AskFunction --event events/ask.json --env-vars env.json

"The handler returns null or hangs"

# Verify that env.json has the correct variables
# Especially OPENAI_API_KEY with a real key
cat env.json | python3 -m json.tool

# Verify that the event has the correct format
cat events/ask.json | python3 -m json.tool

# If it hangs: the OpenAI SDK is waiting for a response
# Check your API key and internet connection

"CORS error when calling from the browser"

# Verify that CorsConfiguration is in template.yaml
# Verify that AllowedOrigin includes your origin
# If you test from localhost:3000, AllowedOrigin should be
# "http://localhost:3000" or "*"

# Manual test:
curl -v -X OPTIONS http://127.0.0.1:3000/ask \
  -H "Origin: http://localhost:3000" \
  -H "Access-Control-Request-Method: POST"

"sam deploy fails with 'Unable to upload artifact'"

# SAM needs an S3 bucket to upload the code
# With --guided, SAM asks if you want it to create one
sam deploy --guided

# If you already have a bucket:
sam deploy --s3-bucket your-bucket --no-confirm-changeset

Connection with the Guide

What you built

Lambda AI Endpoint
├── Ask Function (POST /ask)
│   ├── Input validation
│   ├── API-key auth
│   ├── Invocation to GPT-4o-mini
│   ├── Adaptive timeout
│   ├── Cold start tracking
│   └── Structured logging
├── Health Function (GET /health)
│   └── Status and configuration
├── API Gateway HTTP API v2
│   ├── CORS configured
│   └── Per-route throttling
└── Complete SAM Template

What comes next in the following modules

Module 4 (LocalStack — AWS Local Development):
├── This Lambda deploys on LocalStack
├── Same code, same template, but local and free
├── You add LocalStack as a service in your Docker Compose (from M2)
└── You develop and test Lambda without an AWS account

Module 5 (AWS Services — S3 + Lambda):
├── You add S3 to persist prompts and responses
├── Lambda trigger: S3 event → process file → save result
└── You integrate with the existing endpoint

Module 6 (Cloud Migration):
├── Your Lambda works the same in LocalStack and in AWS
├── You learn to migrate from LocalStack to AWS with confidence
└── Environment abstraction: same code, different infrastructure

Module 8 (Integrator Project):
├── This Lambda can be part of your final system
├── Or you can choose another strategy (VPS, Render, etc.)
└── The decision matrix from M1 guides you

Summary

You built a complete Lambda AI Endpoint with POST /ask and GET /health, invoking GPT-4o-mini and returning structured responses.
You configured API Gateway HTTP API v2 with CORS, throttling, and optional API-key authentication.
You implemented cold-start mitigation: imports and client outside the handler, arm64, and IS_COLD_START tracking.
You deployed with a SAM template that includes both functions, their timeout configurations (768MB, 60s), and memory.
You documented the cost estimation for development (~~$0.79/month) and production (~~$33/month) scenarios.
You integrated input validation, adaptive timeout based on remaining_ms, and structured logging for CloudWatch Insights.

Project Resources

AWS SAM CLI Documentation — Complete SAM CLI reference
SAM Template Specification — Template specification
Lambda Python Handler — Handler in Python
HTTP API Payload Format — v2.0 event format
OpenAI Python SDK — Official SDK
LocalStack SAM Integration — Using SAM with LocalStack
Lambda Best Practices — Official best practices
Lambda Pricing Calculator — Calculate costs

← Previous7. Cost Estimation for Serverless AI Next →1. Introduction: LocalStack — AWS Local Development