Module 3: Serverless & Lambda for AI

6. API Gateway Integration

Next →

Overview

In this capsule you'll connect your Lambda to the internet through API Gateway. A Lambda without an HTTP trigger is a function no one can call — API Gateway is what turns your function into a real endpoint. By the end, you'll have a POST endpoint that receives a prompt, invokes your AI Lambda, and returns the response with CORS configured, basic authentication, and rate limiting.

Context: API Gateway is the AWS service that exposes your Lambdas as HTTP endpoints. There are two versions: REST API (v1) and HTTP API (v2). For AI endpoints, HTTP API is almost always the right choice — cheaper, simpler, and with better performance. This capsule covers both so you understand when to use each one, but the project uses HTTP API.

API Gateway as an HTTP Trigger

The complete flow

Client (browser, app, curl)
    │
    │  POST /ask  {"prompt": "..."}
    ▼
┌──────────────┐
│ API Gateway  │  ← Receives HTTP, validates, transforms
│  (HTTP API)  │
└──────┬───────┘
       │  Invokes Lambda (synchronous)
       ▼
┌──────────────┐
│   Lambda     │  ← Runs handler, calls OpenAI
│ (ai-endpoint)│
└──────┬───────┘
       │  Returns response
       ▼
┌──────────────┐
│ API Gateway  │  ← Transforms response, adds headers
└──────┬───────┘
       │  HTTP 200 {"answer": "..."}
       ▼
Client receives response

Latency of the flow

Component           Typical latency
────────────────────────────────────
API Gateway         5-15ms
Lambda cold start   1-8s (first time)
Lambda warm         <50ms
LLM API call        1-20s (depending on model/tokens)
API Gateway return  3-10ms
────────────────────────────────────
Total (warm):       1.5-20.5s
Total (cold):       2.5-28s

REST API vs HTTP API (v2)

Direct comparison

Feature                REST API (v1)      HTTP API (v2)
──────────────────────────────────────────────────────────
Price (per million)    $3.50              $1.00
Latency overhead       15-30ms            5-10ms
WebSocket support      ✅                  ✅
Request validation     ✅ (built-in)       ❌ (in your code)
Usage plans/API keys   ✅ (native)         ❌ (manual)
Custom authorizers     ✅ (Lambda + IAM)   ✅ (Lambda + JWT)
Request/response map   ✅ (VTL templates)  ❌
WAF integration        ✅                  ❌
Caching                ✅ (built-in)       ❌
CORS                   ✅ (manual config)  ✅ (simple config)
Payload max            10 MB              10 MB
Timeout max            29 seconds         29 seconds ⚠️

When to use each one

Use HTTP API (v2) when:
├── Your AI endpoint is simple: receives prompt → returns response
├── You don't need caching in Gateway (you do it in Lambda/Redis)
├── You want to minimize costs (3.5x cheaper)
├── Latency matters (less overhead)
└── 90% of the cases in this guide

Use REST API (v1) when:
├── You need native API keys with usage plans
├── You need request validation in Gateway (before invoking Lambda)
├── You need WAF (Web Application Firewall) for compliance
├── You need caching in Gateway to reduce Lambda invocations
└── Enterprise with granular access controls

The 29-second limit

⚠️ Both versions of API Gateway have a maximum timeout of 29 seconds. This is a hard limit you can't change. Your Lambda can have a 15-minute timeout, but if you invoke it via API Gateway, it must respond in under 29 seconds.

Implications for AI:
├── Simple calls to gpt-4o-mini (1-3s): ✅ no problem
├── Calls to gpt-4o with many tokens (5-15s): ✅ generally OK
├── Prompt chains (3+ sequential calls): ⚠️ risky
├── Complex RAG pipeline: ⚠️ may exceed 29s
└── Batch processing: ❌ don't use API Gateway, use direct invocation

For workloads >29s:
├── Async pattern: API Gateway starts Lambda → returns requestId
│   Lambda processes in the background → result in S3/DynamoDB
│   Client polls with GET /status/{requestId}
└── Direct invocation: aws lambda invoke (without Gateway)

Configure HTTP API (v2)

With a SAM template

# template.yaml
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: python3.11
    Architectures: [arm64]

Resources:
  AiApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      StageName: prod
      CorsConfiguration:
        AllowOrigins:
          - "https://yourapp.com"
          - "http://localhost:3000"
        AllowMethods:
          - POST
          - GET
          - OPTIONS
        AllowHeaders:
          - Content-Type
          - Authorization
          - X-Api-Key
        MaxAge: 3600

  AskFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handler.handler
      CodeUri: ./src/
      MemorySize: 768
      Timeout: 60
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAiApiKey
          MODEL_NAME: gpt-4o-mini
      Events:
        AskRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /ask
            Method: POST
        HealthRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /health
            Method: GET

  HealthFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: health.handler
      CodeUri: ./src/
      MemorySize: 128
      Timeout: 5
      Events:
        HealthRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /health
            Method: GET

Parameters:
  OpenAiApiKey:
    Type: String
    NoEcho: true

Outputs:
  ApiUrl:
    Description: URL of the API Gateway
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod"

With the AWS CLI

# Create HTTP API
aws apigatewayv2 create-api \
  --name ai-endpoint-api \
  --protocol-type HTTP \
  --cors-configuration '{
    "AllowOrigins": ["http://localhost:3000"],
    "AllowMethods": ["POST", "GET", "OPTIONS"],
    "AllowHeaders": ["Content-Type", "Authorization"],
    "MaxAge": 3600
  }'

# Create the Lambda integration
aws apigatewayv2 create-integration \
  --api-id API_ID \
  --integration-type AWS_PROXY \
  --integration-uri arn:aws:lambda:us-east-1:ACCOUNT:function:ai-endpoint \
  --payload-format-version 2.0

# Create the POST /ask route
aws apigatewayv2 create-route \
  --api-id API_ID \
  --route-key "POST /ask" \
  --target "integrations/INTEGRATION_ID"

# Create the stage and deploy
aws apigatewayv2 create-stage \
  --api-id API_ID \
  --stage-name prod \
  --auto-deploy

CORS: Why your frontend gets errors

The problem

Your frontend (localhost:3000):
  fetch("https://xyz.execute-api.us-east-1.amazonaws.com/prod/ask", {
    method: "POST",
    body: JSON.stringify({prompt: "Hi"})
  })

Without CORS configured:
  → Browser sends a preflight OPTIONS request
  → API Gateway doesn't know what to answer
  → Browser blocks the response
  → Console: "Access to fetch has been blocked by CORS policy"

CORS configuration in HTTP API

# In the SAM template (the simplest way)
AiApi:
  Type: AWS::Serverless::HttpApi
  Properties:
    CorsConfiguration:
      AllowOrigins:
        - "http://localhost:3000"
        - "https://yourapp.com"
      AllowMethods:
        - POST
        - GET
        - OPTIONS
      AllowHeaders:
        - Content-Type
        - Authorization
      MaxAge: 3600  # Browser caches the preflight for 1 hour

CORS headers in your Lambda

HTTP API handles CORS automatically if you configure it in the template. But if you use REST API or want explicit control:

def handler(event, context):
    headers = {
        "Content-Type": "application/json",
        "Access-Control-Allow-Origin": os.environ.get("ALLOWED_ORIGIN", "*"),
        "Access-Control-Allow-Methods": "POST, GET, OPTIONS",
        "Access-Control-Allow-Headers": "Content-Type, Authorization",
    }

    # Preflight request
    if event.get("requestContext", {}).get("http", {}).get("method") == "OPTIONS":
        return {"statusCode": 200, "headers": headers, "body": ""}

    # Your normal logic
    body = json.loads(event.get("body", "{}"))
    # ... LLM call ...

    return {
        "statusCode": 200,
        "headers": headers,
        "body": json.dumps({"answer": "..."})
    }

Debugging CORS

# Simulate the preflight from the terminal
curl -v -X OPTIONS \
  https://xyz.execute-api.us-east-1.amazonaws.com/prod/ask \
  -H "Origin: http://localhost:3000" \
  -H "Access-Control-Request-Method: POST" \
  -H "Access-Control-Request-Headers: Content-Type"

# You should see in the response:
# Access-Control-Allow-Origin: http://localhost:3000
# Access-Control-Allow-Methods: POST, GET, OPTIONS
# Access-Control-Allow-Headers: Content-Type

Request/Response in API Gateway v2

Lambda event format (payload v2.0)

# What your Lambda receives from HTTP API (payload format 2.0):
event = {
    "version": "2.0",
    "routeKey": "POST /ask",
    "rawPath": "/prod/ask",
    "headers": {
        "content-type": "application/json",
        "authorization": "Bearer sk-...",
        "x-forwarded-for": "203.0.113.1",
    },
    "requestContext": {
        "http": {
            "method": "POST",
            "path": "/prod/ask",
            "sourceIp": "203.0.113.1",
        },
        "time": "08/Mar/2026:12:00:00 +0000",
        "requestId": "abc123",
    },
    "body": "{\"prompt\": \"What is serverless?\", \"max_tokens\": 500}",
    "isBase64Encoded": False,
}

Handler that parses correctly

import json
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=25)

CORS_HEADERS = {
    "Content-Type": "application/json",
    "Access-Control-Allow-Origin": os.environ.get("ALLOWED_ORIGIN", "*"),
}

def handler(event, context):
    method = event.get("requestContext", {}).get("http", {}).get("method", "")

    if method == "OPTIONS":
        return {"statusCode": 200, "headers": CORS_HEADERS, "body": ""}

    try:
        body = json.loads(event.get("body", "{}"))
    except json.JSONDecodeError:
        return {
            "statusCode": 400,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": "Invalid JSON body"})
        }

    prompt = body.get("prompt", "").strip()
    if not prompt:
        return {
            "statusCode": 400,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": "prompt is required"})
        }

    max_tokens = min(body.get("max_tokens", 500), 2000)

    try:
        response = client.chat.completions.create(
            model=os.environ.get("MODEL_NAME", "gpt-4o-mini"),
            messages=[
                {"role": "system", "content": "Respond concisely and helpfully."},
                {"role": "user", "content": prompt}
            ],
            max_tokens=max_tokens,
        )

        return {
            "statusCode": 200,
            "headers": CORS_HEADERS,
            "body": json.dumps({
                "answer": response.choices[0].message.content,
                "model": response.model,
                "tokens_used": response.usage.total_tokens,
            })
        }

    except Exception as e:
        return {
            "statusCode": 502,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": f"LLM call failed: {str(e)}"})
        }

Response format

# What your Lambda must return:
{
    "statusCode": 200,
    "headers": {
        "Content-Type": "application/json",
        "Access-Control-Allow-Origin": "http://localhost:3000"
    },
    "body": "{\"answer\": \"Serverless is...\", \"tokens_used\": 42}"
}

# API Gateway takes this and builds the HTTP response for the client.
# The body MUST be a string (serialized JSON), not a dict.

Authentication

Option 1: API Key (simple)

To protect your endpoint without complex infrastructure:

import os
import json

VALID_API_KEYS = set(os.environ.get("API_KEYS", "").split(","))

def handler(event, context):
    api_key = event.get("headers", {}).get("x-api-key", "")

    if api_key not in VALID_API_KEYS:
        return {
            "statusCode": 401,
            "body": json.dumps({"error": "Invalid or missing API key"})
        }

    # ... rest of the handler
# Invoke with an API key
curl -X POST https://xyz.execute-api.us-east-1.amazonaws.com/prod/ask \
  -H "Content-Type: application/json" \
  -H "x-api-key: your-api-key-here" \
  -d '{"prompt": "Hi"}'

Option 2: JWT Authorizer (HTTP API v2)

# SAM template with a JWT authorizer
Resources:
  AiApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      Auth:
        DefaultAuthorizer: JwtAuthorizer
        Authorizers:
          JwtAuthorizer:
            AuthorizationScopes:
              - ai.invoke
            IdentitySource: "$request.header.Authorization"
            JwtConfiguration:
              issuer: "https://cognito-idp.us-east-1.amazonaws.com/us-east-1_POOLID"
              audience:
                - "your-client-id"

  AskFunction:
    Type: AWS::Serverless::Function
    Properties:
      Events:
        AskRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /ask
            Method: POST
            # The JWT authorizer is applied automatically
        HealthRoute:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /health
            Method: GET
            Auth:
              Authorizer: NONE  # Health check without auth

Option 3: IAM Authorization

# Invoke with AWS credentials (service-to-service)
aws lambda invoke \
  --function-name ai-endpoint \
  --payload '{"body": "{\"prompt\": \"Hi\"}"}' \
  response.json

# Or with Signature V4 for HTTP
# Useful when another AWS service invokes your endpoint

Recommendation for this guide

For development and learning:
└── API key in a header (Option 1) — simple, functional

For production:
├── JWT with Cognito/Auth0 (Option 2) — if you have users
└── IAM (Option 3) — if it's service-to-service

Rate Limiting and Throttling

Throttling in HTTP API

HTTP API v2 defaults:
├── Account-level: 10,000 requests/second
├── Route-level: configurable
└── Burst: 5,000 concurrent

For AI endpoints, 10K/s is more than enough.
Your bottleneck is OpenAI rate limits, not API Gateway.

Configure throttling per route

# SAM template
Resources:
  AiApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      StageName: prod
      RouteSettings:
        "POST /ask":
          ThrottlingBurstLimit: 50    # Maximum concurrent
          ThrottlingRateLimit: 100    # Requests per second
        "GET /health":
          ThrottlingBurstLimit: 200
          ThrottlingRateLimit: 500

Rate limiting in your Lambda

API Gateway throttling protects your Lambda, but it doesn't protect your OpenAI account. Implement your own rate limiting:

import json
import os
import time
import hashlib
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=25)

MAX_REQUESTS_PER_MINUTE = int(os.environ.get("RATE_LIMIT", "30"))

# In production, use Redis/DynamoDB for distributed rate limiting
# This is a simplified version for a single Lambda instance
request_log = {}

def check_rate_limit(client_ip):
    now = time.time()
    window_start = now - 60

    if client_ip not in request_log:
        request_log[client_ip] = []

    request_log[client_ip] = [
        t for t in request_log[client_ip] if t > window_start
    ]

    if len(request_log[client_ip]) >= MAX_REQUESTS_PER_MINUTE:
        return False

    request_log[client_ip].append(now)
    return True

def handler(event, context):
    client_ip = (
        event.get("requestContext", {})
        .get("http", {})
        .get("sourceIp", "unknown")
    )

    if not check_rate_limit(client_ip):
        return {
            "statusCode": 429,
            "body": json.dumps({
                "error": "Rate limit exceeded",
                "retry_after_seconds": 60
            })
        }

    # ... rest of the handler

Complete Example: API Gateway → Lambda → OpenAI

Project structure

lambda-ai-api/
├── src/
│   ├── handler.py       # POST /ask handler
│   ├── health.py        # GET /health handler
│   └── requirements.txt
├── template.yaml        # SAM template
└── samconfig.toml       # SAM deploy config

src/handler.py

import json
import os
import time
import logging
from openai import OpenAI

logger = logging.getLogger()
logger.setLevel(os.environ.get("LOG_LEVEL", "INFO"))

client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"],
    timeout=int(os.environ.get("LLM_TIMEOUT", "25")),
    max_retries=1,
)

CORS_HEADERS = {
    "Content-Type": "application/json",
    "Access-Control-Allow-Origin": os.environ.get("ALLOWED_ORIGIN", "*"),
    "Access-Control-Allow-Methods": "POST, OPTIONS",
    "Access-Control-Allow-Headers": "Content-Type, X-Api-Key",
}

IS_COLD_START = True

def handler(event, context):
    global IS_COLD_START
    was_cold = IS_COLD_START
    IS_COLD_START = False

    start = time.time()

    method = event.get("requestContext", {}).get("http", {}).get("method", "")
    if method == "OPTIONS":
        return {"statusCode": 200, "headers": CORS_HEADERS, "body": ""}

    api_key = event.get("headers", {}).get("x-api-key", "")
    valid_keys = set(os.environ.get("API_KEYS", "").split(","))
    if valid_keys != {""} and api_key not in valid_keys:
        return {
            "statusCode": 401,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": "Unauthorized"})
        }

    try:
        body = json.loads(event.get("body", "{}"))
    except json.JSONDecodeError:
        return {
            "statusCode": 400,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": "Invalid JSON"})
        }

    prompt = body.get("prompt", "").strip()
    if not prompt:
        return {
            "statusCode": 400,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": "prompt is required"})
        }

    if len(prompt) > 10000:
        return {
            "statusCode": 400,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": "prompt too long (max 10000 chars)"})
        }

    max_tokens = min(body.get("max_tokens", 500), 2000)
    model = os.environ.get("MODEL_NAME", "gpt-4o-mini")

    remaining_ms = context.get_remaining_time_in_millis()
    if remaining_ms < 10000:
        model = "gpt-4o-mini"
        max_tokens = min(max_tokens, 200)

    try:
        llm_start = time.time()
        response = client.chat.completions.create(
            model=model,
            messages=[
                {"role": "system", "content": "Respond concisely and helpfully."},
                {"role": "user", "content": prompt}
            ],
            max_tokens=max_tokens,
        )
        llm_ms = round((time.time() - llm_start) * 1000)

    except Exception as e:
        logger.error(f"LLM error: {e}")
        return {
            "statusCode": 502,
            "headers": CORS_HEADERS,
            "body": json.dumps({"error": f"LLM call failed: {str(e)}"})
        }

    total_ms = round((time.time() - start) * 1000)

    logger.info(json.dumps({
        "event": "ask",
        "cold_start": was_cold,
        "llm_ms": llm_ms,
        "total_ms": total_ms,
        "tokens": response.usage.total_tokens,
        "model": model,
    }))

    return {
        "statusCode": 200,
        "headers": CORS_HEADERS,
        "body": json.dumps({
            "answer": response.choices[0].message.content,
            "model": response.model,
            "tokens_used": response.usage.total_tokens,
            "duration_ms": total_ms,
            "cold_start": was_cold,
        })
    }

src/health.py

import json
import os
import time

def handler(event, context):
    start = time.time()

    checks = {"lambda": "up"}

    openai_key = os.environ.get("OPENAI_API_KEY", "")
    if openai_key and not openai_key.startswith("sk-"):
        checks["openai_key"] = "invalid_format"
    elif openai_key:
        checks["openai_key"] = "configured"
    else:
        checks["openai_key"] = "missing"

    overall = "healthy" if checks["openai_key"] == "configured" else "degraded"

    return {
        "statusCode": 200 if overall == "healthy" else 503,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({
            "status": overall,
            "checks": checks,
            "region": os.environ.get("AWS_REGION", "unknown"),
            "function": os.environ.get("AWS_LAMBDA_FUNCTION_NAME", "unknown"),
            "memory_mb": os.environ.get("AWS_LAMBDA_FUNCTION_MEMORY_SIZE", "unknown"),
            "latency_ms": round((time.time() - start) * 1000, 1)
        })
    }

src/requirements.txt

openai>=1.0.0

template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: AI Endpoint with API Gateway

Globals:
  Function:
    Runtime: python3.11
    Architectures: [arm64]

Parameters:
  OpenAiApiKey:
    Type: String
    NoEcho: true
  AllowedOrigin:
    Type: String
    Default: "*"
  ApiKeys:
    Type: String
    Default: ""
    Description: Comma-separated API keys (empty = no auth)

Resources:
  AiApi:
    Type: AWS::Serverless::HttpApi
    Properties:
      StageName: prod
      CorsConfiguration:
        AllowOrigins:
          - !Ref AllowedOrigin
        AllowMethods: [POST, GET, OPTIONS]
        AllowHeaders: [Content-Type, X-Api-Key]
        MaxAge: 3600
      RouteSettings:
        "POST /ask":
          ThrottlingBurstLimit: 50
          ThrottlingRateLimit: 100

  AskFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: handler.handler
      CodeUri: ./src/
      MemorySize: 768
      Timeout: 60
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAiApiKey
          MODEL_NAME: gpt-4o-mini
          LLM_TIMEOUT: "25"
          ALLOWED_ORIGIN: !Ref AllowedOrigin
          API_KEYS: !Ref ApiKeys
          LOG_LEVEL: INFO
      Events:
        Ask:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /ask
            Method: POST

  HealthFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: health.handler
      CodeUri: ./src/
      MemorySize: 128
      Timeout: 5
      Environment:
        Variables:
          OPENAI_API_KEY: !Ref OpenAiApiKey
      Events:
        Health:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /health
            Method: GET

Outputs:
  ApiUrl:
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod"
  AskEndpoint:
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod/ask"
  HealthEndpoint:
    Value: !Sub "https://${AiApi}.execute-api.${AWS::Region}.amazonaws.com/prod/health"

Test the endpoint

# Deploy (if you have an AWS account)
sam build && sam deploy --guided

# Test health
curl https://xyz.execute-api.us-east-1.amazonaws.com/prod/health

# Test ask
curl -X POST https://xyz.execute-api.us-east-1.amazonaws.com/prod/ask \
  -H "Content-Type: application/json" \
  -H "x-api-key: your-key" \
  -d '{"prompt": "What is serverless?", "max_tokens": 200}'

# Test from JavaScript (browser)
# fetch("https://xyz.execute-api.us-east-1.amazonaws.com/prod/ask", {
#   method: "POST",
#   headers: {"Content-Type": "application/json", "x-api-key": "your-key"},
#   body: JSON.stringify({prompt: "Hi", max_tokens: 200})
# }).then(r => r.json()).then(console.log)

Troubleshooting

Problem 1: "CORS error in the browser"

# Symptom: "Access to fetch has been blocked by CORS policy"
# The preflight OPTIONS doesn't return the correct headers

# Diagnosis:
curl -v -X OPTIONS https://your-api.execute-api.us-east-1.amazonaws.com/prod/ask \
  -H "Origin: http://localhost:3000" \
  -H "Access-Control-Request-Method: POST"

# If you do NOT see Access-Control-Allow-Origin in the response:
# 1. Check CorsConfiguration in your template.yaml
# 2. HTTP API handles CORS automatically if it's configured
# 3. If you use REST API, you need to configure the OPTIONS method manually
# 4. Redeploy after changing the CORS config

Problem 2: "502 Bad Gateway" or "Internal Server Error"

# API Gateway can't invoke your Lambda or Lambda returned an error

# Diagnosis:
# 1. Check your Lambda's CloudWatch Logs
aws logs tail /aws/lambda/AskFunction --follow

# 2. Verify the response format is correct
# The body MUST be a string, not a dict
# ❌ {"statusCode": 200, "body": {"answer": "..."}}
# ✅ {"statusCode": 200, "body": "{\"answer\": \"...\"}"}

# 3. Check permissions: API Gateway needs permission to invoke Lambda
# SAM configures it automatically, but with manual CLI you need:
aws lambda add-permission \
  --function-name ai-endpoint \
  --statement-id apigateway \
  --action lambda:InvokeFunction \
  --principal apigateway.amazonaws.com

Problem 3: "Timeout 504 — Endpoint request timed out"

# API Gateway timeout is 29 seconds (hard limit)
# If your Lambda takes longer, Gateway returns 504

# Solutions:
# 1. Optimize your Lambda (faster model, fewer tokens)
# 2. Use the async pattern (return requestId, process in the background)
# 3. Invoke Lambda directly (without Gateway) for long workloads

# If the latency is between 25-29s, it's a timing problem.
# Set the OpenAI client timeout to 20s to detect
# the LLM timeout before Gateway cuts you off.

Problem 4: "Missing Authentication Token" (REST API)

# If you use REST API (v1) and see this error,
# you're probably invoking a path that doesn't exist.
# REST API returns 403 "Missing Authentication Token" for nonexistent routes
# (confusing, but that's how it works).

# Verify:
aws apigateway get-resources --rest-api-id API_ID
# Make sure your route exists and the method is configured

Problem 5: "The response arrives truncated"

# API Gateway has a payload limit of 10MB
# But the most common issue is that your Lambda returns a body
# that isn't a serialized string

# Verify that json.dumps() wraps the whole body:
# ✅ "body": json.dumps({"answer": answer, "tokens": 42})
# ❌ "body": {"answer": answer, "tokens": 42}

Hands-On Exercises

Exercise 1: Add a GET /models endpoint

Create a GET /models endpoint that returns the available models, their estimated cost per 1K tokens, and the allowed max_tokens. It requires no authentication.

See solution 
# src/models.py
import json

AVAILABLE_MODELS = {
    "gpt-4o-mini": {
        "cost_per_1k_input": 0.00015,
        "cost_per_1k_output": 0.0006,
        "max_tokens": 4096,
        "description": "Fast and cheap, good for most tasks"
    },
    "gpt-4o": {
        "cost_per_1k_input": 0.0025,
        "cost_per_1k_output": 0.01,
        "max_tokens": 4096,
        "description": "Most capable, higher cost"
    },
}

def handler(event, context):
    return {
        "statusCode": 200,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps({
            "models": AVAILABLE_MODELS,
            "default": "gpt-4o-mini"
        })
    }

In template.yaml, add:

  ModelsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: models.handler
      CodeUri: ./src/
      MemorySize: 128
      Timeout: 5
      Events:
        Models:
          Type: HttpApi
          Properties:
            ApiId: !Ref AiApi
            Path: /models
            Method: GET

Exercise 2: Implement request validation

Modify the /ask handler to validate: non-empty prompt, max_tokens between 1-2000, and that the requested model exists in the list of available models. Return descriptive errors with status 400.

See solution 
import json
import os
from openai import OpenAI

VALID_MODELS = {"gpt-4o-mini", "gpt-4o"}
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=25)

def validate_request(body):
    errors = []

    prompt = body.get("prompt", "").strip()
    if not prompt:
        errors.append("prompt is required and cannot be empty")
    elif len(prompt) > 10000:
        errors.append(f"prompt too long: {len(prompt)} chars (max 10000)")

    max_tokens = body.get("max_tokens", 500)
    if not isinstance(max_tokens, int) or max_tokens < 1 or max_tokens > 2000:
        errors.append("max_tokens must be integer between 1 and 2000")

    model = body.get("model", "gpt-4o-mini")
    if model not in VALID_MODELS:
        errors.append(f"model '{model}' not available. Valid: {', '.join(VALID_MODELS)}")

    return errors

def handler(event, context):
    try:
        body = json.loads(event.get("body", "{}"))
    except json.JSONDecodeError:
        return {
            "statusCode": 400,
            "body": json.dumps({"errors": ["Invalid JSON body"]})
        }

    errors = validate_request(body)
    if errors:
        return {
            "statusCode": 400,
            "body": json.dumps({"errors": errors})
        }

    response = client.chat.completions.create(
        model=body.get("model", "gpt-4o-mini"),
        messages=[{"role": "user", "content": body["prompt"]}],
        max_tokens=body.get("max_tokens", 500),
    )

    return {
        "statusCode": 200,
        "body": json.dumps({
            "answer": response.choices[0].message.content,
            "tokens_used": response.usage.total_tokens,
        })
    }

Exercise 3: Async pattern for long workloads

Implement two endpoints: POST /ask-async that starts processing and returns a request_id, and GET /status/{request_id} that returns the status. Use an in-memory dict as a simplified store (in production you would use DynamoDB).

See solution 
# src/ask_async.py
import json
import os
import uuid
import threading
import time
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"], timeout=50)

results_store = {}

def process_llm(request_id, prompt, max_tokens):
    try:
        results_store[request_id]["status"] = "processing"
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": prompt}],
            max_tokens=max_tokens,
        )
        results_store[request_id] = {
            "status": "completed",
            "answer": response.choices[0].message.content,
            "tokens_used": response.usage.total_tokens,
            "completed_at": time.time(),
        }
    except Exception as e:
        results_store[request_id] = {
            "status": "failed",
            "error": str(e),
        }

def handler(event, context):
    route = event.get("routeKey", "")

    if route.startswith("POST"):
        body = json.loads(event.get("body", "{}"))
        request_id = str(uuid.uuid4())[:8]

        results_store[request_id] = {"status": "queued"}

        thread = threading.Thread(
            target=process_llm,
            args=(request_id, body["prompt"], body.get("max_tokens", 500))
        )
        thread.start()

        return {
            "statusCode": 202,
            "body": json.dumps({
                "request_id": request_id,
                "status": "queued",
                "check_url": f"/status/{request_id}"
            })
        }

    elif route.startswith("GET"):
        request_id = event.get("pathParameters", {}).get("request_id", "")
        result = results_store.get(request_id)

        if not result:
            return {
                "statusCode": 404,
                "body": json.dumps({"error": "Request not found"})
            }

        status_code = 200 if result["status"] == "completed" else 202
        return {
            "statusCode": status_code,
            "body": json.dumps(result)
        }

In template.yaml:

  AsyncFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: ask_async.handler
      CodeUri: ./src/
      MemorySize: 768
      Timeout: 120
      Events:
        Submit:
          Type: HttpApi
          Properties:
            Path: /ask-async
            Method: POST
        Status:
          Type: HttpApi
          Properties:
            Path: /status/{request_id}
            Method: GET

Exercise 4: Custom domain with API Gateway

Document (without implementing) the steps to connect a custom domain (api.yourapp.com) to your HTTP API. Include: ACM certificate, domain name configuration in API Gateway, and a DNS record in Route 53 or your DNS provider.

See solution 
# Step 1: Create an SSL certificate in ACM (us-east-1 for API Gateway)
# AWS Console → Certificate Manager → Request certificate
# Domain: api.yourapp.com
# Validation: DNS (add the CNAME that ACM gives you)

# Step 2: Configure a custom domain in API Gateway
Resources:
  CustomDomain:
    Type: AWS::ApiGatewayV2::DomainName
    Properties:
      DomainName: api.yourapp.com
      DomainNameConfigurations:
        - CertificateArn: arn:aws:acm:us-east-1:ACCOUNT:certificate/CERT-ID
          EndpointType: REGIONAL

  ApiMapping:
    Type: AWS::ApiGatewayV2::ApiMapping
    Properties:
      ApiId: !Ref AiApi
      DomainName: !Ref CustomDomain
      Stage: prod

# Step 3: DNS Record
# In Route 53 or your DNS provider:
# Type: CNAME (or ALIAS in Route 53)
# Name: api.yourapp.com
# Value: d-XXXXXXXX.execute-api.us-east-1.amazonaws.com
#         ↑ You get this value from the custom domain in API Gateway

# Step 4: Verify
# curl https://api.yourapp.com/health
# Should return your Lambda's health check
# With the AWS CLI:
# 1. Create the domain name
aws apigatewayv2 create-domain-name \
  --domain-name api.yourapp.com \
  --domain-name-configurations CertificateArn=arn:aws:acm:us-east-1:ACCOUNT:certificate/ID

# 2. Create the mapping
aws apigatewayv2 create-api-mapping \
  --api-id API_ID \
  --domain-name api.yourapp.com \
  --stage prod

# 3. Get the target domain for DNS
aws apigatewayv2 get-domain-name --domain-name api.yourapp.com
# → ApiGatewayDomainName: d-XXXXXXXX.execute-api.us-east-1.amazonaws.com

# 4. Configure the CNAME in your DNS

Summary

  • API Gateway turns your Lambda into an HTTP endpoint. Without Gateway, your Lambda isn't accessible from the internet.
  • HTTP API (v2) is the right choice for AI endpoints — 3.5x cheaper, lower latency, simpler configuration.
  • API Gateway's timeout is 29 seconds (hard limit). For longer workloads, use direct invocation or the async pattern.
  • CORS must be configured in API Gateway (not just in your Lambda) so browsers can call your endpoint.
  • The Lambda response body MUST be a string (json.dumps()), not a dict — it's the most common error.
  • Authentication: API key for development, JWT for production with users, IAM for service-to-service.
  • Rate limiting: configure throttling in API Gateway AND protect your OpenAI account with rate limits in your code.
  • Payload format 2.0 (HTTP API): the event has a different structure than REST API. Use event["requestContext"]["http"]["method"] for the method.

Additional Resources

  1. HTTP API (v2) Documentation — Complete HTTP API reference
  2. REST API vs HTTP API — Official comparison
  3. CORS Configuration — Configure CORS in HTTP API
  4. Lambda Proxy Integration — Lambda integration with payload v2.0
  5. JWT Authorizers — JWT authentication in HTTP API
  6. API Gateway Throttling — Rate limiting and throttling
  7. SAM Template Reference — SAM reference
  8. Custom Domain Names — Configure a custom domain
← Previous5. Timeout and Memory Configuration for AINext →7. Cost Estimation for Serverless AI