Module 5: Secrets Management
1. Introduction: Secrets Management
Description
In the previous modules you built defenses to protect your AI system's data flow: a documented threat model (M1), OWASP mapping (M2), defense against prompt injection (M3), and complete input/output sanitization (M4). Your data pipeline is solid. But there is a compromise vector that no guardrail, filter, or validator can prevent: if an attacker obtains your API keys, they can use your credentials without touching your application.
Think about what you have right now in your .env file:
OPENAI_API_KEY=sk-proj-abc123...
ANTHROPIC_API_KEY=sk-ant-xyz789...
DATABASE_URL=postgresql://user:password@host:5432/mydb
PINECONE_API_KEY=pcsk_abc...
REDIS_URL=redis://:secretpassword@host:6379
Each of those lines is a high-value credential. With your OpenAI API key, an attacker can generate $10,000+ of consumption in hours. With your Anthropic API key with access to Claude Opus, the cost can be higher. With your DATABASE_URL, they have access to all your data. And unlike a hacked database where the damage is detectable, the theft of an API key is silent — the attacker uses your account and you pay the bill.
The .env file with .gitignore works for local development. But in production, it's insufficient for concrete reasons that this module makes tangible: there is no automatic rotation (if a key leaks, it stays active until someone changes it manually), there is no audit trail (you don't know who accessed what secret or when), there is no granularity (all processes see all keys), and it doesn't scale with teams (sharing secrets via Slack or 1Password is a security anti-pattern).
This module takes you from .env to enterprise-grade secrets management. It doesn't mean everyone needs HashiCorp Vault — it means you need to understand the concepts (rotation, audit trails, least privilege, dynamic secrets) and be able to implement them with the tools appropriate to your scale.
Why a full module for secrets management?
The decision to dedicate a module to secrets management (separate from the data defenses of modules 3-4) is deliberate. There are three reasons:
1. Secrets are a different asset class
Modules 3-4 protect the data flow: what goes in and out of the LLM. Secrets protect access: who can use which service. A system with perfect sanitization but exposed API keys is like a house with an alarm but with the keys under the doormat. The attacker doesn't need to break your alarm — they walk in through the door.
2. LLM API keys are high-value targets
Unlike an API key for a free service, LLM keys grant direct access to expensive compute. A bot that discovers your OpenAI key can make thousands of calls to GPT-4o in minutes. GitHub reports that it scans more than 100 million commits daily looking for exposed secrets — and LLM API keys are among the most sought-after.
3. Compliance requires it
SOC 2, ISO 27001, HIPAA, and PCI-DSS have specific requirements about credential management: periodic rotation, audit trails, least-privilege principle, encryption at rest. If your company needs to comply with any of these frameworks, .env is not enough.
What will you learn in this module?
By the end of this module you'll be able to:
- Articulate why
.envis insufficient for production with concrete technical arguments: no rotation, no audit, no granularity, no encryption-at-rest - Understand the fundamental concepts of secrets management: rotation, audit trails, least privilege, dynamic secrets, encryption at rest, access policies
- Know HashiCorp Vault as the open-source reference: architecture, transit engine, dynamic secrets, policies, and when it's overkill vs necessary
- Implement API key rotation with zero-downtime: dual-key strategies, rotation schedulers, and automation with Python
- Use cloud KMS (AWS Secrets Manager, GCP Secret Manager, Azure Key Vault) with functional Python code for at least one provider
- Configure token lifecycle management: creation with minimal scope, scheduled rotation, immediate revocation when compromise is suspected
- Implement audit trails: logging of who accessed what secret, when, and from where — integrated with structured logging
- Integrate secrets management with FastAPI using dependency injection, with fallback and resilience when the secrets service is unavailable
Module roadmap
This module has 8 lessons that build your secrets management system layer by layer:
| # | Lesson | What you'll learn |
|---|---|---|
| 01 | Introduction: Secrets Management | Why this module, roadmap, connection with the project, setup |
| 02 | Beyond .env: Why It's Not Enough | Limitations of .env, real incidents, transition to production |
| 03 | HashiCorp Vault: Concepts and Setup | Architecture, secrets engines, auth methods, policies, hvac client |
| 04 | API Key Rotation Strategies | Zero-downtime rotation, dual-key, automation, schedules |
| 05 | Cloud KMS: AWS, GCP, Azure | Secrets Manager per provider, unified Python interface, migration |
| 06 | Token Lifecycle and Audit Trails | Lifecycle management, audit logging, compliance, revocation |
| 07 | Least Privilege and Python Integration | Scoped tokens, per-service keys, FastAPI integration, fallback |
| 08 | Project: Secrets Management Setup | Complete setup with secrets client, rotation scheduler, audit logger |
The progression is: context (01) → problem (02) → enterprise solution (03) → rotation (04) → cloud (05) → lifecycle (06) → integration (07) → project (08).
Lessons 02-03 establish the why and the what. Lesson 04 tackles the most critical operation (rotation). Lesson 05 covers the cloud options most people will use. Lessons 06-07 complete the cycle with lifecycle management and integration. Lesson 08 consolidates everything in the project.
Context in the guide
This guide has 8 modules organized into 3 phases:
Phase 1: Security Foundations (Modules 1-3)
├── Module 1: AI Security Landscape & Threat Model ✅ COMPLETED
├── Module 2: OWASP LLM Top 10 Deep Dive ✅ COMPLETED
└── Module 3: Prompt Injection — Attacks & Defenses ✅ COMPLETED
Phase 2: Defense Implementation (Modules 4-6)
├── Module 4: Input & Output Sanitization ✅ COMPLETED
├── Module 5: Secrets Management ← YOU ARE HERE
└── Module 6: Data Privacy & PII Protection
Phase 3: Production Security (Modules 7-8)
├── Module 7: Security Testing & Auditing
└── Module 8: Integration Project — Secured AI System
Module 4 gave you a sanitization pipeline that cleans data before and after the LLM. That pipeline uses API keys to access the model. Until now, those keys come from a .env. After this module, they come from a secrets manager with rotation, audit trails, and least privilege.
The relationship with the previous modules is complementary:
Module 3 (Injection Defense) Module 5 (Secrets Management)
───────────────────────── ─────────────────────────
Protects the data flow Protects access to services
Defends against attacks on the LLM Defends against credential theft
The pipeline's code The pipeline's credentials
Focus: LLM01 Focus: access infrastructure
The practical connection is direct: the code from the previous modules doesn't change — only the source of the credentials. Where you used to have os.getenv("OPENAI_API_KEY"), you now have secrets_client.get("openai-api-key") with automatic rotation and audit trail.
Prerequisites
For this module you need:
- Modules 1-4 completed: Threat Model Document, OWASP Mapping Audit, Injection Defense Pipeline, Sanitization Pipeline
- Python 3.10+ installed
- An OpenAI API key (or a compatible provider) — you'll use it as an example of a secret to manage
- Familiarity with FastAPI — dependency injection, middleware
- Docker installed (optional, for Vault in dev mode)
Technical setup
If you already have the environment from the previous modules, activate it and add the new dependencies:
source security-guide-env/bin/activate # macOS/Linux
# security-guide-env\Scripts\activate # Windows
pip install hvac boto3 cryptography schedule
pip install google-cloud-secret-manager # If you use GCP (optional)
pip install azure-keyvault-secrets azure-identity # If you use Azure (optional)If you're starting from this module:
python -m venv security-guide-env
source security-guide-env/bin/activate
pip install hvac boto3 cryptography schedule fastapi uvicorn pydantic
pip install python-dotenv # Only for dev/transition
export OPENAI_API_KEY="sk-..."Quick verification:
import os
import json
from datetime import datetime
from cryptography.fernet import Fernet
key = Fernet.generate_key()
cipher = Fernet(key)
secret_value = "sk-proj-my-openai-key-12345"
encrypted = cipher.encrypt(secret_value.encode())
decrypted = cipher.decrypt(encrypted).decode()
print(f"Original: {secret_value}")
print(f"Encrypted: {encrypted[:50]}...")
print(f"Decrypted: {decrypted}")
print(f"Match: {secret_value == decrypted}")
audit_entry = {
"timestamp": datetime.utcnow().isoformat(),
"action": "secret_accessed",
"secret_name": "openai-api-key",
"accessor": "setup-check",
}
print(f"\nAudit entry: {json.dumps(audit_entry, indent=2)}")
# Expected output:
# Original: sk-proj-my-openai-key-12345
# Encrypted: gAAAAABn...
# Decrypted: sk-proj-my-openai-key-12345
# Match: True
#
# Audit entry: {
# "timestamp": "2026-03-13T...",
# "action": "secret_accessed",
# "secret_name": "openai-api-key",
# "accessor": "setup-check"
# }If you see the correct outputs, your setup is ready. The key dependencies for this module are:
| Package | What for |
|---|---|
hvac | Python client for HashiCorp Vault |
boto3 | AWS SDK (Secrets Manager, KMS) |
cryptography | Local encryption, Fernet symmetric encryption |
schedule | Rotation scheduler |
google-cloud-secret-manager | GCP Secret Manager (optional) |
azure-keyvault-secrets | Azure Key Vault (optional) |
Connection with the module project
This module closes with the Secrets Management Setup project: a complete secrets management system for your AI application. It's the guide's fifth artifact and it differs from the previous ones in that it's infrastructure, not application code.
The Secrets Management Setup includes:
- Secrets Client — Abstraction layer over Vault or cloud KMS with a unified interface
- Rotation Scheduler — Automatic rotation of API keys with zero-downtime
- Audit Logger — Record of each access to secrets with timestamps and accessor identity
- FastAPI Integration — Dependency injection for secrets in endpoints
- Fallback Strategy — Resilience when the secrets service is unavailable
The setup integrates with your existing pipeline:
Request
│
▼
┌──────────────────────────┐
│ Secrets Client │ ← Gets the API key from the vault/KMS
├──────────────────────────┤
│ Input Sanitizer (M4) │ ← Your Module 4 pipeline
├──────────────────────────┤
│ Injection Detector (M3) │ ← Your Module 3 pipeline
├──────────────────────────┤
│ LLM Processing │ ← Uses the key from the secrets client
├──────────────────────────┤
│ Output Validator (M4) │ ← Response validation
├──────────────────────────┤
│ Audit Logger │ ← Records access to secrets
└──────────────────────────┘
│
▼
Response
In Module 8 (integration project), the Secrets Management Setup integrates with the Injection Defense Pipeline (M3), the Sanitization Pipeline (M4), and the PII Protection Layer (M6) to form the complete secured system.
Module 1: Threat Model Document (base)
Module 2: + OWASP Mapping Audit (detailed mapping)
Module 3: + Injection Defense Pipeline (defense against LLM01)
Module 4: + Sanitization Pipeline (input/output)
Module 5: + Secrets Management Setup (credentials) ← YOU PRODUCE IT HERE
Module 6: + PII Protection Layer (sensitive data)
Module 7: + Security Audit Report (validation)
Module 8: → Secured AI System (full integration)
What makes this module different vs Production Best Practices (#13)
If you completed Production Best Practices (#13), you already use .env with python-dotenv. This module does not repeat that content — it deepens it:
| Concept | Production Best Practices (#13) | This module (M5) |
|---|---|---|
| Secrets storage | .env with python-dotenv | Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault |
| Rotation | "Rotate your keys periodically" | Automation with zero-downtime, dual-key strategy, scheduler |
| Audit | Not covered | Logging of each access with timestamps, accessor, context |
| Least privilege | Basic mention | Scoped tokens, per-service keys, IAM policies per service |
| Encryption | .env doesn't encrypt | Encryption at rest, transit encryption with Vault |
| Fallback | Not covered | Local cache with TTL, fallback to encrypted storage |
| Team scaling | "Don't share keys via Slack" | Secret distribution with policies, per-environment configs |
The rule is: if something sounds familiar from #13, here you see the enterprise version with complete code and production considerations.
What this module does NOT cover
To keep the focus:
- Prompt injection defense: That was Module 3. Secrets don't prevent attacks on the model.
- Input/output sanitization: That was Module 4. Secrets protect access, not data.
- PII protection: That's Module 6. Secrets protect credentials, not personal data.
- Full DevSecOps pipeline: We mention CI/CD integration but don't cover Jenkins, GitHub Actions, or Terraform in depth.
- Complete legal compliance: We mention SOC 2 and ISO 27001 as motivation, not as a compliance guide.
- Hardware Security Modules (HSM): We mention HSM as an enterprise level but don't cover configuration.
The analogy: a building's keys
Imagine your AI system is a corporate building:
Corporate building AI system
──────────────────────── ────────────────────────
Building keys API keys (OpenAI, Anthropic)
Office keys Database credentials
Data center access card Cloud provider keys
Safe key Encryption keys
How do you manage the keys? How do you manage the secrets?
──────────────────────── ────────────────────────
Under the doormat (.env) .env file on the server
In your pocket (env vars) OS environment variables
On a keyring (keychain) Encrypted local storage
In a security office (vault) HashiCorp Vault / Cloud KMS
With .env it's like leaving the keys under the doormat: it works, but anyone who knows where to look has access. A secrets manager is the building's security office: the keys are stored in a safe place, who takes them is recorded, they are changed periodically, and each person only receives the keys they need.
This module's progression follows the same logic:
- 📋 Lesson 02: Understand why the doormat isn't safe
- 🏗️ Lesson 03: Get to know the security office (Vault)
- 🔄 Lesson 04: Learn to change the keys periodically (rotation)
- ☁️ Lesson 05: Cloud options for the security office (KMS)
- 📝 Lesson 06: Record who takes which key (audit trails)
- 🔑 Lesson 07: Give out only the necessary keys (least privilege)
- 🏢 Lesson 08: Set up your complete security office (project)
Your system: pre-assessment exercise
Before starting the technical lessons, take 5 minutes to assess your current system:
- Do you have LLM API keys in a
.envfile in production? → If yes, lesson 02 is urgent - When was the last time you rotated your API keys? → If the answer is "never" or "I don't know", lesson 04 is a priority
- Do you know who accessed which secret in the last 24 hours? → If not, lesson 06 gives you audit trails
- Do all your services see all the API keys? → If yes, lesson 07 implements least privilege
- What happens if your secrets service goes down? → If you don't know, lesson 07 covers fallback and resilience
- Can you revoke a compromised key in under 5 minutes? → If not, lesson 06 has emergency revocation
If you answered in a concerning way to more than three questions, follow the full module in order. If you already have solid answers to some, focus on the lessons that cover your gaps — but read the rest for the advanced techniques that could improve what you already have.
The 4 pillars of secrets management
Every enterprise secrets management system is built on four pillars. Each lesson reinforces one or more of these pillars:
┌────────────────────────────────────────────────────────────────┐
│ SECRETS MANAGEMENT │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │ Encryption │ │ Rotation │ │ Audit │ │
│ │ at Rest │ │ (automatic) │ │ Trails │ │
│ │ │ │ │ │ │ │
│ │ Secrets are │ │ Keys are │ │ Every │ │
│ │ encrypted │ │ rotated │ │ access is │ │
│ │ in storage │ │ periodically│ │ logged: │ │
│ │ │ │ no downtime │ │ who, when, │ │
│ │ │ │ │ │ what │ │
│ │ Lesson 3, 5 │ │ Lesson 4 │ │ Lesson 6 │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ┌──────────────┐ │
│ │ Least │ The complete system: │
│ │ Privilege │ - Encrypts secrets in storage │
│ │ │ - Rotates them automatically │
│ │ Each │ - Records every access │
│ │ service │ - Limits access per service │
│ │ only sees │ │
│ │ what it │ Lessons 7, 8 │
│ │ needs │ │
│ │ Lesson 7 │ │
│ └──────────────┘ │
└────────────────────────────────────────────────────────────────┘
These pillars are not independent — they reinforce each other. Encryption without rotation means a compromised key stays active. Rotation without audit means you don't know whether the rotation happened correctly. Audit without least privilege means the log records legitimate but unnecessary accesses. The lesson 08 project integrates the four pillars.
Common mistakes when approaching secrets management
"I already have .gitignore, my secrets are safe"
.gitignore prevents you from accidentally committing the .env. It doesn't prevent someone from reading it off the server, a backup from including it, a log from capturing it, or a colleague from sharing it via Slack. .gitignore is necessary but absolutely insufficient.
"I'm just a developer, I don't need enterprise security"
If your OpenAI API key leaks, the damage is financial and immediate regardless of your team's size. An indie developer with a stolen key can receive a $10,000 bill. Scale changes the solution (you don't need Vault), but the concepts (rotation, audit) apply to everyone.
"I'll rotate the keys when I have time"
Manual rotation doesn't happen. In practice, if it isn't automatic, it doesn't get done. Module 5 gives you automation so that rotation is a process, not an intention.
"The cloud provider's secrets are secure by default"
OpenAI and Anthropic API keys don't come with encryption at rest, automatic rotation, or audit trails by default. You implement those layers. The LLM provider gives you the key — you decide how you protect it.
"It's too complex for my project"
AWS Secrets Manager or GCP Secret Manager with a Python wrapper are ~100 additional lines of code. The cost is ~$2/month for 5 secrets. The real complexity is learning the concepts — once understood, the implementation is straightforward. This module breaks down that complexity into manageable steps.
Trade-offs: security vs development speed
A theme that runs through the whole module is the trade-off between security and agility:
| Approach | Security | Dev speed | Cost | Ideal for |
|---|---|---|---|---|
.env without protection | ❌ Minimal | ✅ Maximum | ✅ $0 | Prototypes, hackathons |
.env + local encryption | ⚠️ Low | ✅ High | ✅ $0 | Local dev, personal projects |
| Cloud KMS + manual rotation | ✅ Medium | ⚠️ Medium | ⚠️ ~$2-5/month | Startups, MVPs in production |
| Cloud KMS + automatic rotation | ✅ High | ⚠️ Medium | ⚠️ ~$5-20/month | Production with real users |
| Vault + dynamic secrets + audit | ✅ Maximum | ❌ Lower (complex setup) | ❌ ~$100+/month | Enterprise, compliance |
The goal is not to maximize security — it's to choose the level appropriate for your context and scale when necessary. This module prepares you for any level.
Scale of solutions: not everyone needs Vault
A key concept that runs through the whole module: the right solution depends on your scale. Not everyone needs HashiCorp Vault. Everyone needs rotation and audit.
| Scale | Recommended solution | Cost |
|---|---|---|
| Indie/Freelancer | AWS Secrets Manager or GCP Secret Manager with manual rotation every 90 days | ~$0.40/secret/month |
| Startup (2-10 devs) | Cloud KMS with semi-automatic rotation (script + cron) | ~$1-5/month |
| Mid-size team (10-50) | Cloud KMS with automatic rotation + CI/CD integration | ~$5-20/month |
| Enterprise (50+) | HashiCorp Vault (self-hosted or HCP) with dynamic secrets | ~$100+/month |
In each lesson you'll see the options by scale. Lesson 03 covers Vault as the enterprise reference. Lesson 05 covers cloud KMS as the pragmatic option for most. The project (lesson 08) lets you choose the one that applies to your context.
How to use each lesson
Each technical lesson (02-07) follows this structure:
- Context — Why this piece exists and what problem it solves
- Concept — The necessary theory (40% of the content)
- Implementation — Complete, executable Python code (60% of the content)
- Connection with the project — How it fits into the Secrets Management Setup
- Troubleshooting — 3-5 common problems with solutions
- Exercises — 4-6 practical exercises with solutions in
<details> - Summary — Key points of the topic
- Resources — 6-8 references to go deeper
I recommend following the lessons in order (02 → 07) because the progression builds on the previous concepts. Lesson 02 establishes the problem. Lesson 03 introduces the enterprise solution. Lessons 04-05 cover operations and options. Lessons 06-07 complete it with lifecycle and integration.
The complete pipeline as architecture
Before going deep into each component (lessons 02-07), you need to see how secrets management integrates into your AI system. This diagram is the reference for the whole module:
┌──────────────────────────────────────────────────────────────────┐
│ SECRETS MANAGEMENT SETUP │
│ │
│ SECRETS LAYER │
│ ┌────────────────┐ ┌────────────────┐ ┌────────────────┐ │
│ │ 1. Provider │→ │ 2. Policy │→ │ 3. Cache │ │
│ │ (Vault/KMS) │ │ Enforcer │ │ (TTL-based) │ │
│ └────────────────┘ └────────────────┘ └────────────────┘ │
│ │ │ │ │
│ │ Secrets Client │ │
│ ├───────────────────┴───────────────────┤ │
│ │ │ │
│ OPERATIONS LAYER │ │
│ ┌────────────────┐ ┌────────────────┐ │ │
│ │ 4. Rotation │ │ 5. Audit │ │ │
│ │ Scheduler │ │ Logger │ │ │
│ └────────────────┘ └────────────────┘ │ │
│ │ │ │ │
│ APPLICATION LAYER │ │
│ ┌────────────────┐ ┌────────────────┐ │ │
│ │ 6. FastAPI │ │ 7. Fallback │ │ │
│ │ DI │ │ & Circuit │ │ │
│ │ Integration │ │ Breaker │ │ │
│ └────────────────┘ └────────────────┘ │ │
│ │ │ │
│ ▼ │ │
│ ┌────────────────────────────────────────────────┘ │
│ │ │
│ │ Your AI application: │
│ │ Injection Defense (M3) + Sanitization (M4) + LLM API │
│ │ │
│ │ Where you used to have: os.getenv("OPENAI_API_KEY") │
│ │ Now you have: secrets_client.get("openai-api-key")│
│ │ → with rotation, audit, least privilege │
│ │ │
│ └───────────────────────────────────────────────────────────────┘
└──────────────────────────────────────────────────────────────────┘
Each lesson builds a block of this diagram. Lesson 08 integrates them into the Secrets Management Setup as a reusable artifact.
Difference between secrets and configuration
A common mistake is treating all configuration as secrets. Not every environment variable needs enterprise protection:
| Type | Example | Is it a secret? | Treatment |
|---|---|---|---|
| LLM API keys | OPENAI_API_KEY | ✅ Yes — high value | Vault/KMS + rotation + audit |
| Database passwords | DATABASE_PASSWORD | ✅ Yes — data access | Vault/KMS + rotation |
| Encryption keys | JWT_SECRET | ✅ Yes — compromises auth | Vault/KMS + careful rotation |
| Webhook secrets | STRIPE_WEBHOOK_SECRET | ✅ Yes — verification | KMS + rotation |
| Service URLs | DATABASE_HOST | ❌ No — not sensitive | Env vars or config file |
| Feature flags | ENABLE_NEW_UI | ❌ No — not sensitive | Env vars or config service |
| Log level | LOG_LEVEL | ❌ No — not sensitive | Env vars |
| Port | PORT | ❌ No — not sensitive | Env vars |
The rule is: if exposing the value causes harm (financial, unauthorized access, data compromise), it's a secret and needs protection. If not, it's configuration and can live in normal env vars.
Key module concepts: quick glossary
Before entering the lessons, these are the concepts you'll see repeatedly:
- Secret: Any credential, token, or key that grants access to a service or resource
- Rotation: The process of replacing an active secret with a new one, invalidating the previous one
- Audit trail: Immutable record of each operation performed on a secret (read, write, rotation, revocation)
- Least privilege: Principle of giving each component only the minimum access necessary to function
- Dynamic secrets: Credentials generated on-demand with a limited time-to-live (TTL) that are revoked automatically
- Encryption at rest: Secrets are encrypted when stored, not readable as plain text
- Transit encryption: Service that encrypts/decrypts data without exposing the encryption keys
- Secrets engine: Module of a secrets manager that stores, generates, or encrypts data (e.g., KV, Transit, Database in Vault)
- Auth method: Mechanism by which a client authenticates to the secrets manager (token, AppRole, IAM, etc.)
- Policy: Rule that defines which operations an authenticated client can perform on which secret paths
- Lease: Validity period of a dynamic secret — on expiry, the secret is revoked automatically
- Circuit breaker: Resilience pattern that temporarily stops calls to a failing service to avoid cascading failures
- Fallback: Alternative source of secrets when the main provider is unavailable
The real cost of not managing secrets
So you understand the weight of secrets management, these are concrete industry data points:
Documented incidents
| Pattern | What happened | Impact |
|---|---|---|
| Key on GitHub | Developer committed .env with AWS API keys. An automated bot found the keys in <5 minutes and provisioned mining instances. | $50,000+ in billing before detection |
| Key in Docker image | OpenAI API key hardcoded in Dockerfile. Public image on Docker Hub. | Unlimited consumption of the account until revocation |
| Key shared via Slack | Team shared an API key via a Slack channel. Ex-employee kept access to the channel. | Unauthorized access for 6+ months |
| No rotation | Production API key not rotated in 2 years. The employee who created it left. Nobody knew whether it had been compromised. | Risk of undetected access |
| Key in logs | API key accidentally logged in structured logging. Logs sent to Datadog. Accessible to the whole team. | Exposure to a broad audience |
Industry figures
- 🔍 GitHub Secret Scanning detects millions of exposed secrets per year in public repositories
- 💰 The average cost of a credential-related data breach is $4.5M (IBM 2024)
- ⏱️ The average time to detect a compromised credential is 277 days (IBM 2024)
- 🤖 Automated bots find exposed API keys on GitHub in under 5 minutes from the commit
Each of these incidents is prevented with the practices you learn in this module: encryption at rest, automatic rotation, audit trails, and least privilege.
The before and after of your code
So you can visualize the concrete impact of this module, here is the change in your code:
Before (with .env)
import os
from dotenv import load_dotenv
from openai import OpenAI
load_dotenv()
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))Problems: no encryption, no rotation, no audit, no granularity, no fallback.
After (with Secrets Management Setup)
from secrets import ResilientSecretsClient
secrets = ResilientSecretsClient(config=secrets_config)
result = secrets.get("openai-api-key", service_name="api-service")
client = OpenAI(api_key=result.value)Benefits: encrypted at rest, automatic rotation, audit trail of each access, least privilege per service, fallback with circuit breaker.
The application code barely changes — one different line to get the key. The security infrastructure behind it changes completely.
Summary
- This module builds the credential management layer that protects access to the services your AI system uses — separate from the data flow (M3-M4)
- LLM API keys are high-value targets: a stolen OpenAI key can generate thousands of dollars in consumption in minutes
.envworks for local development but is insufficient for production: no rotation, no audit, no granularity, no encryption- The 4 pillars of secrets management are: encryption at rest, automatic rotation, audit trails, and least privilege — each lesson reinforces one or more pillars
- The module covers the full spectrum: from why you need this (02) to how to implement it (03-07) and the integrated project (08)
- The right solution depends on your scale: cloud KMS for most, Vault for enterprise — the concepts (rotation, audit, least privilege) are universal
- The Secrets Management Setup is the guide's fifth artifact and it changes the source of your pipeline's credentials without modifying the application code
- The change in your code is minimal: from
os.getenv("OPENAI_API_KEY")tosecrets_client.get("openai-api-key")— with all the security infrastructure behind it - In Module 8, this setup combines with Injection Defense (M3), Sanitization (M4), and PII Protection (M6) for the complete secured system
Next lesson: In lesson 02 you'll understand in depth why .env is not enough for production, with real incidents of API key exposure, the financial cost of leaked keys, and the transition path from .env to a secrets manager. You'll see code that demonstrates the vulnerabilities and the technical comparison between the available options.
Additional resources
- HashiCorp Vault Documentation — Official Vault documentation, the enterprise reference for secrets management
- AWS Secrets Manager — Documentation for AWS's secrets service, the most adopted cloud option
- GCP Secret Manager — Documentation for Google Cloud's secrets service
- OWASP Secrets Management Cheat Sheet — OWASP guide to secrets management with best practices
- GitHub Secret Scanning — GitHub documentation on secret detection in repositories
- 12-Factor App — Config — Principles for configuration and secrets management in modern applications
- CIS Benchmarks — Secret Management — Security benchmarks for credential management
- Python Cryptography Library — Python library for encryption, the basis for local encryption of secrets
Created: March 2026 Version: 1.0