Module 1: Testing Fundamentals for AI
7. Project: Test Suite Setup
Description
This is the Module 1 mini-project. The goal: take a real LLM app (the provided reference one or your own) and set up a complete testing infrastructure from scratch — pytest configured, fixtures for LLM mocking, directory structure, smoke tests, contract tests, and working regression tests.
It's not a theoretical exercise. By the end you'll have a real suite running, with pytest -m "not integration" executing in under 30 seconds with all tests green. This suite is the foundation on which you'll build all the following modules.
By the end you'll have applied all of Module 1: the test taxonomy, the AAA pattern, the LLM fixtures, the pytest markers, and the layered testing strategy.
Project prerequisites
Before starting, verify that you have:
# Python 3.10+
python --version
# Dependencies installed
pip install pytest pytest-asyncio pytest-mock pytest-cov
pip install openai pydantic python-dotenv
# Verify pytest
pytest --version
# pytest 8.x.xThe reference app
If you don't have your own app, use this sentiment analysis app:
# src/app/__init__.py
# (empty)# src/app/config.py
import os
from dotenv import load_dotenv
load_dotenv()
class Settings:
"""Centralized app configuration."""
model_name: str = os.getenv("LLM_MODEL", "gpt-4o-mini")
max_tokens: int = int(os.getenv("MAX_TOKENS", "500"))
temperature: float = float(os.getenv("TEMPERATURE", "0.3"))
openai_api_key: str = os.getenv("OPENAI_API_KEY", "")
@property
def is_configured(self) -> bool:
return bool(self.openai_api_key)
settings = Settings()# src/app/parsers.py
import json
import re
def parse_json_from_llm_output(raw: str) -> dict:
"""
Extracts and parses JSON from the LLM response.
Handles pure JSON, JSON in markdown blocks, JSON with surrounding text.
Args:
raw: String with the LLM response
Returns:
dict with the parsed JSON
Raises:
ValueError: If no JSON is found in the string
json.JSONDecodeError: If the JSON is malformed
"""
if not raw or not raw.strip():
raise ValueError(f"Empty or None input: {raw!r}")
# Attempt 1: Pure JSON (most common with response_format=json_object)
try:
return json.loads(raw.strip())
except json.JSONDecodeError:
pass
# Attempt 2: Extract from a markdown block ```json ... ```
markdown_match = re.search(r'```(?:json)?\s*\n(.*?)\n```', raw, re.DOTALL)
if markdown_match:
return json.loads(markdown_match.group(1))
# Attempt 3: Extract JSON from mixed text (find first { and last })
start = raw.find('{')
end = raw.rfind('}')
if start != -1 and end != -1 and end > start:
return json.loads(raw[start:end + 1])
raise ValueError(f"No JSON found in: {raw!r}")# src/app/sentiment.py
from openai import OpenAI
from app.config import settings
from app.parsers import parse_json_from_llm_output
client = OpenAI(api_key=settings.openai_api_key)
def validate_input(text: str, max_chars: int = 5000) -> str:
"""Validates and normalizes the input before sending it to the LLM."""
if not text or not text.strip():
raise ValueError("The analysis text cannot be empty")
if len(text) > max_chars:
# Truncate instead of failing (graceful degradation)
text = text[:max_chars]
return text.strip()
def build_sentiment_prompt(text: str) -> str:
"""Builds the prompt for sentiment analysis."""
return f"""Analyze the sentiment of the following text.
Return ONLY a JSON object with this exact structure (no other text):
{{"sentiment": "positive|negative|neutral", "confidence": 0.0-1.0}}
Text: {text}"""
def analyze_sentiment(text: str) -> dict:
"""
Analyzes the sentiment of a text using an LLM.
Returns:
dict with keys "sentiment" (str) and "confidence" (float)
Raises:
ValueError: If the input is invalid
"""
# Validate input
validated_text = validate_input(text)
# Build prompt
prompt = build_sentiment_prompt(validated_text)
# Call the LLM
response = client.chat.completions.create(
model=settings.model_name,
messages=[{"role": "user", "content": prompt}],
temperature=settings.temperature,
max_tokens=settings.max_tokens,
)
# Parse response
raw_content = response.choices[0].message.content
return parse_json_from_llm_output(raw_content)# src/app/main.py
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from app.sentiment import analyze_sentiment
app = FastAPI(title="Sentiment Analysis API", version="1.0.0")
class TextInput(BaseModel):
text: str
class SentimentResponse(BaseModel):
sentiment: str
confidence: float
@app.get("/health")
def health_check():
return {"status": "healthy", "service": "sentiment-api", "version": "1.0.0"}
@app.post("/analyze", response_model=SentimentResponse)
def analyze(input_data: TextInput):
try:
result = analyze_sentiment(input_data.text)
return SentimentResponse(**result)
except ValueError as e:
raise HTTPException(status_code=400, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail="Internal server error")# requirements.txt
openai>=1.0.0
fastapi>=0.100.0
uvicorn>=0.23.0
pydantic>=2.0.0
python-dotenv>=1.0.0
pytest>=7.4.0
pytest-asyncio>=0.21.0
pytest-mock>=3.11.0
pytest-cov>=4.1.0
httpx>=0.24.0
Step 1: Directory structure
Create the following structure:
mkdir -p tests/unit/contracts tests/unit/behavioral tests/unit/regression
mkdir -p tests/integration tests/smoke
touch tests/__init__.py tests/unit/__init__.py
touch tests/smoke/__init__.py tests/integration/__init__.py
touch tests/unit/contracts/__init__.py
touch tests/unit/behavioral/__init__.py
touch tests/unit/regression/__init__.pyExpected result:
my-ai-app/
├── src/
│ └── app/
│ ├── __init__.py
│ ├── config.py
│ ├── parsers.py
│ ├── sentiment.py
│ └── main.py
├── tests/
│ ├── __init__.py
│ ├── conftest.py ← We'll create in Step 2
│ ├── helpers.py ← We'll create in Step 2
│ ├── smoke/
│ │ ├── __init__.py
│ │ └── test_smoke.py ← We'll create in Step 4
│ ├── unit/
│ │ ├── __init__.py
│ │ ├── contracts/
│ │ │ ├── __init__.py
│ │ │ └── test_contracts.py ← We'll create in Step 5
│ │ ├── behavioral/
│ │ │ ├── __init__.py
│ │ │ └── test_behavioral.py ← We'll create in Step 6
│ │ └── regression/
│ │ ├── __init__.py
│ │ └── test_regression.py ← We'll create in Step 7
│ └── integration/
│ ├── __init__.py
│ └── test_e2e.py ← Structure ready (module 3)
├── pytest.ini ← We'll create in Step 3
├── .env
└── requirements.txt
Step 2: Helpers and conftest.py
helpers.py
# tests/helpers.py
"""
Helper functions for testing. They aren't pytest fixtures.
They're imported in conftest.py and in tests that need fine-grained control.
"""
from unittest.mock import MagicMock
def create_openai_chat_response(
content: str,
prompt_tokens: int = 100,
completion_tokens: int = 50,
finish_reason: str = "stop",
model: str = "gpt-4o-mini",
) -> MagicMock:
"""
Creates a mock that exactly replicates the ChatCompletion structure.
Always use this function instead of creating a MagicMock directly.
"""
response = MagicMock()
# Metadata
response.id = "chatcmpl-test-fixture"
response.model = model
# Choice
choice = MagicMock()
choice.index = 0
choice.finish_reason = finish_reason
choice.message.role = "assistant"
choice.message.content = content
response.choices = [choice]
# Usage
response.usage.prompt_tokens = prompt_tokens
response.usage.completion_tokens = completion_tokens
response.usage.total_tokens = prompt_tokens + completion_tokens
return responseconftest.py
# tests/conftest.py
"""
Shared fixtures for the whole testing suite.
Automatically available in all tests.
"""
import os
import sys
import pytest
from unittest.mock import MagicMock
# Ensure that src/ is on the path
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', 'src'))
from tests.helpers import create_openai_chat_response
# ─── Environment variables for tests ──────────────────────────────────
@pytest.fixture(autouse=True)
def set_test_environment(monkeypatch):
"""Sets up the test environment for all tests."""
monkeypatch.setenv("ENVIRONMENT", "test")
monkeypatch.setenv("OPENAI_API_KEY", "sk-test-fake-key-for-testing")
@pytest.fixture
def real_api_key():
"""Real API key. Skip if it's not configured."""
key = os.getenv("OPENAI_API_KEY")
if not key or key.startswith("sk-test"):
pytest.skip("Real API key required for integration tests")
return key
# ─── Response factory ─────────────────────────────────────────────────
@pytest.fixture
def make_llm_response():
"""
Factory fixture to create mock responses.
Returns the create_openai_chat_response function.
"""
return create_openai_chat_response
# ─── Predefined responses ─────────────────────────────────────────────
@pytest.fixture
def sentiment_positive_response(make_llm_response):
"""Mock response: positive sentiment."""
return make_llm_response('{"sentiment": "positive", "confidence": 0.9}')
@pytest.fixture
def sentiment_negative_response(make_llm_response):
"""Mock response: negative sentiment."""
return make_llm_response('{"sentiment": "negative", "confidence": 0.85}')
@pytest.fixture
def sentiment_neutral_response(make_llm_response):
"""Mock response: neutral sentiment."""
return make_llm_response('{"sentiment": "neutral", "confidence": 0.5}')
@pytest.fixture
def malformed_json_response(make_llm_response):
"""Edge case: malformed JSON."""
return make_llm_response('{"sentiment": "positive"') # Incomplete
@pytest.fixture
def empty_content_response(make_llm_response):
"""Edge case: empty content."""
return make_llm_response("")
# ─── Mock client ──────────────────────────────────────────────────────
@pytest.fixture
def mock_openai_client(make_llm_response):
"""Mocked OpenAI client with a default response."""
client = MagicMock()
client.chat.completions.create.return_value = make_llm_response(
'{"sentiment": "positive", "confidence": 0.9}'
)
return client
# ─── FastAPI test client ───────────────────────────────────────────────
@pytest.fixture(scope="module")
def test_client():
"""HTTP client for FastAPI tests."""
from fastapi.testclient import TestClient
from app.main import app
with TestClient(app) as client:
yield clientStep 3: pytest.ini
# pytest.ini
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
markers =
smoke: Smoke tests. No cost. Verify that the system starts up.
contract: Contract tests. No cost. Verify the output structure.
behavioral: Behavior tests. No cost (mocks). Output properties.
regression: Regression tests. No cost. Prevent known bugs.
unit: Tests with mocks. No cost. Fast.
integration: Tests with a real LLM. They have a cost. Slow.
addopts = -v --tb=short
asyncio_mode = autoStep 4: Smoke tests
# tests/smoke/test_smoke.py
"""
Smoke tests: verify that the system is alive.
They're the first tests to run. If they fail, nothing else matters.
"""
import pytest
@pytest.mark.smoke
class TestAppSmoke:
"""Verifies that the app starts up correctly."""
def test_config_module_loads(self):
"""The configuration module loads without errors."""
from app.config import settings
assert settings is not None
assert hasattr(settings, "model_name")
def test_parsers_module_loads(self):
"""The parsers module loads and the functions are callable."""
from app.parsers import parse_json_from_llm_output
assert callable(parse_json_from_llm_output)
def test_sentiment_module_loads(self):
"""The sentiment module loads correctly."""
from app.sentiment import analyze_sentiment, validate_input, build_sentiment_prompt
assert callable(analyze_sentiment)
assert callable(validate_input)
assert callable(build_sentiment_prompt)
@pytest.mark.smoke
class TestAPISmoke:
"""Verifies that the FastAPI API responds."""
def test_health_endpoint_returns_200(self, test_client):
"""The /health endpoint returns 200."""
response = test_client.get("/health")
assert response.status_code == 200
def test_health_response_has_status(self, test_client):
"""The /health response has the 'status' field."""
response = test_client.get("/health")
data = response.json()
assert "status" in data
assert data["status"] == "healthy"
def test_analyze_endpoint_exists(self, test_client):
"""The /analyze endpoint exists (doesn't return 404)."""
# We don't verify the result — only that the endpoint exists
response = test_client.post("/analyze", json={"text": "test"})
assert response.status_code != 404
def test_docs_accessible(self, test_client):
"""The Swagger documentation is accessible."""
response = test_client.get("/docs")
assert response.status_code == 200Step 5: Contract tests
# tests/unit/contracts/test_contracts.py
"""
Contract tests: verify that the components fulfill their contracts.
They all use mocks — no real API calls.
"""
import pytest
import json
from unittest.mock import patch, MagicMock
from app.sentiment import analyze_sentiment
from app.parsers import parse_json_from_llm_output
# ─── Parser contracts ─────────────────────────────────────────────────
@pytest.mark.contract
@pytest.mark.unit
class TestParserContract:
"""
parse_json_from_llm_output:
- For valid JSON: returns a dict
- For JSON in markdown: returns a dict
- For text without JSON: raises ValueError
- For malformed JSON: raises JSONDecodeError
"""
@pytest.mark.parametrize("raw_input,expected", [
('{"sentiment": "positive", "confidence": 0.9}',
{"sentiment": "positive", "confidence": 0.9}),
('```json\n{"sentiment": "positive", "confidence": 0.9}\n```',
{"sentiment": "positive", "confidence": 0.9}),
('Result: {"sentiment": "positive", "confidence": 0.9}',
{"sentiment": "positive", "confidence": 0.9}),
])
def test_returns_dict_for_valid_json(self, raw_input, expected):
result = parse_json_from_llm_output(raw_input)
assert result == expected
def test_raises_value_error_for_no_json(self):
with pytest.raises(ValueError, match="No JSON found|empty"):
parse_json_from_llm_output("There's no JSON in this text")
def test_raises_for_empty_input(self):
with pytest.raises(ValueError):
parse_json_from_llm_output("")
def test_raises_json_decode_error_for_malformed(self):
with pytest.raises((json.JSONDecodeError, ValueError)):
parse_json_from_llm_output('{"incomplete":')
# ─── Sentiment analyzer contracts ────────────────────────────────────
@pytest.mark.contract
@pytest.mark.unit
class TestSentimentAnalyzerContract:
"""
analyze_sentiment:
- For valid text: returns a dict with "sentiment" and "confidence"
- "sentiment" is one of ["positive", "negative", "neutral"]
- "confidence" is a float in [0.0, 1.0]
- For empty text: raises ValueError (without calling the LLM)
"""
@patch("app.sentiment.client.chat.completions.create")
def test_returns_dict_with_required_keys(self, mock_create, sentiment_positive_response):
mock_create.return_value = sentiment_positive_response
result = analyze_sentiment("I love this!")
assert isinstance(result, dict)
assert "sentiment" in result
assert "confidence" in result
@patch("app.sentiment.client.chat.completions.create")
def test_sentiment_is_valid_value(self, mock_create, make_llm_response):
mock_create.return_value = make_llm_response(
'{"sentiment": "positive", "confidence": 0.9}'
)
result = analyze_sentiment("Some text")
assert result["sentiment"] in {"positive", "negative", "neutral"}
@patch("app.sentiment.client.chat.completions.create")
def test_confidence_is_float_in_range(self, mock_create, make_llm_response):
mock_create.return_value = make_llm_response(
'{"sentiment": "negative", "confidence": 0.8}'
)
result = analyze_sentiment("Some text")
assert isinstance(result["confidence"], (int, float))
assert 0.0 <= result["confidence"] <= 1.0
def test_empty_text_raises_value_error(self):
with pytest.raises(ValueError):
analyze_sentiment("")
@patch("app.sentiment.client.chat.completions.create")
def test_empty_text_does_not_call_llm(self, mock_create):
with pytest.raises(ValueError):
analyze_sentiment("")
mock_create.assert_not_called()
@patch("app.sentiment.client.chat.completions.create")
def test_llm_called_exactly_once_per_analysis(self, mock_create, sentiment_positive_response):
mock_create.return_value = sentiment_positive_response
analyze_sentiment("Some text")
mock_create.assert_called_once()Step 6: Behavioral tests
# tests/unit/behavioral/test_behavioral.py
"""
Behavioral tests: verify behavior properties.
They don't compare exact values — they verify invariants.
"""
import pytest
from unittest.mock import patch
from app.sentiment import analyze_sentiment, validate_input
@pytest.mark.behavioral
@pytest.mark.unit
class TestSentimentBehavior:
@patch("app.sentiment.client.chat.completions.create")
def test_confidence_is_not_zero_for_valid_input(self, mock_create, sentiment_positive_response):
"""For valid input, confidence must not be 0."""
mock_create.return_value = sentiment_positive_response
result = analyze_sentiment("Test text")
assert result["confidence"] > 0.0
@patch("app.sentiment.client.chat.completions.create")
def test_same_input_produces_same_output_with_mock(self, mock_create, sentiment_positive_response):
"""With mocks, the same input always produces the same output."""
mock_create.return_value = sentiment_positive_response
result1 = analyze_sentiment("Test text")
result2 = analyze_sentiment("Test text")
assert result1 == result2
@pytest.mark.behavioral
@pytest.mark.unit
class TestValidateInputBehavior:
def test_strips_surrounding_whitespace(self):
"""The validator removes leading and trailing spaces."""
result = validate_input(" text with spaces ")
assert not result.startswith(" ")
assert not result.endswith(" ")
def test_truncates_very_long_text(self):
"""Very long text is truncated (doesn't raise an exception)."""
very_long = "a" * 10000
result = validate_input(very_long, max_chars=5000)
assert len(result) <= 5000
def test_preserves_content_after_validation(self):
"""The text content is preserved after validation."""
text = "This is a specific test phrase."
result = validate_input(text)
# The content must be present
assert "specific test phrase" in resultStep 7: Regression tests
# tests/unit/regression/test_regression.py
"""
Regression tests: prevent known bugs from returning.
Each test has a comment with the date and description of the bug.
DO NOT DELETE these tests.
"""
import pytest
import json
from unittest.mock import patch
from app.parsers import parse_json_from_llm_output
from app.sentiment import analyze_sentiment
@pytest.mark.regression
@pytest.mark.unit
class TestParserRegression:
def test_handles_json_with_unicode_chars(self):
"""
Regression: the parser failed with Unicode characters (ñ, é, ü, emojis).
The problem was in the encoding when extracting the JSON from the text.
"""
raw = '{"texto": "El niño aprendió inglés y matemáticas"}'
result = parse_json_from_llm_output(raw)
assert "ñ" in result["texto"]
assert "é" in result["texto"]
def test_handles_json_with_emojis(self):
"""
Regression: the parser failed when the JSON values contained emojis.
"""
raw = '{"message": "Great product! 🎉 Highly recommended! 🚀"}'
result = parse_json_from_llm_output(raw)
assert "🎉" in result["message"]
def test_handles_nested_quotes(self):
"""
Regression: the parser failed with escaped quotes inside the JSON.
"""
raw = '{"quote": "She said \\"hello\\" to me"}'
result = parse_json_from_llm_output(raw)
assert "hello" in result["quote"]
@pytest.mark.regression
@pytest.mark.unit
class TestSentimentRegression:
@patch("app.sentiment.client.chat.completions.create")
def test_handles_very_long_text_without_error(self, mock_create, sentiment_positive_response):
"""
Regression: for very long texts, the function must truncate the input
instead of sending excessive tokens (which caused timeouts).
"""
mock_create.return_value = sentiment_positive_response
very_long_text = "word " * 5000 # ~25,000 characters
# Must not raise an exception — must truncate and process
result = analyze_sentiment(very_long_text)
assert result is not None
assert "sentiment" in result
@patch("app.sentiment.client.chat.completions.create")
def test_handles_text_with_single_quotes(self, mock_create, sentiment_positive_response):
"""
Regression: texts with single quotes caused problems in the
prompt construction with f-strings.
"""
mock_create.return_value = sentiment_positive_response
text_with_quotes = "I'm really happy with this product! It's amazing!"
result = analyze_sentiment(text_with_quotes)
assert result is not None
assert "sentiment" in resultStep 8: Verify that everything works
Run the complete suite
# All tests without integration (costs nothing)
pytest -m "not integration" -v
# Expected output:
# tests/smoke/test_smoke.py::TestAppSmoke::test_config_module_loads PASSED
# tests/smoke/test_smoke.py::TestAppSmoke::test_parsers_module_loads PASSED
# tests/smoke/test_smoke.py::TestAppSmoke::test_sentiment_module_loads PASSED
# tests/smoke/test_smoke.py::TestAPISmoke::test_health_endpoint_returns_200 PASSED
# tests/smoke/test_smoke.py::TestAPISmoke::test_health_response_has_status PASSED
# tests/smoke/test_smoke.py::TestAPISmoke::test_analyze_endpoint_exists PASSED
# tests/unit/contracts/test_contracts.py::TestParserContract::... PASSED
# tests/unit/contracts/test_contracts.py::TestSentimentAnalyzerContract::... PASSED
# tests/unit/behavioral/test_behavioral.py::... PASSED
# tests/unit/regression/test_regression.py::... PASSED
#
# ============= XX passed in X.XXs =============Verify coverage
pytest -m "not integration" --cov=app --cov-report=term-missing
# Expected output:
# Name Stmts Miss Cover Missing
# -------------------------------------------------
# app/__init__.py 0 0 100%
# app/config.py 12 0 100%
# app/parsers.py 22 0 100%
# app/sentiment.py 28 4 86% 45-48 (real LLM call)
# app/main.py 18 2 89%
# -------------------------------------------------
# TOTAL 80 6 93%Useful commands
# Only smoke (fast sanity check)
pytest -m smoke -v
# Only contracts (after a prompt change)
pytest -m contract -v
# Only regressions (before deploy)
pytest -m regression -v
# Stop at the first failure (debug)
pytest -x -v
# See tests without running (dry run)
pytest --collect-only
# Run a specific test
pytest tests/unit/contracts/test_contracts.py::TestParserContract::test_returns_dict_for_valid_json -vComparison: Before vs After
| Aspect | Before (no tests) | After (with a Test Suite) |
|---|---|---|
| Prompt change | Fear, manual testing | CI catches regressions in seconds |
| Bug in production | "Let's see what happened..." | Regression test added to prevent it |
| Refactoring | Impossible without fear | Safe if the tests pass |
| Code coverage | 0% | >80% |
| Time to verify | 30+ manual minutes | <30 seconds (pytest -m "not integration") |
Troubleshooting
Problem: ModuleNotFoundError: No module named 'app'
Cause: src/ isn't on the Python path.
Solution: The conftest.py has sys.path.insert(0, os.path.join(..., '..', 'src')). Verify that the conftest loads: pytest --co -q should list all the tests without error.
Problem: pytest.ini isn't detected / markers don't work.
Cause: pytest.ini isn't in the project root.
Solution: pytest.ini must be in the same directory from which you run pytest. Use pytest --co to see whether the configuration loads.
Problem: All the integration tests run even when you use pytest -m "not integration".
Cause: The integration tests aren't marked with @pytest.mark.integration.
Solution: Make sure all the tests in tests/integration/ have @pytest.mark.integration or pytestmark = pytest.mark.integration at the start of the file.
Problem: The set_test_environment fixture doesn't work — the tests call the real API.
Cause: The real OPENAI_API_KEY is in the .env and loads before the monkeypatch.
Solution: In .env.test (different from .env) put OPENAI_API_KEY=sk-test-fake. Or use load_dotenv(".env.test") in the conftest.py.
Problem: The FastAPI smoke tests fail with AppNotStarted.
Cause: The test_client fixture has scope="module" but it's used in different classes.
Solution: Change to scope="function" for more flexibility, or make sure all the tests that use test_client are in the same module.
Completion checklist
Verify that you completed the project:
- Structure: directories
smoke/,unit/contracts/,unit/behavioral/,unit/regression/,integration/created - pytest.ini: markers registered, testpaths configured
- helpers.py: function
create_openai_chat_response()working - conftest.py: fixtures
make_llm_response,mock_openai_client,test_client, predefined responses - Smoke tests: at least 5 (2 for modules + 3 for the API)
- Contract tests: parser (4 tests) + sentiment analyzer (5 tests)
- Behavioral tests: at least 3 tests for invariant properties
- Regression tests: at least 3 bug tests (even if they're "preventive")
-
pytest -m "not integration"runs in <30 seconds with everything green - Coverage:
pytest --cov=app --cov-report=term-missingshows >80%
Optional extensions
If you have more time and want to go further:
Extension 1: CI with GitHub Actions
# .github/workflows/tests.yml
name: Tests
on: [push, pull_request]
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with: { python-version: "3.11" }
- run: pip install -r requirements.txt
- run: pytest -m "not integration" --cov=app --cov-fail-under=80Extension 2: Pre-commit hook
# .git/hooks/pre-commit (make executable with chmod +x)
#!/bin/sh
pytest -m "smoke or contract" --no-header -qExtension 3: Tests for the FastAPI endpoint
# tests/unit/contracts/test_api_contracts.py
@pytest.mark.contract
class TestAPIContracts:
@patch("app.main.analyze_sentiment")
def test_analyze_endpoint_returns_sentiment_schema(self, mock_analyze, test_client):
mock_analyze.return_value = {"sentiment": "positive", "confidence": 0.9}
response = test_client.post("/analyze", json={"text": "Great!"})
assert response.status_code == 200
data = response.json()
assert "sentiment" in data
assert "confidence" in data
def test_analyze_endpoint_rejects_empty_text(self, test_client):
response = test_client.post("/analyze", json={"text": ""})
assert response.status_code == 400Additional resources
- pytest Documentation — Complete pytest reference
- pytest-cov — Coverage reports
- FastAPI Testing — FastAPI's TestClient
- GitHub Actions Python — CI for Python projects
- Test structure best practices — pytest best practices