Architecture and design

Before typing a single line of code, we're going to design the interface. This isn't perfectionism — it's the difference between implementing once vs. refactoring three times. Every decision you make here saves you work and gives you consistency across the 5 adapters you're going to write.

By the end you'll be able to:

• Decide the contract of UnifiedClient and BaseAdapter with clear types
• Design the configuration flow (in code vs YAML vs env vars)
• Map the Adapter, Factory and Strategy patterns to concrete pieces
• Document design decisions with reasons (not just what, but why)

---

The decisions we're going to make

Five big decisions, in order of impact:

1. What methods does the public interface expose? (minimal + extensible)
2. What type of inputs/outputs? (simple string vs rich structures)
3. How is it configured? (constructor params vs config object vs YAML)
4. How are errors handled? (typed exceptions vs Result wrapper)
5. How are extensions injected (fallback, metrics, retry) without polluting the interface?

Let's go one by one.

---

Decision 1 — The public interface

Option A — Minimal:

class UnifiedClient:
    def __init__(self, provider: str, **kwargs): ...
    def chat(self, prompt: str) -> str: ...

Option B — Rich:

class UnifiedClient:
    def __init__(self, provider: str, **kwargs): ...
    def chat(self, prompt: str, **opts) -> ChatResponse: ...
    def stream_chat(self, prompt: str, **opts) -> Iterator[str]: ...
    def chat_with_history(self, messages: list[Message], **opts) -> ChatResponse: ...
    def embed(self, texts: list[str]) -> list[Embedding]: ...

Decision: something between the two. We start with:

class UnifiedClient:
    def __init__(self, primary: str, fallback: list[str] = [], **opts): ...

    def chat(
        self,
        prompt: str,
        *,
        max_tokens: int = 256,
        temperature: float = 0.7,
        system: str | None = None,
    ) -> ChatResponse: ...

    def chat_with_messages(
        self,
        messages: list[Message],
        **opts
    ) -> ChatResponse: ...

    def get_metrics(self) -> Metrics: ...

Why:

• chat() for the simple case (90% of usage).
• chat_with_messages() for multi-turn (when your app manages history).
• No streaming, embeddings, function calling (explicit M08 scope).
• Small public methods; extensions (fallback, metrics) injected, not new methods.

---

Decision 2 — Inputs and outputs

Simple strings vs structures:

# Option A — strings: simple but loses info
def chat(self, prompt: str) -> str: ...

# Option B — Pydantic: structured but more code
def chat(self, prompt: str) -> ChatResponse:
    ...

class ChatResponse(BaseModel):
    text: str
    model: str
    provider: str
    tokens_input: int
    tokens_output: int
    duration_ms: int
    cost_usd: float | None
    raw_response: dict  # access to the original response if you need it

Decision: Option B (structured).

Why:

• You need tokens and cost for tracking (capsule 06)
• You need provider to know who answered (important with fallback)
• You need raw_response for advanced cases without polluting the base API
• The extra cost of Pydantic is trivial; the value of structured info is high

---

Decision 3 — Configuration

Option A — Constructor params:

client = UnifiedClient(
    primary="openai",
    fallback=["openrouter", "ollama"],
    openai_api_key="sk-...",
    openrouter_api_key="sk-or-...",
)

Option B — Config object:

config = ClientConfig(
    primary=ProviderConfig(name="openai", model="gpt-4o-mini"),
    fallback=[
        ProviderConfig(name="openrouter", model="mistralai/mistral-7b-instruct"),
        ProviderConfig(name="ollama", model="mistral"),
    ],
)
client = UnifiedClient(config)

Option C — External YAML/JSON:

client = UnifiedClient.from_yaml("clients.yaml")

Decision: A for the API in code, C for production. We implement both.

Why:

• A is ergonomic for development and notebooks
• C is the right thing for production (config in a file, not hardcoded)
• B introduces ceremony that isn't justified in the simple case
• API keys always come from env vars or a secret manager, not constructor params

YAML structure:

# clients.yaml
primary: openai-gpt4o-mini
fallback:
  - openrouter-mistral
  - ollama-mistral

providers:
  openai-gpt4o-mini:
    type: openai
    model: gpt-4o-mini
    api_key_env: OPENAI_API_KEY
    base_url: https://api.openai.com/v1

  openrouter-mistral:
    type: openai_compatible
    model: mistralai/mistral-7b-instruct
    api_key_env: OPENROUTER_API_KEY
    base_url: https://openrouter.ai/api/v1

  ollama-mistral:
    type: openai_compatible
    model: mistral
    api_key_env: OLLAMA_API_KEY  # placeholder; Ollama ignores it
    base_url: http://localhost:11434/v1

  modal-mistral:
    type: modal_custom
    base_url_env: MODAL_BASE_URL
    api_token_env: MODAL_API_TOKEN

---

Decision 4 — Error handling

Option A — Typed exceptions:

class UnifiedClientError(Exception): ...
class ProviderError(UnifiedClientError): ...
class AllProvidersFailedError(UnifiedClientError): ...
class RateLimitError(ProviderError): ...
class AuthError(ProviderError): ...
class TimeoutError(ProviderError): ...

Option B — Result wrapper:

@dataclass
class Result:
    success: bool
    response: ChatResponse | None
    error: str | None

Decision: Option A (typed exceptions).

Why:

• Idiomatic Python uses exceptions
• Specific types allow try/except RateLimitError for selective handling
• Result wrappers are common in Rust/Go, not in Python — they end up feeling foreign
• Stack traces help you debug

---

Decision 5 — Injecting extensions

We need to add fallback, metrics, retry — but without making the class large and messy. Two options:

Option A — Decorators / mixins:

client = with_metrics(with_fallback(BaseUnifiedClient(primary="openai")))

Option B — Declared composition:

client = UnifiedClient(
    primary="openai",
    fallback=["openrouter"],
    metrics_enabled=True,
    retry_on=[RateLimitError, TimeoutError],
)

Decision: Option B (declared composition).

Why:

• Option A is elegant but less discoverable
• Option B works as a "config object by default"; explicit
• The complexity goes inside UnifiedClient, not in the user API

---

The resulting class diagram

            ┌─────────────────────────────────────┐
            │           UnifiedClient             │
            │  - primary: BaseAdapter             │
            │  - fallback: list[BaseAdapter]      │
            │  - metrics: MetricsCollector        │
            │  + chat(prompt, **opts) -> Response │
            │  + chat_with_messages(...) -> Resp  │
            │  + get_metrics() -> Metrics         │
            └────────────────┬────────────────────┘
                             │ delegates to
                             ▼
            ┌─────────────────────────────────────┐
            │   BaseAdapter (ABC)                 │
            │  + chat(...) -> Response (abstract) │
            │  + name: str                        │
            └────────────────┬────────────────────┘
                             │
       ┌─────────────────────┼─────────────────────┐
       ▼                     ▼                     ▼
┌──────────────┐      ┌──────────────┐      ┌──────────────┐
│OpenAIAdapter │      │OllamaAdapter │      │ModalAdapter  │
│              │      │              │      │              │
└──────────────┘      └──────────────┘      └──────────────┘
       │                     │                     │
       ▼                     ▼                     ▼
   OpenAI API            Ollama API            Modal endpoint

┌──────────────────────────┐
│  ProviderFactory         │
│  + create(config) ->     │
│    BaseAdapter           │
└──────────────────────────┘

ProviderFactory maps config → an instance of the correct adapter.

┌──────────────────────────┐
│   MetricsCollector       │
│  + record(provider, ...) │
│  + summary() -> Metrics  │
└──────────────────────────┘

MetricsCollector is a collaborator that UnifiedClient optionally uses (capsule 06).

---

The data models (Pydantic)

# unified_ai_client/models.py
from pydantic import BaseModel, Field
from typing import Literal

class Message(BaseModel):
    role: Literal["system", "user", "assistant"]
    content: str

class ChatResponse(BaseModel):
    text: str
    model: str
    provider: str
    tokens_input: int = 0
    tokens_output: int = 0
    duration_ms: int = 0
    cost_usd: float | None = None
    raw_response: dict | None = None

class ProviderConfig(BaseModel):
    name: str           # unique identifier of this config (e.g. "openai-gpt4o-mini")
    type: Literal["openai", "openai_compatible", "modal_custom"]
    model: str | None = None
    base_url: str | None = None
    api_key_env: str | None = None
    base_url_env: str | None = None
    api_token_env: str | None = None
    # Optional pricing for metrics:
    price_input_per_1m: float | None = None
    price_output_per_1m: float | None = None

class ClientConfig(BaseModel):
    primary: str
    fallback: list[str] = Field(default_factory=list)
    providers: dict[str, ProviderConfig]
    metrics_enabled: bool = True

---

The BaseAdapter contract

# unified_ai_client/adapters/base.py
from abc import ABC, abstractmethod
from ..models import Message, ChatResponse, ProviderConfig

class BaseAdapter(ABC):
    def __init__(self, config: ProviderConfig):
        self.config = config
        self.name = config.name

    @abstractmethod
    def chat(
        self,
        messages: list[Message],
        max_tokens: int = 256,
        temperature: float = 0.7,
    ) -> ChatResponse:
        """Generates a response from messages. Per-provider implementation."""
        ...

Rules that all adapters must follow:

1. They accept list[Message], not prompt: str (it's UnifiedClient's responsibility to convert a simple prompt to [Message(role="user", content=prompt)])
2. They return a complete ChatResponse (text + tokens + duration + provider name)
3. They raise typed exceptions (RateLimitError, AuthError, etc.) on error
4. They measure their own duration_ms internally (with time.perf_counter())

---

Project file structure

unified_ai_client/
├── __init__.py
├── client.py             # UnifiedClient
├── factory.py            # ProviderFactory
├── exceptions.py         # typed exceptions
├── models.py             # ChatResponse, Message, configs
├── metrics.py            # MetricsCollector
├── adapters/
│   ├── __init__.py
│   ├── base.py           # BaseAdapter
│   ├── openai_adapter.py
│   ├── ollama_adapter.py
│   └── modal_adapter.py
└── tests/
    ├── test_unified_client.py
    ├── test_adapters.py
    ├── test_fallback.py
    └── test_metrics.py

pyproject.toml            # package metadata
README.md                 # docs and examples
clients.yaml.example      # config example

---

Common traps in the design

Trap 1 — "Designing too completely before implementing." Many people design 5 features and only implement 1. Build an MVP that works (capsule 03) before continuing to add layers.

Trap 2 — "Each feature in its own subclass." UnifiedClientWithFallback, UnifiedClientWithMetrics, UnifiedClientWithRetryAndFallback → combinatorial explosion. That's why we decided on composition with flags instead of a subclass hierarchy.

Trap 3 — "Support for all the features of all the providers." If OpenAI has 47 parameters and Anthropic has 38, don't expose 85 parameters in chat(). Expose the common ones (max_tokens, temperature, system). The exclusive stuff goes in provider_specific_options or is avoided.

Trap 4 — "Types without Pydantic or dataclasses." "My return is a dict with random keys" → maintenance hell. Use Pydantic from day 1.

Trap 5 — "API key in code." OpenAIAdapter(api_key="sk-real-key"). Never. Always via env var (which YAML references with api_key_env).

---

Design exercise

Before implementing (capsule 03), answer for your own project:

1. Are you going to support streaming? If so, do you add stream_chat() or unify it with a stream=True parameter?
2. Your product receives prompts in multiple languages. Do your Pydantic models handle that or assume ASCII?
3. Should your fallback be automatic or opt-in per request (chat(prompt, use_fallback=False))?
4. Some providers charge for a failed request. Does your metrics tracking include errors? Does it charge for errors too?

There are no single answers — only explicit decisions. Write down yours.

Defaults we're going to use in the following capsules

1. No streaming (explicit scope). You add it later yourself if you need it.
2. UTF-8 by default in Pydantic strings (handled automatically).
3. Automatic fallback. A chat(prompt) call tries primary → first fallback → second fallback → fails.
4. Errors are counted in metrics but not charged (some providers do charge; adjust if it applies).

---

Summary

You decided:

• ✅ Public interface: chat() + chat_with_messages() + get_metrics()
• ✅ Inputs/outputs: Pydantic models (Message, ChatResponse)
• ✅ Configuration: constructor params + YAML for production
• ✅ Error handling: typed exceptions (RateLimitError, AuthError, etc.)
• ✅ Declared composition (fallback, metrics as flags) vs subclass hierarchy

Checkpoint: if you have the 5 decisions above clear and can draw the class diagram without looking, you're ready to implement.

---

Next capsule

03 — Base implementation. We move from design to code. You build the skeleton of the package + the first adapter (OpenAI) + the minimum viable UnifiedClient. By the end of the capsule you'll have something importable and executable, even if only with one provider.

---

Resources

1. Refactoring.guru — Adapter, Factory, Strategy patterns — canonical definitions with Python examples.
2. Pydantic v2 docs — models for your dataclasses.
3. LiteLLM source code — open source implementation; study its Router for fallback.
4. Python typing — Protocol vs ABC — alternative to ABC with duck typing.
5. Architectural Decision Records template — to document your decisions.