Module 8: Unified AI Client — Final integrating project

Module 8: Unified AI Client — Final integrating project

Next →

Welcome to the close of the path. So far you know five ways to access an LLM, you know how to choose between them with data, and you have the Module 7 decision tool that automates the choice. One piece is missing: a Python client that abstracts all the providers behind a single interface, so your application code isn't tied to any of them.

That's what you're going to build in this module. And you're going to do it at the level of something you put in your portfolio: solid architecture, automatic fallback, declarative configuration, monitoring, and tests.

By the end of the module you'll have:

  • A Python library of your own, unified_ai_client, that abstracts OpenAI, OpenRouter, Ollama, LM Studio and Modal
  • Automatic fallback between providers when one fails
  • Priority routing (cost-first, quality-first, balanced)
  • Observable metrics (latency, cost, errors per provider)
  • Contract and behavior tests that validate each provider
  • Documentation so your team (or your future self) can use it without relearning it

It's the deliverable that closes the path and the one you take to interviews as proof of mastery of the topic.

Why this module matters professionally

Four concrete reasons:

1. It's exactly the pattern that serious startups implement. Companies like Vercel AI SDK, LiteLLM, LangChain, Portkey — they all build equivalent abstractions. Having your own (even if simpler) shows that you understand the problem, not just the usage.

2. You reduce vendor lock-in in production. If your CTO asks you "how dependent are we on OpenAI?", the right answer is "we can switch to another provider in an hour". You only achieve that with this abstraction.

3. It enables advanced strategies. Automatic fallback, A/B testing between providers, cost-based routing in production — everything becomes trivial with a unified client.

4. It's defensible code in code review. Unlike a demo that only works for you, this has architecture, tests, and recognizable patterns (factory, strategy, adapter). It's senior code, not junior.

Mental model: one interface, several adapters

Let's think about how the pieces are going to interact:

                          ┌─────────────────────────────────┐
                          │   Your application code         │
                          │   client = UnifiedClient(...)   │
                          │   r = client.chat("...")        │
                          └────────────────┬────────────────┘
                                           │
                                  ┌────────▼────────┐
                                  │  UnifiedClient  │
                                  │   (facade)      │
                                  └────────┬────────┘
                                           │
                  ┌────────────────────────┼────────────────────────┐
                  │                        │                        │
            ┌─────▼─────┐          ┌──────▼──────┐         ┌───────▼──────┐
            │OpenAIAdapt│          │OllamaAdapter│         │ ModalAdapter │
            └─────┬─────┘          └──────┬──────┘         └───────┬──────┘
                  │                       │                        │
            ┌─────▼─────┐          ┌──────▼──────┐         ┌───────▼──────┐
            │ OpenAI API│          │  Ollama (local)│      │ Modal (cloud)│
            └───────────┘          └────────────────┘      └──────────────┘

Three design patterns apply naturally:

PatternFor what
AdapterEach provider has its own API; the adapter translates it to your common interface
FactoryYou decide which adapter to use from configuration (provider="openai")
StrategyFallback, cost/quality routing — different strategies behind the same interface

You're going to use all three. Not for academicism — because they solve real problems.

A scenario that illustrates the module

Mike works at a B2B startup. His product uses OpenAI today. The team debates:

  • Product asks to lower costs to scale (head of product)
  • Compliance requires an option for EU deployments (enterprise client)
  • Engineering wants to reduce lock-in (CTO)
  • DevOps needs visibility of spend per provider (finance)

Without abstraction, each of these requirements is an independent project with a major refactor.

With the Unified Client you're going to build, these problems become configuration:

# Regular chat product: OpenAI with fallback to OpenRouter
client_normal = UnifiedClient(
    primary="openai_gpt-4o-mini",
    fallback=["openrouter_mistral", "ollama_mistral"],
    metrics_enabled=True,
)

# EU client: only Modal in an EU region
client_eu = UnifiedClient(
    primary="modal_eu_mistral",
    fallback=["selfhosted_ollama_eu"],
    metrics_enabled=True,
)

# Same call interface
response = client_normal.chat("How does X work?")
response_eu = client_eu.chat("How does X work?")

Provider changes are config changes, not code rewrites. That's what this module delivers.

Connection with the rest of the path

This module is where everything before it becomes useful:

  • Module 1 gave you the framework to decide → it influences the primary you choose
  • Modules 2-6 gave you experience with each provider → each one becomes an adapter
  • Module 7 gave you benchmarks → they inform when to switch providers
  • Module 8 gives you the abstraction to make those changes painlessly

At the end of this module you close a loop: from analysis (M07) to the implementation (M08) that executes your decisions.

Module map

CapsuleTopicWhat you build
01Introduction (this one)Mental model + objectives
02Architecture and designClass diagrams, design decisions, interface contracts
03Base implementationSkeleton: UnifiedClient + first adapter (OpenAI) working
04Fallback strategyAutomatic failover with backoff, simple circuit breaker
05Cost optimizationPriority routing (cost/quality/balanced) based on config
06Monitoring and metricsTracking latency, cost, error rate per provider
07Testing and validationContract tests, mocks, contract testing per provider
08Final projectComplete version with all providers + docs + examples

Each capsule builds on the previous one. At the end of 08 you have something deployable.

What is NOT covered in this module

To keep scope:

  • ❌ Streaming (SSE / async iterators). The client returns complete responses. Streaming adds complexity that deserves its own module.
  • ❌ Function calling / multi-turn tool use. Each provider implements it differently; abstracting it well is a serious project. We see a basic hook, not full support.
  • ❌ Embeddings and other endpoints. The client focuses on chat completion. The same architecture applies to embeddings, but for scope we don't implement it.
  • ❌ Production-grade observability (Datadog, OpenTelemetry). We do simple in-memory metrics. For real production, you connect to your existing observability stack.
  • ❌ Rate limiting / complex circuit breaker. We implement a simple circuit breaker. For serious production, use a dedicated library (tenacity, pybreaker).

Common traps while taking the module

Trap 1 — "I'm going to make the perfect abstraction before implementing." No. Do it iteratively: OpenAI adapter working first, then you add the rest. The optimal abstraction emerges by implementing, not by designing in the abstract.

Trap 2 — "My abstraction supports all the features of all the providers." Impossible and counterproductive. Support the intersection of common features. For exclusive features (e.g., OpenAI's strict JSON mode), expose an escape hatch (provider_specific_options).

Trap 3 — "I copy LiteLLM." LiteLLM exists and it's excellent. If you were going to use it in production, you'd probably take LiteLLM. But building your own teaches you. Your goal isn't to compete with LiteLLM — it's to understand the problem.

Trap 4 — "No tests because it's a demo project." Without tests, it's not a portfolio. It's a script. The difference between the client you show in an interview and the one you're embarrassed to show is testing.

Self-assessment question

Before moving to capsule 02:

  • What's the difference between the Adapter and Factory patterns? What is each one for?
  • Why should the abstraction support the intersection of features and not the union?
  • If your unified client goes down because OpenAI had an outage, what's the difference between fallback and circuit breaker?
Answer guide
  • Adapter: translates one interface into another. Here: the OpenAI/Anthropic/Ollama API is "translated" into your common interface chat(prompt) -> str. Factory: decides which object to create from configuration. Here: given provider="openai", it returns the OpenAIAdapter. A Factory uses Adapters; they're not competitors, they're collaborators.
  • Because if you support the union (all the features of all the providers), your interface becomes enormous and most of it doesn't work for most providers. The intersection (common features) gives you a small interface that always works. For exclusive features, you have an escape hatch.
  • Fallback: when a request fails, retry with another provider. A per-request mechanism. Circuit breaker: if a provider fails repeatedly, stop trying it for a while (you avoid paying timeouts over and over). A stateful mechanism across many requests. The two are complementary.

Evidence of success on completing the module

You'll know you finished well if:

  • ✅ pip install -e . installs your library locally
  • ✅ You can do client = UnifiedClient(provider="openai") or provider="modal" and the same chat(...) call works
  • ✅ If OpenAI goes down, your client automatically tries the next configured provider
  • ✅ client.get_metrics() returns real usage tracking per provider
  • ✅ pytest runs green tests against at least 2 providers (mocks acceptable for those you don't have an API key for)
  • ✅ Your README has copy-pasteable examples that a teammate understands without your help

Next capsule

02 — Architecture and design. Before typing code, we're going to design the interface: what methods UnifiedClient exposes, what contract each Adapter fulfills, how it's configured. Explicit design decisions avoid painful refactors in capsule 03 when you start implementing.

Resources

  1. LiteLLM — reference implementation (more complete than yours; useful for inspiration).
  2. LangChain Chat Models — equivalent abstraction in the LangChain ecosystem.
  3. Vercel AI SDK — TypeScript version of the same pattern.
  4. Design Patterns — Adapter, Factory, Strategy — conceptual refresher.
  5. Architectural Decision Records — to document your design decisions.
← PreviousProject: Decision ToolNext →Architecture and design