Scaling Patterns

Horizontal Scaling

# Multiple workers processing in parallel
# - Worker 1: Processes batch 1-1000
# - Worker 2: Processes batch 1001-2000
# - Worker 3: Processes batch 2001-3000

---

Message Queue (Celery + Redis)

from celery import Celery

app = Celery('embeddings', broker='redis://localhost:6379')

@app.task
def generate_embedding_async(text):
    """Async task"""
    embedding = get_embedding(text)
    save_to_db(embedding)
    return embedding

# Enqueue
result = generate_embedding_async.delay("Python is great")

# Check status
if result.ready():
    embedding = result.get()

---

Batch Processing

def process_batch(texts, batch_size=100):
    """Efficient batch processing"""
    for i in range(0, len(texts), batch_size):
        batch = texts[i:i+batch_size]
        
        # API call (batch)
        embeddings = client.embeddings.create(
            model="text-embedding-3-small",
            input=batch
        ).data
        
        # Save
        save_embeddings_bulk(embeddings)
        
        # Rate limit
        time.sleep(0.1)

---

Load Balancing

# Multiple API keys (round-robin)
api_keys = [key1, key2, key3]
current_key_idx = 0

def get_client():
    global current_key_idx
    client = OpenAI(api_key=api_keys[current_key_idx])
    current_key_idx = (current_key_idx + 1) % len(api_keys)
    return client

---

Summary

• ✅ Horizontal: Multiple workers
• ✅ Queue: Celery + Redis
• ✅ Batch: Process 100-1000 at once
• ✅ Load balancing: Round-robin API keys

---

Module 7 - Capsule 05