Composition Strategies

Averaging (Mean)

def compose_mean(embeddings):
    """Simple average"""
    return np.mean(embeddings, axis=0)

# Document = average of its sentences
sent_embs = [get_emb(s) for s in sentences]
doc_emb = compose_mean(sent_embs)

---

Weighted Average

def compose_weighted(embeddings, weights):
    """Weighted average"""
    weighted = [w * emb for w, emb in zip(weights, embeddings)]
    return np.sum(weighted, axis=0)

# Example: important sentences weigh more
weights = [0.5, 0.3, 0.2]  # The first sentence is more important
doc_emb = compose_weighted(sent_embs, weights)

---

Max Pooling

def compose_max(embeddings):
    """Max per dimension"""
    return np.max(embeddings, axis=0)

# Captures the most salient features
doc_emb = compose_max(sent_embs)

---

Concatenation

def compose_concat(embeddings):
    """Concatenate vectors"""
    return np.concatenate(embeddings)

# WARNING: Increases dimensionality
# 3 embeddings of 1536 dims → 4608 dims

---

Summary

Strategy	Final dims	When to use
Mean	Same	Default (balanced)
Weighted	Same	Variable importance
Max	Same	Salient features
Concat	Sum	Preserve everything (expensive)

---

Module 6 - Capsule 04