Ensamblando el pipeline de ingestión

Descripción

Todo lo que sigue en este proyecto —el índice, la tool, el agente, la reingesta, la evaluación, la capa operativa— depende de una sola cosa: que el corpus de partida sea exactamente el mismo, carácter por carácter, en cada pieza. Esta lección deja ese cimiento en tu propio directorio de trabajo: el bloque RAW_DOCS + el chunker completo, copiado verbatim del Módulo 1, sin una coma de diferencia. No hay ingeniería nueva que aprender aquí —cada parser, cada regla de limpieza, cada estrategia de chunking ya la construiste y probaste en su propio módulo—; el trabajo de esta lección es la disciplina de reproducirlo exacto, y confirmar con tu propia ejecución que el resultado sigue siendo 57 chunks sobre 13 documentos.

Conexión con el módulo

Esta es la primera pieza del pipeline del proyecto: reservo_corpus.py es la base sobre la que se construyen, sin excepción, las seis lecciones siguientes. Un error de una sola palabra aquí se propaga a todo el resto del proyecto —cambiaría scores de BM25, resultados de search_docs, y hasta el score de evaluación de la Lección 06—, así que esta lección termina con una verificación explícita antes de avanzar.

Analogía: la sala de recepción de documentos, el primer día

De las siete personas de la analogía del mostrador completo, esta lección es la primera que entra a trabajar: quien recibe los documentos crudos de Reservo —las políticas en markdown, los manuales en HTML, el manual de operaciones con el desorden típico de una extracción automática— y los deja limpios, chunkeados y con su ficha de metadatos, listos para que el resto del equipo los use. Si esta persona etiqueta mal un documento o pierde un párrafo en la limpieza, nadie que trabaje después de ella —ni quien arma el catálogo, ni quien atiende el mostrador, ni quien audita la calidad— puede corregir ese error aguas abajo. Por eso el trabajo de hoy termina, antes de nada más, con una verificación explícita de que la recepción se hizo bien.

Paso 1: un directorio de trabajo desechable

mkdir -p "$(mktemp -d)/reservo-capstone" && cd "$_"
pwd

Qué esperar: una ruta nueva bajo el directorio temporal del sistema, vacía — el mismo patrón de directorio desechable que ya usaste en los mini-proyectos de los Módulos 2, 4, 5 y 6.

Paso 2: reservo_corpus.py, verbatim

Guarda este archivo completo, tal cual, en tu directorio de trabajo. Es exactamente el bloque canónico del Módulo 1 — 13 documentos crudos (RAW_DOCS) y el chunker completo (parse_markdown, parse_html, clean_text, sectionize_cleaned, chunk_by_sentence, chunk_by_structure, ingest_document, build_corpus) — sin ningún cambio:

# reservo_corpus.py
# =============================================================================
# CORPUS CANONICO DE RESERVO -- bloque unico, auto-contenido y ejecutable.
# Python 3.14.0 + stdlib (re, html.parser, dataclasses) -- sin numpy, sin red.
# INGESTION_DATE fija (sin datetime.now(), sin random).
# =============================================================================
import re
from dataclasses import dataclass
from html.parser import HTMLParser

INGESTION_DATE = "2026-01-15"

RAW_DOCS: dict[str, tuple[str, str]] = {}  # doc_id -> (fmt, raw_text)

RAW_DOCS["cancellation-policy"] = ("md", """# Cancellation Policy

## Basic Tier Cancellation Window

Basic members can cancel a booking up to 24 hours before the reserved start time with no penalty. Cancellations made less than 24 hours in advance forfeit the full booking amount.

## Pro Tier Cancellation Window

Pro members get a shorter, friendlier window: cancellations up to 4 hours before the reserved start time are free of charge. This is one of the perks of the pro tier, alongside the 20% discount on hourly rates.

## How to Cancel

Cancellations go through the same booking system used to reserve the room. There is no phone line for cancellations; the system timestamp is what determines whether the cancellation was made in time.

## Related Policies

See `refund-policy` for what happens to the money once a cancellation is processed, and `no-show-policy` for what happens if you simply do not show up without cancelling.
""")

RAW_DOCS["no-show-policy"] = ("md", """# No-Show Policy

## What Counts as a No-Show

A no-show is a booking where the member never checks in during the reserved hours and never cancelled beforehand. This is different from a late cancellation, which is covered in `cancellation-policy`.

## What Happens on a No-Show

No-shows are charged the full amount of the booking. Unlike a late cancellation, there is no partial leniency: the no-show fee equals the entire reserved price, and it is never refunded under `refund-policy`.

## Repeated No-Shows

Members with three or more no-shows in a rolling 30-day window lose the ability to book same-day reservations; all future bookings must be made at least 24 hours in advance until the pattern clears.

## Why This Policy Exists

Rooms held for a no-show cannot be re-offered to another member during that window, so the fee reflects real lost capacity, not a punitive charge.
""")

RAW_DOCS["refund-policy"] = ("md", """# Refund Policy

## What Qualifies for a Refund

A refund applies when a booking is cancelled within the free window described in `cancellation-policy`, or when Reservo cancels a confirmed booking because of a facility issue, such as a maintenance problem or a power outage.

## Refund Amount and Timing

Eligible refunds return the full amount charged for the booking to the original payment method. Processing takes up to 5 business days once the cancellation is confirmed in the booking system.

## What Does Not Qualify for a Refund

Cancellations made outside the free window are not eligible; the booking amount is forfeited under `cancellation-policy`. No-shows are never refunded, regardless of membership tier; see `no-show-policy` for that separate case.

## How to Request a Refund

Eligible refunds are issued automatically once a qualifying cancellation is processed; members do not need to submit a separate request. Questions about a specific refund go to member support through the booking system.
""")

RAW_DOCS["booking-faq"] = ("md", """# Booking FAQ

## How Do I Book a Room?

Bookings are made through the Reservo booking system by choosing a room, a date, and a start and end time. A confirmation appears immediately, and the room is held exclusively for that window.

## Is a Deposit Required to Book?

Yes. A deposit equal to the full session amount is charged at the time of booking, through the payment method on file; see `payment-methods-faq` for what is accepted.

## Can I Book More Than One Room at a Time?

Yes, a member can hold bookings in multiple rooms at once, as long as the times do not overlap for the same member. Each room booking is billed and cancelled separately.

## How Far in Advance Can I Book a Room?

Rooms can be booked up to 60 days in advance. Same-day booking is allowed, subject to the same-day restriction described in `no-show-policy` for members with repeated no-shows.
""")

RAW_DOCS["membership-tiers-faq"] = ("md", """# Membership Tiers FAQ

## What Is the Difference Between Basic and Pro?

Basic is the default tier for every new member, with no monthly fee. Pro is a paid upgrade that adds a shorter cancellation window, see `cancellation-policy`, and a discount on every booking.

## How Much Discount Does the Pro Tier Get?

Pro members receive a 20% discount on the hourly rate of every room, applied automatically at checkout. No code or coupon is needed; the discount is tied to the membership tier on the account.

## How Do I Upgrade to Pro?

Upgrading to Pro takes effect immediately from the account settings page. The new cancellation window and the 20% discount apply starting with the very next booking made after the upgrade.

## Can I Downgrade Back to Basic?

Yes, at any time from the account settings page. Downgrading takes effect on the next booking; any booking already confirmed under Pro keeps its Pro-tier terms.
""")

RAW_DOCS["payment-methods-faq"] = ("md", """# Payment Methods FAQ

## What Payment Methods Does Reservo Accept?

Reservo accepts major credit and debit cards on file with the account. The same card charged for a booking deposit is used automatically for any no-show or late-cancellation charge.

## Does Reservo Accept Cash?

No. All bookings, deposits, and no-show charges are processed electronically through the payment method on file.

## What Happens if a Card Is Declined?

A declined card cancels the booking hold immediately; the room is released back to the schedule. The member is notified and can retry with the same card or add a different one.

## Can I Split a Payment Between Two Cards?

No, a single booking can only be charged to one card on file at a time. Members who want to change which card is used should update the default payment method before booking.
""")

RAW_DOCS["wifi-and-equipment-faq"] = ("md", """# Wifi and Equipment FAQ

## Is Wifi Included in Every Room?

Yes, building-wide wifi reaches every room, from Phonebooth to Boardroom, at no extra cost. Coverage is the same in every room regardless of size or hourly rate.

## What Is the Wifi Network Name and Password?

The network name and password are posted on a card inside each room and also shown on the booking confirmation screen. The password rotates monthly for security.

## What Common Equipment Is Available Outside the Rooms?

The building shares a printer and a water station on the ground floor, available to any member with an active booking. Room-specific equipment is listed in each room manual.

## Who Do I Contact if the Wifi Is Down?

Report a wifi outage through the booking system's support option; staff follow the internal reset procedure and typically restore the connection within a few minutes.
""")

RAW_DOCS["focus-room-manual"] = ("html", """<!DOCTYPE html>
<html>
<head><title>Focus Room Manual</title></head>
<body>
<h1>Focus Room</h1>
<p>Focus is Reservo's single-occupancy room, designed for calls and deep work
that needs a closed door. It is the smallest and least expensive room in the
building.</p>
<h2>Capacity &amp; Layout</h2>
<p>Capacity: 1 person. The room has one desk, one chair, and a soundproofed
door. There is no window, by design, to minimize visual distraction.</p>
<h2>Equipment</h2>
<ul>
<li>27-inch monitor with HDMI input</li>
<li>Adjustable desk lamp</li>
<li>Wall outlet with two USB-C ports</li>
<li>Building-wide wifi (see wifi-and-equipment-faq)</li>
</ul>
<h2>Booking &amp; Rate</h2>
<p>Base rate: $25.00 per hour (2500 cents), the lowest rate in the building.
Pro members receive the standard 20% discount on every booking.</p>
<h2>House Rules</h2>
<p>Focus is not soundproof against phone ringtones; members are asked to keep
devices on silent. Food is allowed but no hot meals, due to the room's small
size and lack of ventilation.</p>
</body>
</html>
""")

RAW_DOCS["studio-room-manual"] = ("html", """<!DOCTYPE html>
<html>
<head><title>Studio Room Manual</title></head>
<body>
<h1>Studio Room</h1>
<p>Studio is Reservo's small-team room, built for a working session that
needs a table and a whiteboard rather than a single desk. It sits between
Focus and Boardroom in both size and price.</p>
<h2>Capacity &amp; Layout</h2>
<p>Capacity: 4 people. The room has a round table, four chairs, and a
wall-mounted whiteboard. A large window faces the courtyard.</p>
<h2>Equipment</h2>
<ul>
<li>43-inch monitor with HDMI and USB-C input</li>
<li>Wall-mounted whiteboard with markers</li>
<li>Conference speakerphone</li>
<li>Building-wide wifi (see wifi-and-equipment-faq)</li>
</ul>
<h2>Booking &amp; Rate</h2>
<p>Base rate: $40.00 per hour (4000 cents). Pro members receive the standard
20% discount on every booking.</p>
<h2>House Rules</h2>
<p>Studio can be booked for up to 4 consecutive hours per reservation. The
whiteboard must be wiped clean before the next booking begins.</p>
</body>
</html>
""")

RAW_DOCS["boardroom-room-manual"] = ("html", """<!DOCTYPE html>
<html>
<head><title>Boardroom Room Manual</title></head>
<body>
<h1>Boardroom</h1>
<p>Boardroom is Reservo's largest room, reserved for formal meetings,
client presentations, and full-team gatherings. It is the only room with a
dedicated presentation screen.</p>
<h2>Capacity &amp; Layout</h2>
<p>Capacity: 10 people. The room has a long table, ten chairs, and a
wall-mounted presentation screen at the head of the table.</p>
<h2>Equipment</h2>
<ul>
<li>75-inch presentation screen with HDMI and wireless casting</li>
<li>Conference speakerphone with ceiling microphones</li>
<li>Wall-mounted whiteboard with markers</li>
<li>Building-wide wifi (see wifi-and-equipment-faq)</li>
</ul>
<h2>Booking &amp; Rate</h2>
<p>Base rate: $80.00 per hour (8000 cents), the highest rate in the
building. Pro members receive the standard 20% discount on every booking.</p>
<h2>House Rules</h2>
<p>Boardroom receives a full clean every evening regardless of usage,
because of its size. Food is allowed only during bookings longer than 2
hours, and must be cleared before the room is released.</p>
</body>
</html>
""")

RAW_DOCS["lounge-room-manual"] = ("html", """<!DOCTYPE html>
<html>
<head><title>Lounge Room Manual</title></head>
<body>
<h1>Lounge</h1>
<p>Lounge is Reservo's informal meeting room, built for a relaxed
conversation rather than a formal presentation. It is open-plan, with no
door separating it from the hallway.</p>
<h2>Capacity &amp; Layout</h2>
<p>Capacity: 6 people. The room has two low sofas, a coffee table, and
extra chairs stacked against the wall for larger groups.</p>
<h2>Equipment</h2>
<ul>
<li>32-inch monitor with HDMI input</li>
<li>Bluetooth speaker</li>
<li>Coffee and tea station</li>
<li>Building-wide wifi (see wifi-and-equipment-faq)</li>
</ul>
<h2>Booking &amp; Rate</h2>
<p>Base rate: $50.00 per hour (5000 cents). Pro members receive the standard
20% discount on every booking.</p>
<h2>House Rules</h2>
<p>Lounge is open-plan and does not require a keypad code, unlike Focus,
Phonebooth, and Boardroom. Because there is no door, members should keep
calls at conversational volume.</p>
</body>
</html>
""")

RAW_DOCS["phonebooth-room-manual"] = ("html", """<!DOCTYPE html>
<html>
<head><title>Phonebooth Room Manual</title></head>
<body>
<h1>Phonebooth</h1>
<p>Phonebooth is Reservo's smallest room, built for a single short call
rather than a working session. It is the only room designed to be used
standing up.</p>
<h2>Capacity &amp; Layout</h2>
<p>Capacity: 1 person. The room has a narrow shelf-desk and a single stool,
with just enough space to stand and pace during a call.</p>
<h2>Equipment</h2>
<ul>
<li>Wall-mounted phone charging dock</li>
<li>Small desk fan</li>
<li>Building-wide wifi (see wifi-and-equipment-faq)</li>
</ul>
<h2>Booking &amp; Rate</h2>
<p>Base rate: $15.00 per hour (1500 cents), the lowest rate in the building
alongside its small size. Pro members receive the standard 20% discount on
every booking.</p>
<h2>House Rules</h2>
<p>Phonebooth bookings are capped at 1 hour per reservation, since the room
is designed for short calls. Rooms with a physical door -- Focus,
Phonebooth, and Boardroom -- use a keypad code that rotates weekly.</p>
</body>
</html>
""")

# operations-manual-raw.txt: deliberately DIRTY (M1 practices parsing/cleaning
# on this one). Built so that clean_text(raw) reproduces M1's exact verbatim
# cleaned output (len(raw)=1473, len(clean)=1172, header repeated 4x, 5
# hyphen-breaks: week-\nends, ope-\nning, run-\nning, min-\nutes, re-\nset).
_HEADER = "RESERVO OPERATIONS MANUAL - INTERNAL - CONFIDENTIAL"


def _insert_break(text: str, word: str, nth: int = 1) -> str:
    """Split the nth occurrence of `word` at its midpoint into 'wo-\\nrd'."""
    start = 0
    count = 0
    idx = -1
    while True:
        idx = text.find(word, start)
        if idx == -1:
            raise ValueError(f"word {word!r} not found (occurrence {nth})")
        count += 1
        if count == nth:
            break
        start = idx + 1
    mid = len(word) // 2
    return text[:idx] + word[:mid] + "-\n" + word[mid:] + text[idx + len(word):]


_body1 = ("The building opens at 07:00 and closes at 22:00 on weekdays. On weekends the "
          "building opens at 09:00 and closes at 18:00. Staff must complete a "
          "walkthrough of every room before opening to confirm no equipment was left "
          "running overnight.")
_body1 = _insert_break(_body1, "weekends", 1)
_body1 = _insert_break(_body1, "opening", 1)
_body1 = _insert_break(_body1, "running", 1)

_body2 = ("Rooms are cleaned between every booking when the gap is 30 minutes or longer. "
          "For back-to-back bookings under 30 minutes, cleaning is limited to wiping the "
          "table and checking for left-behind belongings. Boardroom receives a full "
          "clean every evening regardless of usage, because of its size.")
_body2 = _insert_break(_body2, "minutes", 1)  # only the FIRST "minutes" breaks

_body3 = ("Rooms with a physical door - Focus, Phonebooth, and Boardroom - use a keypad "
          "code that rotates weekly. Studio and Lounge are open-plan and do not require "
          "a code. Staff must update the keypad codes every Monday before 07:00 and log "
          "the change in the access log.")
# page 3 has no hyphen-break in the source

_body4 = ("If a member reports the wifi is down, staff should first check the router in "
          "the utility closet before escalating. A full reset takes approximately 3 "
          "minutes and drops every room's connection at once, so it should only be done "
          "between bookings, never during an active reservation.")
_body4 = _insert_break(_body4, "reset", 1)

RAW_DOCS["operations-manual-raw"] = ("txt", (
    _HEADER + "\n" + "Page 1 of 4" + "\n" + ("\n" * 3) +
    "Opening and Closing Procedures" + "\n\n" + _body1 + ("\n" * 10) +
    _HEADER + "\n" + "Page 2 of 4" + "\n" + ("\n" * 3) +
    "Cleaning Procedures" + "\n\n" + _body2 + ("\n" * 10) +
    _HEADER + "\n" + "Page 3 of 4" + "\n" + ("\n" * 3) +
    "Key and Access Handling" + "\n\n" + _body3 + ("\n" * 9) +
    _HEADER + "\n" + "Page 4 of 4" + "\n" + ("\n" * 3) +
    "Wifi Reset Procedure" + "\n\n" + _body4
))


@dataclass(frozen=True)
class Chunk:
    chunk_id: str
    doc_id: str
    title: str
    section: str
    position: int
    text: str


# --- parse HTML (M1 L03) ----------------------------------------------------
WS_RE = re.compile(r"\s+")


class RoomManualParser(HTMLParser):
    """Extracts (title, [(section, text), ...]) from a room manual."""

    CAPTURE_TAGS = {"p", "li", "h1", "h2", "title"}

    def __init__(self):
        super().__init__()
        self.title = ""
        self.sections: list[tuple[str, list[str]]] = []
        self._current_section = "Overview"
        self._section_index: dict[str, int] = {}
        self._tag_stack: list[str] = []
        self._buffer: list[str] = []

    def handle_starttag(self, tag, attrs):
        self._tag_stack.append(tag)
        if tag in self.CAPTURE_TAGS:
            self._buffer = []

    def handle_endtag(self, tag):
        if self._tag_stack and self._tag_stack[-1] == tag:
            self._tag_stack.pop()
        text = "".join(self._buffer).strip()
        if tag == "title":
            self.title = text
        elif tag == "h2":
            self._current_section = text
        elif tag == "p" and text:
            self._add(text)
        elif tag == "li" and text:
            self._add(f"- {text}")

    def handle_data(self, data):
        if self._tag_stack and self._tag_stack[-1] in self.CAPTURE_TAGS:
            self._buffer.append(data)

    def _add(self, text):
        if self._current_section not in self._section_index:
            self._section_index[self._current_section] = len(self.sections)
            self.sections.append((self._current_section, []))
        idx = self._section_index[self._current_section]
        self.sections[idx][1].append(text)


def parse_html(raw_html: str) -> tuple[str, list[tuple[str, str]]]:
    parser = RoomManualParser()
    parser.feed(raw_html)
    sections = []
    for heading, paras in parser.sections:
        clean_paras = [WS_RE.sub(" ", p).strip() for p in paras]
        sep = "; " if all(p.startswith("- ") for p in clean_paras) else " "
        sections.append((heading, sep.join(clean_paras)))
    return parser.title, sections


# --- parse markdown (M1 L04) -------------------------------------------------
H2_SPLIT_RE = re.compile(r"(?m)^##\s+(.+)$")


def parse_markdown(raw_md: str) -> tuple[str, list[tuple[str, str]]]:
    lines = raw_md.strip("\n").split("\n")
    title = ""
    if lines and lines[0].startswith("# "):
        title = lines[0][2:].strip()
        lines = lines[1:]
    body = "\n".join(lines)

    parts = H2_SPLIT_RE.split(body)
    sections: list[tuple[str, str]] = []

    preamble = parts[0].strip()
    if preamble:
        sections.append(("Overview", preamble))

    for i in range(1, len(parts), 2):
        heading = parts[i].strip()
        content = parts[i + 1].strip() if i + 1 < len(parts) else ""
        content = re.sub(r"\s+", " ", content)
        sections.append((heading, content))

    return title, sections


# --- clean dirty .txt (M1 L04) -----------------------------------------------
HEADER_FOOTER_RE = re.compile(r"^RESERVO OPERATIONS MANUAL.*$\n?", re.MULTILINE)
PAGE_MARKER_RE = re.compile(r"^Page \d+ of \d+\s*$\n?", re.MULTILINE)
HYPHEN_BREAK_RE = re.compile(r"(\w)-\n(\w)")
BLANK_RUN_RE = re.compile(r"\n{2,}")
INNER_WS_RE = re.compile(r"[ \t]+")


def clean_text(raw: str) -> str:
    """Undoes the damage of a naive PDF/OCR-style text extraction."""
    text = HEADER_FOOTER_RE.sub("", raw)
    text = PAGE_MARKER_RE.sub("", text)
    text = HYPHEN_BREAK_RE.sub(r"\1\2", text)
    text = BLANK_RUN_RE.sub("\n\n", text)
    paragraphs = [p.replace("\n", " ").strip()
                  for p in text.split("\n\n") if p.strip()]
    paragraphs = [INNER_WS_RE.sub(" ", p) for p in paragraphs]
    return "\n\n".join(paragraphs)


def sectionize_cleaned(cleaned: str) -> list[tuple[str, str]]:
    """Cleaned text alternates short heading and body, one pair per old 'page'."""
    paragraphs = cleaned.split("\n\n")
    sections = []
    i = 0
    while i < len(paragraphs):
        heading = paragraphs[i]
        body = paragraphs[i + 1] if i + 1 < len(paragraphs) else ""
        sections.append((heading, body))
        i += 2
    return sections


# --- chunking strategies (M1 L05) --------------------------------------------
SENTENCE_RE = re.compile(r"(?<=[.!?])\s+(?=[A-Z(`])")


def split_sentences(text: str) -> list[str]:
    return [s.strip() for s in SENTENCE_RE.split(text) if s.strip()]


def chunk_by_sentence(text: str, max_size: int = 400) -> list[str]:
    """Packs complete sentences into chunks of up to max_size characters."""
    sentences = split_sentences(text)
    chunks: list[str] = []
    current: list[str] = []
    current_len = 0
    for sent in sentences:
        extra = len(sent) + (1 if current else 0)
        if current and current_len + extra > max_size:
            chunks.append(" ".join(current))
            current, current_len = [], 0
        current.append(sent)
        current_len += len(sent) + (1 if len(current) > 1 else 0)
    if current:
        chunks.append(" ".join(current))
    return chunks


def chunk_by_structure(
    sections: list[tuple[str, str]], max_size: int = 400
) -> list[tuple[str, str]]:
    """Respects section boundaries; only sub-splits a section past max_size."""
    out: list[tuple[str, str]] = []
    for heading, body in sections:
        if len(body) <= max_size:
            out.append((heading, body))
        else:
            for piece in chunk_by_sentence(body, max_size=max_size):
                out.append((heading, piece))
    return out


# --- metadata + ingestion (M1 L07/L08) ---------------------------------------
def ingest_document(doc_id: str, fmt: str, raw_text: str, max_size: int = 400) -> list[Chunk]:
    """Parses + (cleans) + chunks one document, attaching metadata."""
    if fmt == "md":
        title, sections = parse_markdown(raw_text)
    elif fmt == "html":
        title, sections = parse_html(raw_text)
    elif fmt == "txt":
        cleaned = clean_text(raw_text)
        title = "Operations Manual (raw, cleaned)"
        sections = sectionize_cleaned(cleaned)
    else:
        raise ValueError(f"unknown format: {fmt}")

    section_chunks = chunk_by_structure(sections, max_size=max_size)
    chunks = []
    for position, (section, text) in enumerate(section_chunks):
        chunk_id = f"{doc_id}-{position:03d}"
        chunks.append(Chunk(chunk_id, doc_id, title, section, position, text))
    return chunks


def build_corpus(max_size: int = 400) -> list[Chunk]:
    """Ingests all 13 RAW_DOCS into the single canonical list[Chunk] -- 57 chunks."""
    all_chunks: list[Chunk] = []
    for doc_id, (fmt, raw_text) in RAW_DOCS.items():
        all_chunks.extend(ingest_document(doc_id, fmt, raw_text, max_size=max_size))
    return all_chunks

El bloque de arriba es el corpus canónico completo: los 13 documentos de RAW_DOCS sin abreviar ninguno, más el chunker entero — idéntico, carácter a carácter, al que fija el DISEÑO.md de esta guía en su sección "🛑 CORPUS CANÓNICO — BLOQUE EXACTO". No hace falta ir a buscarlo a otro módulo: copiándolo tal cual queda en tu directorio de trabajo, listo para el resto del proyecto.

Ejemplo trabajado: reconstruir los 57 chunks

Con reservo_corpus.py guardado completo (los 13 documentos, sin abreviar), confirma que produce exactamente el corpus canónico:

from reservo_corpus import build_corpus, RAW_DOCS

chunks = build_corpus()
assert len(chunks) == 57, f"esperado 57, obtuve {len(chunks)}"

print("documentos:", len(RAW_DOCS), " chunks totales:", len(chunks))
print()

by_doc = {}
for c in chunks:
    by_doc.setdefault(c.doc_id, []).append(c)

for doc_id, (fmt, _) in RAW_DOCS.items():
    doc_chunks = by_doc[doc_id]
    print(f"  {doc_id:26s} .{fmt:4s} -> {len(doc_chunks)} chunks")

sizes = [len(c.text) for c in chunks]
print()
print(f"tamaño de chunk -> min={min(sizes)} max={max(sizes)} avg={sum(sizes)/len(sizes):.1f}")

Qué esperar (ejecutado):

documentos: 13  chunks totales: 57

  cancellation-policy        .md  -> 4 chunks
  no-show-policy              .md  -> 4 chunks
  refund-policy                .md  -> 4 chunks
  booking-faq                   .md  -> 4 chunks
  membership-tiers-faq           .md  -> 4 chunks
  payment-methods-faq             .md  -> 4 chunks
  wifi-and-equipment-faq           .md  -> 4 chunks
  focus-room-manual                 .html -> 5 chunks
  studio-room-manual                .html -> 5 chunks
  boardroom-room-manual              .html -> 5 chunks
  lounge-room-manual                  .html -> 5 chunks
  phonebooth-room-manual               .html -> 5 chunks
  operations-manual-raw                 .txt -> 4 chunks

tamaño de chunk -> min=103 max=290 avg=174.4

Estos números —57 chunks, 7 documentos markdown con 4 chunks cada uno, 5 manuales HTML con 5 chunks cada uno, y operations-manual-raw (ya limpio) con 4— son idénticos a los que el Módulo 1 reportó por primera vez y a los que el DISEÑO.md de esta guía pineó como fuente única de verdad. 7×4 + 5×5 + 1×4 = 28 + 25 + 4 = 57. Ningún módulo posterior a M1 —ni el índice del Módulo 2, ni la tool del Módulo 3, ni este capstone— regenera un corpus distinto: todos importan o reproducen este mismo bloque.

Verificación adicional: el chunk_id es 0-indexado, con relleno de ceros

sample_ids = [c.chunk_id for c in chunks if c.doc_id == "no-show-policy"]
print("chunk_ids de no-show-policy:", sample_ids)

no_show_001 = next(c for c in chunks if c.chunk_id == "no-show-policy-001")
print("no-show-policy-001 ->", no_show_001.section, "| position =", no_show_001.position)

Qué esperar:

chunk_ids de no-show-policy: ['no-show-policy-000', 'no-show-policy-001', 'no-show-policy-002', 'no-show-policy-003']
no-show-policy-001 -> What Happens on a No-Show | position = 1

Confirma la convención fijada desde el Módulo 1 y verificada por ejecución en el DISEÑO.md: chunk_id = f"{doc_id}-{position:03d}", 0-indexado — no-show-policy-000 es el primer chunk ("What Counts as a No-Show"), no-show-policy-001 es el segundo ("What Happens on a No-Show", position=1), no el primero. Cualquier pieza de este proyecto que asuma 1-indexado —o que le falte el relleno de ceros ("no-show-policy-1" en vez de "no-show-policy-001")— no es compatible con el resto del pipeline.

Errores comunes

1. Abreviar RAW_DOCS "para ahorrar espacio" y quedarte con menos de 13 documentos. El assert len(chunks) == 57 de esta lección existe exactamente para atrapar este error temprano — si copiaste 10 documentos en vez de 13, el assert falla ruidosamente aquí, no silenciosamente tres lecciones después con un score de evaluación que no coincide con nada.

2. Cambiar una palabra del texto "porque no se ve importante". Este corpus alimenta un índice léxico — un solo carácter de diferencia en un documento puede cambiar qué chunk gana una query cerrada más adelante en el proyecto (Lecciones 03, 04 y 06). Copia el bloque RAW_DOCS exacto, no una paráfrasis.

3. Confundir max_size con un límite duro de caracteres por chunk. chunk_by_structure respeta los límites de sección primero — solo sub-divide con chunk_by_sentence cuando una sección completa supera max_size=400. En este corpus, ninguna sección lo supera, así que cada (sección, texto) se vuelve exactamente un Chunk — por eso el conteo final (57) coincide exactamente con la suma de secciones de los 13 documentos.

4. Reordenar RAW_DOCS y esperar el mismo resultado. El orden de inserción del diccionario determina el orden de build_corpus(), que a su vez determina el orden de chunk_ids que ve el índice del Módulo 2. No cambia el conteo final ni los scores de BM25 (que no dependen del orden), pero sí puede cambiar el orden en el que aparecen resultados con score empatado — usa el mismo orden de inserción del bloque original para reproducibilidad exacta.

Ejercicios

Ejercicio 1: Confirma el desglose de un formato (Fácil)

Sin ejecutar nada nuevo: según la tabla de "Qué esperar" de esta lección, ¿cuántos chunks totales aportan los 5 manuales HTML? ¿Y los 7 documentos markdown? Verifica tu respuesta sumando len(by_doc[doc_id]) para cada grupo.

html_docs = [d for d, (fmt, _) in RAW_DOCS.items() if fmt == "html"]
md_docs = [d for d, (fmt, _) in RAW_DOCS.items() if fmt == "md"]

html_total = sum(len(by_doc[d]) for d in html_docs)
md_total = sum(len(by_doc[d]) for d in md_docs)

print(f"HTML: {len(html_docs)} documentos -> {html_total} chunks")
print(f"Markdown: {len(md_docs)} documentos -> {md_total} chunks")

HTML: 5 documentos -> 25 chunks
Markdown: 7 documentos -> 28 chunks

Ejercicio 2: Verifica clean_text sobre el documento sucio (Medio)

Confirma, en tu propio directorio, que clean_text(RAW_DOCS["operations-manual-raw"][1]) produce exactamente len(raw)=1473 de entrada y len(clean)=1172 de salida — los dos números que el Módulo 1 verificó por ejecución y que el DISEÑO.md volvió a pinear.

from reservo_corpus import clean_text

raw = RAW_DOCS["operations-manual-raw"][1]
cleaned = clean_text(raw)

print("len(raw):", len(raw))
print("len(clean):", len(cleaned))
print("header count en raw:", raw.count("RESERVO OPERATIONS MANUAL"))
print("header count en clean:", cleaned.count("RESERVO OPERATIONS MANUAL"))

len(raw): 1473
len(clean): 1172
header count en raw: 4
header count en clean: 0

Ejercicio 3: Mide cuántos chunks tiene cada documento antes de ejecutar nada (Difícil)

Sin ejecutar código: para cada uno de los 13 documentos, predice cuántos chunks va a producir, basándote solo en el número de encabezados ## (para markdown) o <h2> (para HTML) que tiene, y en que ninguna sección de este corpus supera max_size=400. Después, ejecuta y compara tu predicción con el resultado real.

predicted = {}
for doc_id, (fmt, raw) in RAW_DOCS.items():
    if fmt == "md":
        predicted[doc_id] = raw.count("\n## ")
    elif fmt == "html":
        predicted[doc_id] = raw.count("<h2>") + 1  # +1 por el Overview implicito
    else:
        predicted[doc_id] = 4  # operations-manual-raw: 4 "paginas" fijas

for doc_id in RAW_DOCS:
    actual = len(by_doc[doc_id])
    match = "OK" if predicted[doc_id] == actual else "MISMATCH"
    print(f"  [{match}] {doc_id:26s} predicho={predicted[doc_id]}  real={actual}")

  [OK] cancellation-policy        predicho=4  real=4
  [OK] no-show-policy              predicho=4  real=4
  [OK] refund-policy                predicho=4  real=4
  [OK] booking-faq                   predicho=4  real=4
  [OK] membership-tiers-faq           predicho=4  real=4
  [OK] payment-methods-faq             predicho=4  real=4
  [OK] wifi-and-equipment-faq           predicho=4  real=4
  [OK] focus-room-manual                 predicho=5  real=5
  [OK] studio-room-manual                predicho=5  real=5
  [OK] boardroom-room-manual              predicho=5  real=5
  [OK] lounge-room-manual                  predicho=5  real=5
  [OK] phonebooth-room-manual               predicho=5  real=5
  [OK] operations-manual-raw                predicho=4  real=4

Resumen y siguiente paso

• reservo_corpus.py quedó armado en tu directorio de trabajo, verbatim del Módulo 1: 13 documentos crudos, el chunker completo, y build_corpus() produciendo exactamente 57 chunks.
• Confirmaste, ejecutado, el desglose por formato (7 markdown × 4, 5 HTML × 5, 1 txt limpio × 4) y la convención de chunk_id 0-indexado con relleno de ceros.
• Este archivo es el cimiento del resto del proyecto — cada lección siguiente importa Chunk y build_corpus desde aquí, sin volver a tocar una línea del chunker.

Siguiente lección: 03 — El índice y la tool. Construyes el índice BM25 sobre estos 57 chunks (k1=1.5, b=0.75) y lo envuelves en search_docs, la tool que el resto del proyecto invoca.

Recursos adicionales

1. production-rag-and-document-ingestion-guide — Módulo 1, Lección 08 (mini-proyecto): el bloque RAW_DOCS completo, con los 13 documentos sin abreviar, para copiar en tu propio directorio.
2. production-rag-and-document-ingestion-guide — DISEÑO.md, sección "🛑 CORPUS CANÓNICO — BLOQUE EXACTO": la fuente única de verdad que fija el conteo (57), el esquema de chunk_id, y los seis campos de Chunk.
3. Python — html.parser y Python — re — la base stdlib completa de parse_html/parse_markdown/clean_text.
4. Python — dataclasses — el @dataclass(frozen=True) detrás de Chunk, inmutable por diseño.