Updating only what changed

Description

Previous lessons tested idempotency against a corpus that never actually changed — Lesson 05's central piece of evidence was that reingesting the same RAW_DOCS twice duplicates nothing. This lesson tests the other case, just as important: a document that genuinely changes. refund-policy gains a new sentence in its refund-timing section, exactly the kind of edit that happens all the time in production — someone clarifies a policy, fixes a detail, adds an exception.

The question this lesson answers with executed evidence: when a single document out of 13 changes, how much work does reingest() do? The answer isn't "reprocess the whole corpus", or even "reprocess all of refund-policy" — it's more precise than that: it reparses only the document that changed, and within that document, it rewrites only the chunk whose content actually changed.

Connection to the module

This lesson adds no new function — it uses reingest(), detect_changes, and upsert_chunk exactly as they stood at the end of Lessons 03-05, and measures with real instrumentation how much work they avoid on a modified document. Lesson 07 does the same exercise with a deleted document.

Analogy: one page replaced, not the whole folder

Today the clerk from earlier lessons gets a specific update: the "Refund Policy" folder has a page with a new sentence, added to the end of its "Refund Amount and Timing" section. The other three pages in that folder — "What Qualifies for a Refund", "What Does Not Qualify for a Refund", "How to Request a Refund" — haven't changed by a single comma. And the other 12 folders in the archive haven't changed either.

A clerk with no judgment would redo the entire "Refund Policy" folder — or worse, all 13 folders — every time they detect any change anywhere. This module's clerk, with their cover label and their per-page fingerprint notebook, does something more surgical: "Refund Policy"'s cover label doesn't match what's on record, so they open that folder — no other — scan its 4 pages, and discover that 3 of them have exactly the same fingerprint as always. They only replace the page that changed. The rest of the archive — 53 pages from the other 12 documents, plus 3 of the 4 pages of "Refund Policy" — stays exactly as it was, with no one touching it again.

The document that changes: refund-policy gains a sentence

RAW_DOCS_V2 is a copy of RAW_DOCS with a single change: refund-policy's "Refund Amount and Timing" section gains a sentence clarifying what happens when the refund is for a facility issue rather than a member's cancellation. The rest of the 13 documents — including refund-policy's other three sections — stays identical, character for character.

from reservo_corpus import RAW_DOCS

RAW_DOCS_V2 = dict(RAW_DOCS)
fmt, raw = RAW_DOCS_V2["refund-policy"]

old_section = (
    "## Refund Amount and Timing\n\n"
    "Eligible refunds return the full amount charged for the booking to the "
    "original payment method. Processing takes up to 5 business days once "
    "the cancellation is confirmed in the booking system."
)
new_section = old_section + (
    " For refunds tied to a facility issue rather than a cancellation, "
    "processing begins from the date Reservo confirms the outage, not the "
    "date of the original booking."
)
assert old_section in raw  # confirms the anchor text is exactly what's there
RAW_DOCS_V2["refund-policy"] = (fmt, raw.replace(old_section, new_section))

print("len(original raw):", len(raw))
print("len(modified raw):", len(RAW_DOCS_V2["refund-policy"][1]))
print("the other 12 documents are identical:",
      all(RAW_DOCS_V2[d] == RAW_DOCS[d] for d in RAW_DOCS if d != "refund-policy"))

What to expect (run):

len(original raw): 1014
len(modified raw): 1178
the other 12 documents are identical: True

RAW_DOCS (the canonical corpus) isn't touched — RAW_DOCS_V2 is a new dictionary, copied with a single value replaced. This simulates exactly what would happen in a real system: today's run finds the source directory has 13 files, where 12 are bit-for-bit the same as yesterday and one changed.

Reingesting RAW_DOCS_V2: only one document, only one chunk

Starting from a store that already has the original corpus ingested (as at the end of Lesson 05), reingest RAW_DOCS_V2:

conn = create_store(":memory:")
reingest(conn, RAW_DOCS)  # run 1: the original corpus, 57 chunks

r2 = reingest(conn, RAW_DOCS_V2)  # run 2: refund-policy changed
total = conn.execute("SELECT COUNT(*) FROM chunks").fetchone()[0]

print("modified:", r2["modified"])
print("unchanged_count:", r2["unchanged_count"])
print("chunks_written:", r2["chunks_written"])
print("chunks_purged:", r2["chunks_purged"])
print("total chunks in the store:", total)

What to expect (run):

modified: ['refund-policy']
unchanged_count: 12
chunks_written: 1
chunks_purged: 0
total chunks in the store: 57

detect_changes classified refund-policy as the one and only modified document, and the other 12 as unchanged — exactly what building RAW_DOCS_V2 this way guarantees. chunks_written: 1 is the number that matters: of refund-policy's 4 chunks, reingest() rewrote exactly one. The store's total stays at 57 — it didn't go up or down — because nothing was added or deleted, only an existing chunk was updated in place.

Looking at refund-policy's 4 chunks one by one, with their content_hash before and after:

from reservo_corpus import ingest_document

old_chunks = ingest_document("refund-policy", "md", raw)
new_chunks = ingest_document("refund-policy", "md", RAW_DOCS_V2["refund-policy"][1])

for oc, nc in zip(old_chunks, new_chunks):
    changed = content_hash(oc) != content_hash(nc)
    print(f"{oc.chunk_id}: len_old={len(oc.text)} len_new={len(nc.text)} changed={changed}")

What to expect (run):

refund-policy-000: len_old=225 len_new=225 changed=False
refund-policy-001: len_old=192 len_new=356 changed=True
refund-policy-002: len_old=226 len_new=226 changed=False
refund-policy-003: len_old=219 len_new=219 changed=False

Only refund-policy-001 — the "Refund Amount and Timing" section, exactly where the new sentence was added — changed length and content_hash. The other three chunks of the same document (-000, -002, -003) have exactly the same length and the same hash as before: upsert_chunk, called the same way for all 4, wrote only the one that genuinely changed — the same behavior you already saw in Lesson 03, now confirmed within a document that had a real edit.

How much work was avoided: instrumenting ingest_document

The chunks_written: 1 number already shows that the store wrote very little. To confirm that the pipeline also didn't reparse the documents that didn't change — not just that it didn't rewrite them — you can wrap ingest_document with a counter before running the reingest:

parsed_docs = []
_original_ingest_document = ingest_document


def counting_ingest_document(doc_id, fmt, raw_text, max_size=400):
    parsed_docs.append(doc_id)
    return _original_ingest_document(doc_id, fmt, raw_text, max_size=max_size)


# Replaces the global name `ingest_document` -- upsert_doc_chunks (Lesson 05)
# looks it up by name at call time, so from here on, any call it makes goes
# through the counter first.
ingest_document = counting_ingest_document

conn2 = create_store(":memory:")
reingest(conn2, RAW_DOCS)
parsed_docs.clear()  # only run 2's count matters

reingest(conn2, RAW_DOCS_V2)
print("documents reparsed in run 2:", parsed_docs)
print(f"total: {len(parsed_docs)} of the {len(RAW_DOCS_V2)} documents in the corpus")

What to expect (run):

documents reparsed in run 2: ['refund-policy']
total: 1 of the 13 documents in the corpus

Of the corpus's 13 documents, ingest_document was called exactly once, and only for refund-policy. The other 12 — cancellation-policy, no-show-policy, the 5 room manuals, and the rest — never went through parse_markdown, parse_html, or clean_text in this run: detect_changes ruled them out up front, comparing only their doc_hash, with no need to open them. This is direct proof of Lesson 04's central claim: separating detection (cheap) from processing (expensive) means the cost of a reingest() run scales with how much changed, not with the total size of the corpus.

Common mistakes

1. Measuring "how much this optimized" only with chunks_written, without looking at ingest_document. chunks_written: 1 confirms the store wrote very little, but it doesn't, by itself, confirm the pipeline avoided parsing the unchanged documents — for that you need the instrumentation above, or trust that detect_changes (Lesson 04) never calls ingest_document, which is exactly what its design guarantees.
2. Thinking "a modified document" always rewrites all of its chunks. If the edit falls within a single section (as in this example), only that section's chunk changes hash. If the edit reordered whole sections or changed the chunking max_size, more chunks could be affected — but that's not typical of a real content edit, and upsert_chunk handles either case correctly with no code change.
3. Forgetting parsed_docs.clear() before measuring the run that matters. If the list isn't cleared after run 1 (the initial ingestion, which does reparse all 13 documents because they're all new), run 2's count gets contaminated with the 13 documents from the previous run, and the result stops showing the real savings.
4. Assuming monkey-patch instrumentation is needed in production. The trick of replacing ingest_document with a call-counting version is a tool for this lesson, to demonstrate and verify behavior — not something a real pipeline needs in its production code. In a real system, this same information would come from normal logging (logger.info(f"parsing {doc_id}")) or an exported metric, not a runtime patch.

Exercises

Exercise 1: Modify a different document (Easy)

Build RAW_DOCS_V2B, a copy of RAW_DOCS where wifi-and-equipment-faq gains a fifth question at the end ("## Does the Wifi Password Ever Change Without Notice?" with any short answer). Reingest against a store that already has the original corpus and confirm which document shows up as modified and how many new chunks it writes.

RAW_DOCS_V2B = dict(RAW_DOCS)
fmt, raw = RAW_DOCS_V2B["wifi-and-equipment-faq"]
new_raw = raw.rstrip("\n") + (
    "\n\n## Does the Wifi Password Ever Change Without Notice?\n\n"
    "No. Password changes are always posted on the room card and the "
    "booking confirmation screen at least one day in advance."
)
RAW_DOCS_V2B["wifi-and-equipment-faq"] = (fmt, new_raw)

conn3 = create_store(":memory:")
reingest(conn3, RAW_DOCS)
r = reingest(conn3, RAW_DOCS_V2B)
total = conn3.execute("SELECT COUNT(*) FROM chunks").fetchone()[0]
print("modified:", r["modified"])
print("chunks_written:", r["chunks_written"])
print("total chunks:", total)

modified: ['wifi-and-equipment-faq']
chunks_written: 1
total chunks: 58

Exercise 2: Confirm the rest of the corpus lost nothing (Medium)

After Exercise 1, write a query that confirms the 53 chunks from the 12 unmodified documents (everything except wifi-and-equipment-faq) are still exactly the same chunk_id values build_corpus() produces for those documents — not one extra, not one missing, not one with different content.

from reservo_corpus import build_corpus

expected_other_ids = {c.chunk_id for c in build_corpus() if c.doc_id != "wifi-and-equipment-faq"}
stored_other_ids = {
    row[0] for row in conn3.execute(
        "SELECT chunk_id FROM chunks WHERE doc_id != 'wifi-and-equipment-faq'"
    ).fetchall()
}
print("matches exactly:", expected_other_ids == stored_other_ids)
print("count:", len(stored_other_ids))

matches exactly: True
count: 53

Exercise 3: What happens if the edit crosses the max_size boundary? (Hard)

ingest_document uses max_size=400 by default (Module 1). Build a much more aggressive edit to refund-policy than the worked example: replace the entire "Refund Amount and Timing" section with a text over 400 characters long (for example, repeat the worked example's clarifying sentence three times). How many chunks does that section have after reingesting? Is it still only one chunk that changes?

RAW_DOCS_V2C = dict(RAW_DOCS)
fmt, raw = RAW_DOCS_V2C["refund-policy"]
old_section = (
    "## Refund Amount and Timing\n\n"
    "Eligible refunds return the full amount charged for the booking to the "
    "original payment method. Processing takes up to 5 business days once "
    "the cancellation is confirmed in the booking system."
)
extra = (
    " For refunds tied to a facility issue rather than a cancellation, "
    "processing begins from the date Reservo confirms the outage, not the "
    "date of the original booking."
)
new_section = old_section + extra * 3  # deliberately long, past max_size=400
RAW_DOCS_V2C["refund-policy"] = (fmt, raw.replace(old_section, new_section))

section_chunks = ingest_document("refund-policy", "md", RAW_DOCS_V2C["refund-policy"][1])
for c in section_chunks:
    print(c.chunk_id, c.section, len(c.text))

refund-policy-000 What Qualifies for a Refund 225
refund-policy-001 Refund Amount and Timing 356
refund-policy-002 Refund Amount and Timing 327
refund-policy-003 What Does Not Qualify for a Refund 226
refund-policy-004 How to Request a Refund 219

Summary and next step

• A modified document (refund-policy, with a new sentence in one section) makes detect_changes classify it as modified — and only it; the other 12 documents stay unchanged.
• reingest() reparses (ingest_document) only the modified document, confirmed with real instrumentation: 1 of 13 documents went through the parser in run 2.
• Within that document, upsert_chunk rewrites only the chunk whose content_hash changed (refund-policy-001, out of 4 total chunks) — the other 3 stay exactly as they were, same hash, same row.
• If the edit crosses a section's max_size boundary, chunking can shift the following sections' positions, and then more than one chunk_id gets rewritten — a real case worth recognizing, not a failure of the idempotency system.

Next lesson: 07 — Handling a deleted document. phonebooth-room-manual stops existing in the corpus (the room closes for renovation) — purge_doc removes its 5 chunks from the store, and a query confirms no orphaned chunk is left behind.

Additional resources

1. Python — replacing module attributes at runtime — the basis for the instrumentation technique (sys.modules[__name__].ingest_document = ...) used to count real calls.
2. production-rag-and-document-ingestion-guide — Module 1, Lesson 05 (05-chunking-strategies-fixed-vs-structure-aware.md) and Lesson 06 (06-the-chunk-size-tradeoff.md): chunk_by_structure and why max_size=400 is the right choice for the canonical corpus — the basis for Exercise 3 in this lesson.
3. Python — dict and equality comparison — the basis for the RAW_DOCS_V2[d] == RAW_DOCS[d] check used to confirm the other 12 documents stayed intact.