Sixty-three comments said what the next statement already said: # Connect to
LanceDB above connect_lancedb, # Path object above isinstance(source, Path),
# Get page numbers from provenance above the prov loop, # Clear and populate
results above list_view.clear(). They cost a read and carry nothing.
The line is whether a comment restates one statement or labels a phase. Phase
labels stay: the migrations keep # Create staging table with new schema and
# Copy from staging to final table in batches, each heading ten lines of a
long procedure. So do comments carrying a fact the code cannot: the
merge_insert update-only note on document_meta, why the poller builds sources
eagerly, why create_document_from_source returns a list for directories, that
indexes need training data, the field-group markers in the config models, and
the file:// URL-encoding note in create_document_from_source.
capabilities/ is untouched. Its docstrings sit next to prompt surface, and
changing them needs an eval to back it.
The cassette-recording docs were wrong three ways. They named
tests/test_qa.py::test_qa_anthropic, which no longer exists; they targeted
whole modules, so a rewrite would re-record cassettes for services the
recorder is not running; and they used COHERE_API_KEY where the SDK reads
CO_API_KEY. docs/development.md now names exact tests with -n0, and the keyed
example is test_cohere_reranker, which owns the one cassette recording
api.cohere.com.
`expand_with_items` fetched its own inputs per document: one query to resolve
refs to positions, one for the window of items around them. A result set spanning
N documents cost 2N queries, which was 10 of the 18 measured for a limit=5 search
on a remote object-store corpus.
`expand_context` now does both fetches once for every document it is expanding,
and `expand_with_items` takes the positions and items it needs. Two queries for
one document, and two for five.
Each document keeps its own inclusive window in `get_items_in_ranges`. Positions
repeat across documents, so a shared range would splice one document's items into
another's context.
Delete repository methods with no callers (SettingsRepository CRUD,
ChunkRepository.update/delete/get_chunks_in_range), the DataFrame branch of
_process_search_results whose only caller always passes a query, and guards
that cannot be reached from their call sites: the rebuild mode=None default,
the staging drop already performed by _resolve_rebuild_recovery, the empty
batch skip, two context fast paths, the doctor prefix guard, the poller
_task attribute that is never assigned, and a docling caption fallback for
a field name no item class defines.
set_haiku_version built a recreated settings row from the process-global
Config rather than the store's own, so a store opened with a custom config
stamped global settings into the database.
check_source_accessible called urlparse outside its try block, so a stored
URI with a malformed IPv6 host raised ValueError instead of reporting the
source as inaccessible, aborting the whole rebuild sweep.
A merged search result took its chunk_id from whichever constituent
sorted earliest in the document, while its score was the max across the
group — so the citation's identity could point at a different, less
relevant chunk. Anchor chunk_id and the content/refs fallbacks on the
max-score constituent, clip the budget window around that same chunk so
its evidence is never trimmed away, and narrow page_numbers, doc_item_refs,
and attached image bytes to the items that survive the clip.
A single oversized document_items row (e.g. a spreadsheet converted to one
table) expanded far past search.max_context_chars and could overflow the
model context window. _expand_outward only used the budget as a soft
accumulation threshold and expand_with_items never capped the joined result.
Add _clip_to_budget to clip each expanded result to max_context_chars,
returning a window centered on the matched chunk (via _evidence_anchors) so
the retrieved evidence survives the cut.
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
