Commit graph

11 commits

Author SHA1 Message Date
Yiorgis Gozadinos
ee5b7b7313
Split merged context-expansion groups that cannot afford every constituent 2026-07-14 10:15:28 +03:00
Yiorgis Gozadinos
c424f51056
Anchor and clip merged citations on the highest-scoring chunk
A merged search result took its chunk_id from whichever constituent
sorted earliest in the document, while its score was the max across the
group — so the citation's identity could point at a different, less
relevant chunk. Anchor chunk_id and the content/refs fallbacks on the
max-score constituent, clip the budget window around that same chunk so
its evidence is never trimmed away, and narrow page_numbers, doc_item_refs,
and attached image bytes to the items that survive the clip.
2026-07-09 11:05:41 +03:00
Yiorgis Gozadinos
0bcf34363a
Carry merged chunk ids on SearchResult and Citation 2026-07-09 11:05:41 +03:00
Yiorgis Gozadinos
b14152a45b
Keep picture and table hits section-bounded during context expansion 2026-07-09 11:05:40 +03:00
Yiorgis Gozadinos
b94e13083f
Hard-cap context expansion at max_context_chars
A single oversized document_items row (e.g. a spreadsheet converted to one
table) expanded far past search.max_context_chars and could overflow the
model context window. _expand_outward only used the budget as a soft
accumulation threshold and expand_with_items never capped the joined result.

Add _clip_to_budget to clip each expanded result to max_context_chars,
returning a window centered on the matched chunk (via _evidence_anchors) so
the retrieved evidence survives the cut.
2026-06-27 09:46:36 +03:00
Yiorgis Gozadinos
e3314e36fd
search: only attach picture bytes from pre-expansion chunks 2026-05-19 15:03:22 +03:00
Yiorgis Gozadinos
3224d1f20c
Extract document orchestration into client/documents.py 2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
ff8ad0879c
fix context expansion: respect section boundaries, remove max_context_items 2026-04-20 15:18:16 +03:00
Yiorgis Gozadinos
d3c8ed62a8
preserve original chunk when expansion produces less content 2026-04-16 12:11:54 +03:00
Yiorgis Gozadinos
39b9d0f683
Additional tests 2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
b6113bf8ab
replace fixed-radius expansion with section-bounded algorithm
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
2026-04-16 12:11:53 +03:00