haiku.rag/tests/store
Yiorgis Gozadinos af6a6b0bbe
Batch search enrichment across documents
`_populate_image_data` ran its stages once per result document, so a result set
spanning N documents cost 4N `document_items` queries. Measured on a remote
object-store corpus, a limit=5 search with expansion was 18 queries, 16 of them
against `document_items`.

The stages now run once each across every document, and flat in document count:
two queries for the dependent caption-to-picture mapping when results ranked on a
caption, one for the picture bytes. Two queries for a picture-ref result set,
three at most.

Picture text comes back with the bytes rather than from a second query, since it
is on the same rows.

Predicates are per document, `(document_id = 'a' AND self_ref IN (…)) OR (…)`,
rather than `self_ref IN (union)`. self_ref and position values repeat across
documents, so a union predicate would return other documents' rows: for
picture_data that fetches blobs nobody asked for, and it can hand one document
another document's picture.
2026-08-18 16:23:16 +03:00
..
__init__.py Split meta document attributes into a document_meta table 2026-06-12 10:17:17 +03:00
legacy_documents.py Serialize and roll back the multi-table document delete 2026-06-12 10:17:18 +03:00
test_citation.py Expose all chunk metadata on search results and citations through SearchResult.chunk_meta and Citation.chunk_meta 2026-08-17 19:42:23 +02:00
test_compression.py Collapse docling compression to a single function 2026-06-24 10:43:27 +03:00
test_database_info.py Extract gather_database_info shared by the info command 2026-06-09 11:09:28 +03:00
test_document_item_grouped.py Batch search enrichment across documents 2026-08-18 16:23:16 +03:00
test_document_items.py Reach and enforce 100% coverage 2026-07-26 20:11:02 +03:00
test_document_meta_split.py Rename document_meta identity column document_id to id 2026-07-08 16:32:55 +03:00
test_engine.py Similar fix for haiku.rag.ingester, and formatting 2026-08-12 15:20:11 +01:00
test_indexes.py Trim comments, docstrings and docs to what they need to say 2026-08-18 12:38:02 +03:00
test_migrations.py Stop eagerly importing Store from haiku.rag.store 2026-08-11 17:26:51 +01:00
test_open_path.py Read the table list and settings once per open 2026-08-18 14:58:17 +03:00
test_read_only.py Stop eagerly importing Store from haiku.rag.store 2026-08-11 17:26:51 +01:00
test_restore.py Stop eagerly importing Store from haiku.rag.store 2026-08-11 17:26:51 +01:00
test_tags.py Stop eagerly importing Store from haiku.rag.store 2026-08-11 17:26:51 +01:00
test_v0_40_0_migration.py Similar fix for haiku.rag.ingester, and formatting 2026-08-12 15:20:11 +01:00
test_v0_45_0_migration.py Stop eagerly importing Store from haiku.rag.store 2026-08-11 17:26:51 +01:00
test_v0_48_0_migration.py Collapse docling compression to a single function 2026-06-24 10:43:27 +03:00
test_v0_50_0_migration.py Reach and enforce 100% coverage 2026-07-26 20:11:02 +03:00
test_v0_58_0_migration.py Load document blobs only when asked 2026-08-06 13:04:51 +03:00
test_v0_64_0_migration.py Rename document_meta identity column document_id to id 2026-07-08 16:32:55 +03:00
test_v0_75_0_migration.py Trim comments, docstrings and docs to what they need to say 2026-08-18 12:38:02 +03:00