haiku.rag/tests
Yiorgis Gozadinos 397b553528
Search several configured databases and fuse the results
`lancedb.databases` maps a name to a location, mutually exclusive with `uri`.
`search(sources=[…])` selects which to search, `sources=None` searches all of them
and `sources=[]` searches none; `SearchResult.source` carries the configured name,
so a path or URI never leaves the configuration. A database named in config keeps
its name even when it is the only one configured; only a legacy single `uri`
leaves `source` unset.

Databases open on first use, not at entry. Which are searched is a per-query
choice, so a set of 25 queried a few at a time opens a few, and a database nobody
asked for can neither fail a query nor be opened for nothing.

A named database that fails to open raises `SourceUnavailableError` naming it,
raised outside the handler so the original is not attached at all. A local failure
spells out the absolute path and an object-store failure can carry the bucket;
`from None` would only stop that being printed, leaving it on `__context__` for
anything that walks the chain. A legacy `uri` client has no name to report
instead, so its error passes through unchanged.

Candidates are fetched concurrently, then fused before anything is ranked. A
configured reranker scores the union, which is what makes ranking across
databases tractable: it compares query against document and does not care where a
candidate came from. Without one, reciprocal rank fusion over the per-database
rankings, since scores from separate indexes are not comparable. Enrichment then
runs on the survivors through the database each came from, concurrently, so it
costs what a single-database search costs.

The over-fetch decision and the reranker belong to the federating client alone.
Deciding per database would have each consult its own, and a local reranker loads
model weights per instance. It is built only for a text query, and closed once by
the client that owns it.

A location without a scheme is opened as a local path rather than through
`lancedb.uri`. Routing it through `uri` had `ConnectionMode` classify it as object
storage, which opens a missing database instead of reporting it.

With several databases configured, `store` and the repositories are left unset:
they have no unambiguous meaning across a set, and picking one silently would be
worse than the error.
2026-08-24 10:03:45 +03:00
..
capabilities Support Pydantic AI agent specs via from_spec 2026-08-21 13:04:09 +03:00
cassettes Put multi-table writes behind one transaction boundary 2026-08-19 14:05:40 +03:00
chat Give the capsule to one return, and the run its own copy of state 2026-08-13 13:00:02 +03:00
data
docker
ingester Move source adapters out of the ingester package 2026-08-20 11:46:55 +03:00
sandbox
sources Resolve file:// URIs to paths through url2pathname 2026-08-21 10:22:26 +03:00
store Split the store module by responsibility 2026-08-20 12:13:51 +03:00
tools Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
__init__.py
conftest.py
json_body_serializer.py Keep cross-encoder rerank scores apart when they saturate 2026-08-07 14:07:53 +03:00
services.py
test_app.py Measure the CLI and its application layer 2026-08-20 13:23:00 +03:00
test_ask_images.py
test_cascade_delete.py
test_chunk.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_chunker.py Reject unknown and out-of-range configuration values 2026-08-19 15:32:51 +03:00
test_cli.py Measure the CLI and its application layer 2026-08-20 13:23:00 +03:00
test_client.py Coerce a string db_path to Path at the Store and capability boundaries 2026-08-21 12:49:45 +03:00
test_client_analyze.py
test_config.py Make the documented configuration match the code 2026-08-19 15:52:50 +03:00
test_context.py Remove the single-document item accessors the batching replaced 2026-08-18 17:12:10 +03:00
test_context_enhancement.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_converters.py Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
test_database_autocreate.py
test_docling_serve_client.py Wire providers.docling_serve.timeout through to the client 2026-08-19 13:30:45 +03:00
test_doctor.py Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
test_document.py Load document blobs only when asked 2026-08-06 13:04:51 +03:00
test_download_models.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_duplicate_ingestion.py
test_embedder.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_embedder_config.py Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
test_enrichment_batching.py Stop the reranker fetch reading text it discards 2026-08-18 18:00:00 +03:00
test_filter.py
test_info.py Split the store module by responsibility 2026-08-20 12:13:51 +03:00
test_inspector.py Do not duplicate metadata items in inspector 2026-08-18 12:30:13 +02:00
test_lancedb_connection.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_logging.py
test_mcp.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_multi_db.py Search several configured databases and fuse the results 2026-08-24 10:03:45 +03:00
test_pdf_attachments.py Move source adapters out of the ingester package 2026-08-20 11:46:55 +03:00
test_pdf_concurrency.py
test_pdf_split.py
test_picture_description.py
test_picture_in_context.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_processing.py Resolve file:// URIs to paths through url2pathname 2026-08-21 10:22:26 +03:00
test_reader.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_rebuild.py Split the store module by responsibility 2026-08-20 12:13:51 +03:00
test_reranker.py Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
test_s3_integration.py Split the store module by responsibility 2026-08-20 12:13:51 +03:00
test_s3_source.py
test_search.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_search_phases.py Separate fetching from ranking in search 2026-08-24 10:03:45 +03:00
test_settings.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_telemetry.py
test_title_generation.py
test_uri.py Resolve file:// URIs to paths through url2pathname 2026-08-21 10:22:26 +03:00
test_utils.py Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
test_vacuum_debounce.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00
test_versioning.py Make get_config the only configuration lookup 2026-08-19 14:43:40 +03:00