`title` is the empty string for every cloud and fiqa passage, so two of the
four domains have exactly one title covering 72,442 and 61,022 passages, and
govt's titles are web-scrape artifacts with one 10,192-passage bucket. Only
clapnq has titles that identify a document.
Keeping whole titles therefore put the pooled gold floor at 146,543
passages: a budget of 120,000 yielded zero distractors, and 58 gold titles
alone accounted for 135,479 passages.
Passage level costs nothing the heterogeneous comparison needs. At alpha=0
the domain places a collection, so a query's gold is concentrated by
construction rather than by the atom, and a titleless domain now spreads
across its own collections instead of collapsing into one.
Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc
`mtrag_federated` partitions one domain by article title, which is
round-robin fusion's friendliest case: no collection is ever off-topic for a
query, so the guaranteed-slot waste that hurts a real deployment is never
exercised. Every fusion conclusion measured on it is therefore provisional.
`mtrag_pooled` pools all four MTRAG domains, so a query belongs to one and
the rest are genuinely off-topic. `collection_of` gains `alpha`, which now
means something: 0 keeps a collection to one domain, 1 ignores the domain
and shards titles uniformly. Domains map onto collections proportionally,
subdividing by title where there are more collections than domains and
grouping where there are fewer.
Passage ids are checked for collisions across domains, since gold is
uri-keyed and a shared id would make it ambiguous.
Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc
The chunks FTS index is created once when the table is created, over zero
rows, and nothing folds later rows into it but an optimize. build_databases
bypasses populate_db, and with it the closing vacuum, so every database it
built had an index covering nothing.
Full-text search then returns near-arbitrary rows while still returning
results, so nothing looks wrong: measured FTS recall@5 of 0.0000 across 208
queries on a 40k pool, with hybrid losing 9.9pp to pure vector because the
dead branch still consumed half the fused slots.
Reachable outside the eval too, on any storage.auto_vacuum: false, which
every reference config sets. Reproduced through create_document alone:
five documents, five chunks, zero indexed rows.
Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc
Hybrid search inside one database fuses its own vector and FTS rankings
with lancedb's RRFReranker over exactly the requested limit, and both
branch queries derive from the same inner query, so there is no
branch-depth knob. Below roughly 50 candidates the two rankings stop
overlapping, nothing sums, and the fusion degenerates: measured recall@5
on a single database was 0.000 at fetch 5, 10 and 20, then 0.267 at 50 and
0.350 at 100.
A dataset's retrieval_limit therefore fixes which regime it measures, and
comparing regimes would otherwise need one dataset per depth.
Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc
Measures whether cross-database fusion reaches what a query needs, scored
on retrieval alone so no model or judge sits between the fusion and the
number.
The corpus is MTRAG ClapNQ partitioned by article title, whole titles to a
collection, so an article's passages never split and a query's gold stays
concentrated in one collection, which is the condition a per-collection
depth quota punishes. collection_of keys on sha256 rather than hash(),
which is salted per process: the partition is never stored, and scoring
recomputes it in a different process than the one that ingested.
The 148 titles holding a gold passage carry 10,723 passages between them,
so a budget near that floor leaves no cross-topic distractors and inflates
recall. The default is 40,000 and the build reports the gold/distractor
split, warning when there are none.
build_databases opens each collection by configured name with a scope of
one, since populate_db writes to a single database. The operator entry
point emits the config for the partition it just built, so a config cannot
search a differently-partitioned build.
Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc
