Say what makes a cited chunk id ambiguous
The documented rule was existence in more than one selected database. The rule is narrower and does not need to be wider: a copy the search never returned grounded nothing, so the retrieved result is the citation and its database is recorded rather than guessed. The page now states the whole rule, including the fallback's own check for an id no search returned. Two retrieved copies are refused, and so is an id already cited from another database in the conversation. The case between them had no test.
This commit is contained in:
parent
206d29b74a
commit
87367759f9
3 changed files with 48 additions and 6 deletions
|
|
@ -11,8 +11,8 @@
|
|||
- `haiku-rag search`, `ask`, `analyze` and `chat` can cover a configured database
|
||||
set. Commands that access one database select it with `--db-name NAME` or
|
||||
`--db PATH`.
|
||||
- Citation resolution rejects chunk IDs shared by multiple selected databases
|
||||
with `AmbiguousCitationError`.
|
||||
- Citation resolution rejects a chunk ID retrieved from, or previously cited
|
||||
from, more than one selected database with `AmbiguousCitationError`.
|
||||
|
||||
### Fixed
|
||||
|
||||
|
|
|
|||
|
|
@ -260,10 +260,17 @@ on every result.
|
|||
#### Duplicate IDs
|
||||
|
||||
IDs are unique within a database, not across databases. Copies of a database
|
||||
therefore retain the same IDs. Citation resolution raises
|
||||
`AmbiguousCitationError` when a cited chunk ID exists in more than one selected
|
||||
database; a shared ID that nothing cites is ignored. The analysis sandbox
|
||||
rejects shared document IDs because its mount path is `/documents/{id}/`.
|
||||
therefore retain the same IDs.
|
||||
|
||||
Citation ambiguity is evaluated against evidence available to the run. A cited
|
||||
chunk ID is rejected with `AmbiguousCitationError` if search returned it from
|
||||
multiple databases, or it was previously cited from another database. If only
|
||||
one retrieved result has the ID, that result is cited. For an ID absent from
|
||||
search results, the fallback checks every selected database and rejects
|
||||
multiple holders. A shared ID that nothing cites is ignored.
|
||||
|
||||
The analysis sandbox rejects shared document IDs because its mount path is
|
||||
`/documents/{id}/`.
|
||||
|
||||
The chat document filter selects by document ID and applies `id IN (...)` to
|
||||
every covered database, so selecting an ID that copies share matches the
|
||||
|
|
|
|||
|
|
@ -232,6 +232,41 @@ class TestSharedChunkIds:
|
|||
with pytest.raises(ModelRetry, match="another database"):
|
||||
await capability._cite(["c1"])
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_one_retrieved_copy_of_a_shared_id_cites_where_it_came_from(
|
||||
self, tmp_path
|
||||
):
|
||||
"""Rejection is about evidence the run holds, not about what the other
|
||||
databases contain. A copy the search never returned did not ground the
|
||||
answer, so the retrieved one is the citation and its database is not a
|
||||
guess. A later question that does retrieve the twin is refused by the
|
||||
citation index."""
|
||||
from tests.capabilities.test_capabilities import Deps, make_context
|
||||
|
||||
config = _config(tmp_path, ["alpha", "beta"])
|
||||
capability = create_capability(config=config, defer_loading=False)
|
||||
deps = Deps(state={"rag": RAGState().model_dump(mode="json")})
|
||||
run = await capability.for_run(make_context(deps))
|
||||
assert run.state is not None
|
||||
# What a run holds when fusion returned one copy of a shared id and
|
||||
# truncated the other: the run never saw the twin.
|
||||
run.state.searches["cats"] = [
|
||||
SearchResult(
|
||||
content="alpha body",
|
||||
score=0.9,
|
||||
source="alpha",
|
||||
chunk_id="c1",
|
||||
document_id="d1",
|
||||
document_uri="test://alpha/one",
|
||||
)
|
||||
]
|
||||
|
||||
await run._cite(["c1"])
|
||||
|
||||
[cited] = run.state.citation_index.values()
|
||||
assert cited.source == "alpha"
|
||||
assert cited.content == "alpha body"
|
||||
|
||||
|
||||
class TestCitationSource:
|
||||
def test_a_citation_carries_the_result_source(self):
|
||||
|
|
|
|||
Loading…
Reference in a new issue