Tighten the multi-database documentation
Restructure the storage page into search and provenance, duplicate ids, ranking, Python operations and CLI commands, and state each in reference voice. `Several Databases` becomes `Multiple Databases`, with the anchor carried through every referrer, and the instruction files follow the same name. Example databases are `papers`, `wiki` and `notes`. `AmbiguousCitationError` is raised for a cited chunk id held by more than one selected database, not for any shared id.
This commit is contained in:
parent
1995a7da36
commit
8ed24d0e24
11 changed files with 157 additions and 158 deletions
24
CHANGELOG.md
24
CHANGELOG.md
|
|
@ -4,17 +4,27 @@
|
||||||
|
|
||||||
### Added
|
### Added
|
||||||
|
|
||||||
- `lancedb.databases`: a name-to-location mapping for searching several databases at once, mutually exclusive with `lancedb.uri`. `client.search(..., sources=[...])` selects which to search, `sources=None` searches all of them, and `SearchResult.source` carries the configured name a result came from. `Document.source` names it on a document from a listing or a lookup, so a listing that spans databases says which one each came from. Candidates are fused by the configured reranker over the union, or by reciprocal rank fusion when none is configured. Databases searched together must have been written with the same embedder; two that disagree raise `ConfigMismatchError`. The query is embedded once for the whole selection. `SearchResult.format_for_agent` names the database, so the model can attribute evidence to one while it answers. `haiku-rag search`, `ask`, `analyze` and `chat` cover the configured set and label each result and citation with its database. `settings`, `init-config` and `download-models` open no database; every other command works on one, named with `--db-name NAME` or `--db PATH`, or resolved from a configured set of one.
|
- `lancedb.databases` configures a named set of local or remote databases. `search`,
|
||||||
- `client.ask(..., sources=[...])` asks across the selected databases, and `Citation.source` names the one a cited chunk came from. The cite fallback for an id absent from the run's results looks only in the selected databases, so a question scoped to some cannot cite another. A chunk id held by two of the databases searched raises `AmbiguousCitationError`, which reaches the model as a retry; the fallback refuses it too, rather than answering from the first database that holds it.
|
`ask` and `analyze` accept a `sources` subset; results, documents and citations
|
||||||
- `client.analyze(..., sources=[...])` analyzes across the selected databases: the sandbox mounts their documents under one flat `/documents/{id}/` namespace, resolving each id to the database holding it, and in-code `search()` covers the same selection.
|
include the originating database in `source`. Search results are combined with
|
||||||
|
the configured reranker, or reciprocal rank fusion when reranking is disabled.
|
||||||
|
- `haiku-rag search`, `ask`, `analyze` and `chat` can cover a configured database
|
||||||
|
set. Commands that access one database select it with `--db-name NAME` or
|
||||||
|
`--db PATH`.
|
||||||
|
- Citation resolution rejects chunk IDs shared by multiple selected databases
|
||||||
|
with `AmbiguousCitationError`.
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
- `client.chunk()` and `client.embedder` work on a client covering `lancedb.databases`: an embedder is a function of configuration, so the client builds one on first use and closes it on teardown. Operations that need one database (`create_document`, `import_document(s)`, `create_document_from_source`, `update_document`, `delete_document`, `rebuild_database`, `vacuum`, `visualize_chunk`, `close`) raise `AmbiguousDatabaseError` naming the databases covered, instead of `AttributeError`.
|
- `client.chunk()` and `client.embedder` work when the client covers several
|
||||||
- The chat TUI's document filter selects documents by id and names each document's database, instead of matching the displayed title or URI as a substring across every database.
|
databases. Operations that require one database raise `AmbiguousDatabaseError`.
|
||||||
|
- The chat document filter selects by document ID and shows each document's
|
||||||
|
database.
|
||||||
- `haiku-rag` prints the message and exits when the configured embedder does not match the database, instead of raising a traceback.
|
- `haiku-rag` prints the message and exits when the configured embedder does not match the database, instead of raising a traceback.
|
||||||
- A capability built without a client opens the databases the configuration places — `lancedb.uri` or the whole `lancedb.databases` set — instead of the default under `storage.data_dir`.
|
- Capabilities created without a client now honor `lancedb.uri` and
|
||||||
- A `lancedb.uri` with no scheme is a local path, as it already is in `lancedb.databases`: `haiku-rag init` creates it and every command that opens an existing database requires it to exist, where a missing path was opened as object storage and became an empty database. `--db PATH` overrides `lancedb.uri`.
|
`lancedb.databases`.
|
||||||
|
- A `lancedb.uri` without a scheme is treated as a local path. `--db PATH`
|
||||||
|
overrides it.
|
||||||
|
|
||||||
## [0.78.0] - 2026-08-24
|
## [0.78.0] - 2026-08-24
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -24,7 +24,7 @@ Built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/
|
||||||
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
||||||
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
||||||
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, Cohere, LM Studio, vLLM (multimodal via `multimodal: true` on vLLM/VoyageAI/Cohere). QA: any model supported by Pydantic AI
|
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, Cohere, LM Studio, vLLM (multimodal via `multimodal: true` on vLLM/VoyageAI/Cohere). QA: any model supported by Pydantic AI
|
||||||
- **Several databases** — Name databases in `lancedb.databases` and search, ask, analyze or chat across them at once, with each result and citation carrying the database it came from
|
- **Multi-database search** — Search, ask, analyze, or chat across named databases with source attribution on results and citations
|
||||||
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
||||||
- **CLI & Python API** — Full functionality from command line or code
|
- **CLI & Python API** — Full functionality from command line or code
|
||||||
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
||||||
|
|
|
||||||
|
|
@ -143,18 +143,17 @@ Capabilities use a plain `state: dict[str, Any]` attribute on agent dependencies
|
||||||
|
|
||||||
Applications serving AG-UI should adapt the agent with Pydantic AI's `AGUIAdapter`. Native model and tool events require no haiku.rag-specific bridge.
|
Applications serving AG-UI should adapt the agent with Pydantic AI's `AGUIAdapter`. Native model and tool events require no haiku.rag-specific bridge.
|
||||||
|
|
||||||
## Which databases a capability covers
|
## Database Selection
|
||||||
|
|
||||||
Both factories resolve this once, in order:
|
RAG and analysis capabilities select databases in this order:
|
||||||
|
|
||||||
1. The `db_path` argument, which covers that one database.
|
1. The `db_path` argument.
|
||||||
2. `HAIKU_RAG_DB`, the same way.
|
2. `HAIKU_RAG_DB`.
|
||||||
3. [`lancedb.databases`](../configuration/storage.md#several-databases), covering the
|
3. [`lancedb.databases`](../configuration/storage.md#multiple-databases), which
|
||||||
whole configured set. A capability covering several says so in its instructions, so
|
selects the full configured set.
|
||||||
the model can attribute evidence to one while it answers.
|
|
||||||
4. [`lancedb.uri`](../configuration/storage.md#changing-the-default-database-path),
|
4. [`lancedb.uri`](../configuration/storage.md#changing-the-default-database-path),
|
||||||
covering the one database it places.
|
which selects one database.
|
||||||
5. `config.storage.data_dir / "haiku.rag.lancedb"`.
|
5. `config.storage.data_dir / "haiku.rag.lancedb"`.
|
||||||
|
|
||||||
Passing a live client through `rag=` overrides all of it: the capability reads what that
|
Passing a client through `rag=` bypasses this selection. The capability uses the
|
||||||
client covers, and never closes it.
|
databases covered by that client and does not close it.
|
||||||
|
|
|
||||||
12
docs/cli.md
12
docs/cli.md
|
|
@ -20,15 +20,15 @@ The `haiku-rag` CLI provides complete document management functionality.
|
||||||
haiku-rag --config /path/to/config.yaml list
|
haiku-rag --config /path/to/config.yaml list
|
||||||
haiku-rag --config /path/to/config.yaml list --db /path/to/custom.db
|
haiku-rag --config /path/to/config.yaml list --db /path/to/custom.db
|
||||||
haiku-rag --read-only search "query"
|
haiku-rag --read-only search "query"
|
||||||
haiku-rag --db-name medic list
|
haiku-rag --db-name papers list
|
||||||
haiku-rag add -h
|
haiku-rag add -h
|
||||||
```
|
```
|
||||||
|
|
||||||
With `lancedb.databases` configured, `search`, `ask`, `analyze` and `chat`
|
With `lancedb.databases` configured, `search`, `ask`, `analyze`, and `chat`
|
||||||
cover every database in it. `settings`, `init-config` and `download-models`
|
use the full set by default. Select one database for other commands with
|
||||||
open no database at all. Every other command works on one, named with
|
`--db-name` or `--db`. `settings`, `init-config`, and `download-models` do
|
||||||
`--db-name` or `--db`. See
|
not open a database. See
|
||||||
[Several Databases](configuration/storage.md#several-databases).
|
[Multiple Databases](configuration/storage.md#multiple-databases).
|
||||||
|
|
||||||
## Document Management
|
## Document Management
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -80,7 +80,7 @@ lancedb:
|
||||||
An explicit `--db PATH` overrides `lancedb.uri` for that invocation.
|
An explicit `--db PATH` overrides `lancedb.uri` for that invocation.
|
||||||
|
|
||||||
This places one database without naming it. Its `source` is `None` in search
|
This places one database without naming it. Its `source` is `None` in search
|
||||||
results, citations and documents, since only [`lancedb.databases`](#several-databases)
|
results, citations and documents, since only [`lancedb.databases`](#multiple-databases)
|
||||||
assigns the names that carry provenance. A path here changes where the database
|
assigns the names that carry provenance. A path here changes where the database
|
||||||
lives, not what it is called.
|
lives, not what it is called.
|
||||||
|
|
||||||
|
|
@ -198,132 +198,129 @@ The recommended layout for production is "different buckets, same account, separ
|
||||||
|
|
||||||
Each process picks up its own credentials from the AWS default chain (env vars, IAM instance role, AWS profile), so no credentials are hard-coded in the configuration files.
|
Each process picks up its own credentials from the AWS default chain (env vars, IAM instance role, AWS profile), so no credentials are hard-coded in the configuration files.
|
||||||
|
|
||||||
## Several Databases
|
## Multiple Databases
|
||||||
|
|
||||||
`lancedb.databases` maps a name to a location, for searching several databases at
|
Use `lancedb.databases` to name local or remote databases that should be searched
|
||||||
once:
|
together:
|
||||||
|
|
||||||
```yaml
|
```yaml
|
||||||
lancedb:
|
lancedb:
|
||||||
databases:
|
databases:
|
||||||
medic: s3://my-bucket/medic.lancedb
|
papers: s3://my-bucket/papers.lancedb
|
||||||
st: s3://my-bucket/st.lancedb
|
wiki: s3://my-bucket/wiki.lancedb
|
||||||
local: /data/notes.lancedb
|
notes: /data/notes.lancedb
|
||||||
```
|
```
|
||||||
|
|
||||||
A location is a URI or a local path. `databases` and `uri` are mutually
|
A location can be a URI or local path. `databases` and `uri` are mutually
|
||||||
exclusive, and setting both fails validation.
|
exclusive.
|
||||||
|
|
||||||
Results, citations, model input and errors opening a named database carry the
|
Results, documents, citations, and model context use the configured name as
|
||||||
configured name rather than the location, so a path or a bucket does not reach a
|
`source`. Commands such as `info` and path-related errors still show locations.
|
||||||
trace or a model. Commands that report on a database — `info`, `init`, `tag` —
|
|
||||||
print its location, as does an error about a path.
|
|
||||||
|
|
||||||
Every database in the set is opened with the same embedding configuration. A
|
All databases in a search must use compatible embeddings because the query is
|
||||||
different `vector_dim` raises `ConfigMismatchError` on open. A different provider
|
embedded once. A dimension mismatch raises `ConfigMismatchError`. A provider or
|
||||||
or model name at the same dimension is a warning on a read-only open, since the
|
model-name mismatch at the same dimension warns in read-only mode and raises in
|
||||||
same model served by another stack is spelled differently, and raises on a
|
writable mode.
|
||||||
writable one.
|
|
||||||
|
|
||||||
Searching embeds the query once for the whole selection, so the databases
|
### Search and Provenance
|
||||||
searched together must have been written with the same embedder. Two that
|
|
||||||
disagree with each other raise `ConfigMismatchError` naming both, whatever the
|
|
||||||
configuration says.
|
|
||||||
|
|
||||||
### Searching a set
|
`search`, `ask`, and `analyze` use the full set by default. Pass `sources` to
|
||||||
|
select a subset:
|
||||||
`search`, `ask` and `analyze` cover the whole set, or the subset named by
|
|
||||||
`sources`:
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
results = await client.search("query") # every database
|
results = await client.search("query") # every database
|
||||||
results = await client.search("query", sources=["medic"]) # one of them
|
results = await client.search("query", sources=["papers"]) # one of them
|
||||||
```
|
```
|
||||||
|
|
||||||
Candidates from each database are fused into one ranked list, by the configured
|
Candidates are combined into one ranked list with the configured reranker, or
|
||||||
reranker where there is one and by reciprocal rank fusion otherwise. Each result
|
with reciprocal rank fusion when reranking is disabled. `SearchResult.source`,
|
||||||
carries `source`, the name of the database it came from, and so does each
|
`Citation.source`, and `Document.source` contain the configured database name.
|
||||||
citation. A document from `list_documents`, `get_document_by_id`,
|
The name is retained when a client covers only one named database. Databases
|
||||||
`get_document_by_uri` or `resolve_document` carries it too, and a database
|
configured through `lancedb.uri` are unnamed, so their `source` is `None`.
|
||||||
named in `lancedb.databases` keeps that name even when it is the only one a
|
|
||||||
client covers. Only a database placed by `lancedb.uri`, which names none, has no
|
|
||||||
name to carry. The CLI does not print the name of a single database it was told
|
|
||||||
to use, since the caller just named it.
|
|
||||||
|
|
||||||
Chunk ids are unique within a database and say nothing across them, so a database
|
The CLI labels results and citations only when the operation spans multiple
|
||||||
copied from another holds the same ids. Results are told apart by the database
|
databases. A command already narrowed with `--db-name` does not repeat the name
|
||||||
and the id together. A chunk id held by two of the databases searched cannot be
|
on every result.
|
||||||
cited: `resolve_citations` raises `AmbiguousCitationError`, and the capability
|
|
||||||
asks the model for other evidence instead.
|
|
||||||
|
|
||||||
Document ids behave the same way, and two places treat a collision differently.
|
#### Duplicate IDs
|
||||||
The analysis sandbox mounts one document per id and refuses a set where two
|
|
||||||
databases claim one, since the mount has a single path per id. The chat document
|
|
||||||
filter does not: it selects by document id and applies `id IN (...)` to every
|
|
||||||
covered database, so selecting an id that exists in copies of a database matches
|
|
||||||
the document in each of them. Independently built databases use UUID document
|
|
||||||
ids and do not collide.
|
|
||||||
|
|
||||||
**Configure a reranker when searching several databases.** Reciprocal rank fusion
|
IDs are unique within a database, not across databases. Copies of a database
|
||||||
compares ranks, not scores, so every database contributes its own best matches
|
therefore retain the same IDs. Citation resolution raises
|
||||||
whether or not they are relevant to the question, and results from databases
|
`AmbiguousCitationError` when a cited chunk ID exists in more than one selected
|
||||||
holding nothing relevant displace better ones. A reranker scores the whole union
|
database; a shared ID that nothing cites is ignored. The analysis sandbox
|
||||||
instead, which removes the effect. Measured on one corpus split three ways, with
|
rejects shared document IDs because its mount path is `/documents/{id}/`.
|
||||||
the same queries: retrieval MAP 0.9914 with a reranker against 0.9918 for the
|
|
||||||
same corpus in a single database, and 0.6044 without one against 0.9798. The cost
|
|
||||||
is that a reranker scores candidates in proportion to the number of databases.
|
|
||||||
|
|
||||||
Without one, consider raising `search.limit` with the number of databases
|
The chat document filter selects by document ID and applies `id IN (...)` to
|
||||||
searched. Each contributes its own best matches to a list that is then truncated
|
every covered database, so selecting an ID that copies share matches the
|
||||||
at the limit, so with three full rankings and a limit of 5 any one database may
|
document in each of them.
|
||||||
contribute only one or two results. Raising the limit raises what the caller and
|
|
||||||
the model receive, since without a reranker nothing is over-fetched to absorb
|
|
||||||
it.
|
|
||||||
|
|
||||||
Creating names a database: `create=True` on a client covering the set raises
|
#### Ranking
|
||||||
`AmbiguousDatabaseError`, and `HaikuRAG(config=config, create=True,
|
|
||||||
sources=["name"])` creates that one.
|
|
||||||
|
|
||||||
Converting, chunking and title generation are functions of the configuration
|
Configure a reranker when searching multiple databases. Reciprocal rank fusion
|
||||||
rather than of a database, so they work on a client covering the set. Writing,
|
compares positions rather than scores, so each database contributes top-ranked
|
||||||
rebuilding and vacuuming name one database: asking a set-covering client raises
|
results even when another database has stronger matches. A reranker scores the
|
||||||
`AmbiguousDatabaseError`, and `client.clients_for(["name"])` returns a client for
|
combined candidate set directly.
|
||||||
one of them, writable when the covering client is.
|
|
||||||
|
|
||||||
A database that cannot be opened fails the whole query and is named in the error.
|
In a 3,045-query evaluation over a corpus split across three databases,
|
||||||
A result set silently missing one of the databases asked for cannot be told apart
|
reranking produced retrieval MAP 0.9914, compared with 0.9918 for the same corpus
|
||||||
from a complete one.
|
in one database. Without a reranker, MAP was 0.6044, compared with 0.9798 in one
|
||||||
|
database. Reranking cost grows with the number of databases because each
|
||||||
|
contributes candidates.
|
||||||
|
|
||||||
### How commands treat the set
|
Without a reranker, consider increasing `search.limit` with the number of
|
||||||
|
databases. With three complete rankings and a limit of 5, a database may
|
||||||
|
contribute only one or two results. A higher limit also sends more results to
|
||||||
|
the caller and model.
|
||||||
|
|
||||||
Commands fall into three groups:
|
If any selected database fails to open, the operation fails and identifies that
|
||||||
|
database.
|
||||||
|
|
||||||
- **Set-capable**: `search`, `ask`, `analyze` and `chat` cover the whole
|
### Python Operations
|
||||||
|
|
||||||
|
Creating, writing, rebuilding, and vacuuming require one database. Calling these
|
||||||
|
operations on a client that covers several raises `AmbiguousDatabaseError`.
|
||||||
|
Select one at creation time or obtain a single-database client:
|
||||||
|
|
||||||
|
```python
|
||||||
|
async with HaikuRAG(config=config, create=True, sources=["papers"]) as papers:
|
||||||
|
...
|
||||||
|
|
||||||
|
async with HaikuRAG(config=config) as client:
|
||||||
|
papers = (await client.clients_for(["papers"]))[0]
|
||||||
|
```
|
||||||
|
|
||||||
|
Conversion, chunking, and title generation do not access a database and remain
|
||||||
|
available on a multi-database client.
|
||||||
|
|
||||||
|
### CLI Commands
|
||||||
|
|
||||||
|
Commands use database sets as follows:
|
||||||
|
|
||||||
|
- **Set-capable**: `search`, `ask`, `analyze`, and `chat` use the full
|
||||||
configured set, or the subset named by `--db-name`.
|
configured set, or the subset named by `--db-name`.
|
||||||
- **Config-only**: `settings`, `init-config` and `download-models` open no
|
- **Config-only**: `settings`, `init-config`, and `download-models` do not open
|
||||||
database, so the set is irrelevant to them.
|
a database.
|
||||||
- **Single-database**: everything else — document writes, `rebuild`, `vacuum`,
|
- **Single-database**: everything else — document writes, `rebuild`, `vacuum`,
|
||||||
`migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`,
|
`migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`,
|
||||||
`visualize` and `mcp` — works on one database, named with the global
|
`visualize`, and `mcp` — works on one database, selected with the global
|
||||||
`--db-name` option.
|
`--db-name` option.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
haiku-rag search "query" # every configured database
|
haiku-rag search "query" # every configured database
|
||||||
haiku-rag --db-name medic list # one of them
|
haiku-rag --db-name papers list # one of them
|
||||||
haiku-rag --db-name medic migrate
|
haiku-rag --db-name papers migrate
|
||||||
```
|
```
|
||||||
|
|
||||||
`--db-name` takes a name from `lancedb.databases`, which is how a database
|
`--db-name` selects an entry from `lancedb.databases`, including remote entries.
|
||||||
behind a URI is reached. `--db` takes a path, and overrides the configured
|
`--db` selects a local path and overrides the configured location. A
|
||||||
location with that one database. A single-database command given neither fails
|
single-database command requires one of these options when multiple databases are
|
||||||
rather than choosing for you, unless `lancedb.databases` names exactly one: a
|
configured. A configured set of one is selected automatically.
|
||||||
set of one is unambiguous and is used, keeping its configured name.
|
|
||||||
|
|
||||||
Each database is created, migrated and vacuumed on its own:
|
Each database is created, migrated and vacuumed on its own:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
haiku-rag --db-name medic init
|
haiku-rag --db-name papers init
|
||||||
haiku-rag --db-name st init
|
haiku-rag --db-name wiki init
|
||||||
```
|
```
|
||||||
|
|
||||||
## Vector Indexing
|
## Vector Indexing
|
||||||
|
|
|
||||||
|
|
@ -230,56 +230,51 @@ for result in results:
|
||||||
print(f"Document Title: {result.document_title}") # when available
|
print(f"Document Title: {result.document_title}") # when available
|
||||||
```
|
```
|
||||||
|
|
||||||
### Searching Several Databases
|
### Searching Multiple Databases
|
||||||
|
|
||||||
With [`lancedb.databases`](configuration/storage.md#several-databases)
|
With [`lancedb.databases`](configuration/storage.md#multiple-databases)
|
||||||
configured, a client covers every database in it. `sources` narrows a call to
|
configured, a client covers the full set. Use `sources` to select a subset.
|
||||||
some of them, and each result names the database it came from:
|
Each result includes its database name:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
results = await client.search("machine learning") # all of them
|
results = await client.search("machine learning") # all of them
|
||||||
results = await client.search("machine learning", sources=["medic"]) # one of them
|
results = await client.search("machine learning", sources=["papers"]) # one of them
|
||||||
|
|
||||||
for result in results:
|
for result in results:
|
||||||
print(f"{result.source}: {result.content}")
|
print(f"{result.source}: {result.content}")
|
||||||
```
|
```
|
||||||
|
|
||||||
`ask` and `analyze` take `sources` too, and every citation carries the database
|
`ask` and `analyze` also accept `sources`. Citations include the database name:
|
||||||
it was drawn from:
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
answer, citations = await client.ask("What changed?", sources=["medic", "st"])
|
answer, citations = await client.ask("What changed?", sources=["papers", "wiki"])
|
||||||
for cite in citations:
|
for cite in citations:
|
||||||
print(f"[{cite.source}] {cite.document_title or cite.document_uri}")
|
print(f"[{cite.source}] {cite.document_title or cite.document_uri}")
|
||||||
|
|
||||||
result = await client.analyze("How many documents mention it?", sources=["medic"])
|
result = await client.analyze("How many documents mention it?", sources=["papers"])
|
||||||
```
|
```
|
||||||
|
|
||||||
A question scoped to some databases can only cite those, and the analysis
|
A scoped question can cite only the selected databases. Analysis mounts only
|
||||||
sandbox mounts only their documents.
|
their documents.
|
||||||
|
|
||||||
`sources=None` covers every database the client covers. `sources=[]` covers
|
`sources=None` covers every database the client covers. `sources=[]` covers
|
||||||
none: `search` returns no results, and `ask` and `analyze` run with no evidence
|
none: `search` returns no results, and `ask` and `analyze` run with no evidence
|
||||||
from any database.
|
from any database.
|
||||||
|
|
||||||
#### Asking a client what it covers
|
#### Inspecting the client scope
|
||||||
|
|
||||||
```python
|
```python
|
||||||
client.covers_multiple # True while reading more than one database
|
client.covers_multiple # whether the client covers more than one database
|
||||||
client.source_names # the configured names covered, in configured order
|
client.source_names # configured names, in order
|
||||||
client.source # the one name, or None while covering a set
|
client.source # one configured name, or None for a set or unnamed database
|
||||||
|
|
||||||
owner = await client.reader_for("medic") # the client reading that database
|
owner = await client.reader_for("papers") # the client reading that database
|
||||||
medic, st = await client.clients_for(["medic", "st"])
|
papers, wiki = await client.clients_for(["papers", "wiki"])
|
||||||
```
|
```
|
||||||
|
|
||||||
`clients_for` opens the databases it names, on first use rather than at entry,
|
`reader_for` and `clients_for` open databases lazily and return borrowed clients.
|
||||||
and returns a client for each. Those clients borrow their databases from the
|
They remain valid while the covering client is open and inherit its read-only
|
||||||
covering one: they are valid only while it is open, and closing or entering one
|
mode. The covering client owns and closes their database sessions.
|
||||||
leaves its database alone. The covering client closes them all on teardown.
|
|
||||||
|
|
||||||
A borrowed client reads one database, so it has the `store` and repositories a
|
|
||||||
covering client cannot have, and it is writable when the covering client is.
|
|
||||||
|
|
||||||
### Filtering Search Results
|
### Filtering Search Results
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -90,20 +90,18 @@ By default, evaluation databases are stored in the haiku.rag data directory:
|
||||||
|
|
||||||
You can override this with the `--db` option.
|
You can override this with the `--db` option.
|
||||||
|
|
||||||
### Evaluating over several databases
|
### Evaluating over Multiple Databases
|
||||||
|
|
||||||
With [`lancedb.databases`](https://ggozad.github.io/haiku.rag/configuration/storage/#several-databases)
|
With [`lancedb.databases`](https://ggozad.github.io/haiku.rag/configuration/storage/#multiple-databases)
|
||||||
configured, `evaluations run <dataset> --skip-db` benchmarks the configured set:
|
configured, `evaluations run <dataset> --skip-db` benchmarks the full set.
|
||||||
retrieval, QA and live conversations all search every database in it, and each
|
Retrieval, QA, and live conversations preserve the database name on results and
|
||||||
result and citation names the one it came from. A mapping of one follows the same
|
citations. A configured set of one follows the same path and retains its name.
|
||||||
path and keeps its configured name.
|
|
||||||
|
|
||||||
Population is not part of that. It writes one database, so it needs a path:
|
Population writes one database and therefore requires `--db`:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
evaluations run hotpotqa --db /path/to/one.lancedb # populate, then benchmark
|
evaluations run hotpotqa --db /path/to/one.lancedb # populate, then benchmark
|
||||||
evaluations run hotpotqa --skip-db # benchmark the configured set
|
evaluations run hotpotqa --skip-db # benchmark the configured set
|
||||||
```
|
```
|
||||||
|
|
||||||
A `--db` path names one database and overrides the configured set for the whole
|
`--db` overrides the configured set for both population and benchmarks.
|
||||||
run, population and benchmarks alike.
|
|
||||||
|
|
|
||||||
|
|
@ -25,8 +25,8 @@ STATE_NAMESPACE = "analysis"
|
||||||
_CAPABILITY_ID = "haiku-rag-analysis"
|
_CAPABILITY_ID = "haiku-rag-analysis"
|
||||||
_TOOL_NAMES = frozenset({"analysis_search", "analysis_execute_code", "analysis_cite"})
|
_TOOL_NAMES = frozenset({"analysis_search", "analysis_execute_code", "analysis_cite"})
|
||||||
_instructions_path = Path(__file__).parent / "instructions" / "analysis.md"
|
_instructions_path = Path(__file__).parent / "instructions" / "analysis.md"
|
||||||
_several_databases_path = (
|
_multiple_databases_path = (
|
||||||
Path(__file__).parent / "instructions" / "analysis_several_databases.md"
|
Path(__file__).parent / "instructions" / "analysis_multiple_databases.md"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -44,10 +44,10 @@ def instructions() -> str:
|
||||||
|
|
||||||
|
|
||||||
@cache
|
@cache
|
||||||
def several_databases_instructions() -> str:
|
def multiple_databases_instructions() -> str:
|
||||||
"""Appended only where the capability covers several databases, so a single
|
"""Appended only where the capability covers several databases, so a single
|
||||||
database is instructed exactly as it was before they could be named."""
|
database is instructed exactly as it was before they could be named."""
|
||||||
return _several_databases_path.read_text().rstrip()
|
return _multiple_databases_path.read_text().rstrip()
|
||||||
|
|
||||||
|
|
||||||
def _recovery_hint(stderr: str) -> str:
|
def _recovery_hint(stderr: str) -> str:
|
||||||
|
|
@ -217,7 +217,7 @@ def create_capability(
|
||||||
# A lent client covers what it covers; otherwise the scope says.
|
# A lent client covers what it covers; otherwise the scope says.
|
||||||
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
||||||
if several:
|
if several:
|
||||||
instruction_text += several_databases_instructions()
|
instruction_text += multiple_databases_instructions()
|
||||||
return AnalysisCapability(
|
return AnalysisCapability(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
config=config,
|
config=config,
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
|
|
||||||
## Several databases
|
## Multiple databases
|
||||||
|
|
||||||
The corpus spans several databases, and each interface names them differently:
|
The corpus spans several databases, and each interface names them differently:
|
||||||
|
|
||||||
|
|
@ -1,5 +1,5 @@
|
||||||
|
|
||||||
## Several databases
|
## Multiple databases
|
||||||
|
|
||||||
The corpus spans several databases. Each result carries a `Database:` line naming
|
The corpus spans several databases. Each result carries a `Database:` line naming
|
||||||
the configured database it came from. Say which database an answer draws on when
|
the configured database it came from. Say which database an answer draws on when
|
||||||
|
|
@ -29,8 +29,8 @@ STATE_NAMESPACE = "rag"
|
||||||
_CAPABILITY_ID = "haiku-rag"
|
_CAPABILITY_ID = "haiku-rag"
|
||||||
_TOOL_NAMES = frozenset({"rag_search", "rag_cite"})
|
_TOOL_NAMES = frozenset({"rag_search", "rag_cite"})
|
||||||
_instructions_path = Path(__file__).parent / "instructions" / "rag.md"
|
_instructions_path = Path(__file__).parent / "instructions" / "rag.md"
|
||||||
_several_databases_path = (
|
_multiple_databases_path = (
|
||||||
Path(__file__).parent / "instructions" / "rag_several_databases.md"
|
Path(__file__).parent / "instructions" / "rag_multiple_databases.md"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -44,10 +44,10 @@ def instructions() -> str:
|
||||||
|
|
||||||
|
|
||||||
@cache
|
@cache
|
||||||
def several_databases_instructions() -> str:
|
def multiple_databases_instructions() -> str:
|
||||||
"""Appended only where the capability covers several databases, so a single
|
"""Appended only where the capability covers several databases, so a single
|
||||||
database is instructed exactly as it was before they could be named."""
|
database is instructed exactly as it was before they could be named."""
|
||||||
return _several_databases_path.read_text().rstrip()
|
return _multiple_databases_path.read_text().rstrip()
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
@dataclass
|
||||||
|
|
@ -120,7 +120,7 @@ def create_capability(
|
||||||
# A lent client covers what it covers; otherwise the scope says.
|
# A lent client covers what it covers; otherwise the scope says.
|
||||||
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
||||||
if several:
|
if several:
|
||||||
instruction_text += several_databases_instructions()
|
instruction_text += multiple_databases_instructions()
|
||||||
return RAGCapability(
|
return RAGCapability(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
config=config,
|
config=config,
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue