Tighten the multi-database documentation
Restructure the storage page into search and provenance, duplicate ids, ranking, Python operations and CLI commands, and state each in reference voice. `Several Databases` becomes `Multiple Databases`, with the anchor carried through every referrer, and the instruction files follow the same name. Example databases are `papers`, `wiki` and `notes`. `AmbiguousCitationError` is raised for a cited chunk id held by more than one selected database, not for any shared id.
This commit is contained in:
Yiorgis Gozadinos
parent
1995a7da36
commit
8ed24d0e24
11 changed files with 157 additions and 158 deletions
24
CHANGELOG.md
24
CHANGELOG.md
|
|
@ -4,17 +4,27 @@
|
|||
|
||||
### Added
|
||||
|
||||
- `lancedb.databases`: a name-to-location mapping for searching several databases at once, mutually exclusive with `lancedb.uri`. `client.search(..., sources=[...])` selects which to search, `sources=None` searches all of them, and `SearchResult.source` carries the configured name a result came from. `Document.source` names it on a document from a listing or a lookup, so a listing that spans databases says which one each came from. Candidates are fused by the configured reranker over the union, or by reciprocal rank fusion when none is configured. Databases searched together must have been written with the same embedder; two that disagree raise `ConfigMismatchError`. The query is embedded once for the whole selection. `SearchResult.format_for_agent` names the database, so the model can attribute evidence to one while it answers. `haiku-rag search`, `ask`, `analyze` and `chat` cover the configured set and label each result and citation with its database. `settings`, `init-config` and `download-models` open no database; every other command works on one, named with `--db-name NAME` or `--db PATH`, or resolved from a configured set of one.
|
||||
- `client.ask(..., sources=[...])` asks across the selected databases, and `Citation.source` names the one a cited chunk came from. The cite fallback for an id absent from the run's results looks only in the selected databases, so a question scoped to some cannot cite another. A chunk id held by two of the databases searched raises `AmbiguousCitationError`, which reaches the model as a retry; the fallback refuses it too, rather than answering from the first database that holds it.
|
||||
- `client.analyze(..., sources=[...])` analyzes across the selected databases: the sandbox mounts their documents under one flat `/documents/{id}/` namespace, resolving each id to the database holding it, and in-code `search()` covers the same selection.
|
||||
- `lancedb.databases` configures a named set of local or remote databases. `search`,
|
||||
`ask` and `analyze` accept a `sources` subset; results, documents and citations
|
||||
include the originating database in `source`. Search results are combined with
|
||||
the configured reranker, or reciprocal rank fusion when reranking is disabled.
|
||||
- `haiku-rag search`, `ask`, `analyze` and `chat` can cover a configured database
|
||||
set. Commands that access one database select it with `--db-name NAME` or
|
||||
`--db PATH`.
|
||||
- Citation resolution rejects chunk IDs shared by multiple selected databases
|
||||
with `AmbiguousCitationError`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- `client.chunk()` and `client.embedder` work on a client covering `lancedb.databases`: an embedder is a function of configuration, so the client builds one on first use and closes it on teardown. Operations that need one database (`create_document`, `import_document(s)`, `create_document_from_source`, `update_document`, `delete_document`, `rebuild_database`, `vacuum`, `visualize_chunk`, `close`) raise `AmbiguousDatabaseError` naming the databases covered, instead of `AttributeError`.
|
||||
- The chat TUI's document filter selects documents by id and names each document's database, instead of matching the displayed title or URI as a substring across every database.
|
||||
- `client.chunk()` and `client.embedder` work when the client covers several
|
||||
databases. Operations that require one database raise `AmbiguousDatabaseError`.
|
||||
- The chat document filter selects by document ID and shows each document's
|
||||
database.
|
||||
- `haiku-rag` prints the message and exits when the configured embedder does not match the database, instead of raising a traceback.
|
||||
- A capability built without a client opens the databases the configuration places — `lancedb.uri` or the whole `lancedb.databases` set — instead of the default under `storage.data_dir`.
|
||||
- A `lancedb.uri` with no scheme is a local path, as it already is in `lancedb.databases`: `haiku-rag init` creates it and every command that opens an existing database requires it to exist, where a missing path was opened as object storage and became an empty database. `--db PATH` overrides `lancedb.uri`.
|
||||
- Capabilities created without a client now honor `lancedb.uri` and
|
||||
`lancedb.databases`.
|
||||
- A `lancedb.uri` without a scheme is treated as a local path. `--db PATH`
|
||||
overrides it.
|
||||
|
||||
## [0.78.0] - 2026-08-24
|
||||
|
||||
|
|
|
|||
|
|
@ -24,7 +24,7 @@ Built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/
|
|||
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
||||
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
||||
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, Cohere, LM Studio, vLLM (multimodal via `multimodal: true` on vLLM/VoyageAI/Cohere). QA: any model supported by Pydantic AI
|
||||
- **Several databases** — Name databases in `lancedb.databases` and search, ask, analyze or chat across them at once, with each result and citation carrying the database it came from
|
||||
- **Multi-database search** — Search, ask, analyze, or chat across named databases with source attribution on results and citations
|
||||
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
||||
- **CLI & Python API** — Full functionality from command line or code
|
||||
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
||||
|
|
|
|||
|
|
@ -143,18 +143,17 @@ Capabilities use a plain `state: dict[str, Any]` attribute on agent dependencies
|
|||
|
||||
Applications serving AG-UI should adapt the agent with Pydantic AI's `AGUIAdapter`. Native model and tool events require no haiku.rag-specific bridge.
|
||||
|
||||
## Which databases a capability covers
|
||||
## Database Selection
|
||||
|
||||
Both factories resolve this once, in order:
|
||||
RAG and analysis capabilities select databases in this order:
|
||||
|
||||
1. The `db_path` argument, which covers that one database.
|
||||
2. `HAIKU_RAG_DB`, the same way.
|
||||
3. [`lancedb.databases`](../configuration/storage.md#several-databases), covering the
|
||||
whole configured set. A capability covering several says so in its instructions, so
|
||||
the model can attribute evidence to one while it answers.
|
||||
1. The `db_path` argument.
|
||||
2. `HAIKU_RAG_DB`.
|
||||
3. [`lancedb.databases`](../configuration/storage.md#multiple-databases), which
|
||||
selects the full configured set.
|
||||
4. [`lancedb.uri`](../configuration/storage.md#changing-the-default-database-path),
|
||||
covering the one database it places.
|
||||
which selects one database.
|
||||
5. `config.storage.data_dir / "haiku.rag.lancedb"`.
|
||||
|
||||
Passing a live client through `rag=` overrides all of it: the capability reads what that
|
||||
client covers, and never closes it.
|
||||
Passing a client through `rag=` bypasses this selection. The capability uses the
|
||||
databases covered by that client and does not close it.
|
||||
|
|
|
|||
12
docs/cli.md
12
docs/cli.md
|
|
@ -20,15 +20,15 @@ The `haiku-rag` CLI provides complete document management functionality.
|
|||
haiku-rag --config /path/to/config.yaml list
|
||||
haiku-rag --config /path/to/config.yaml list --db /path/to/custom.db
|
||||
haiku-rag --read-only search "query"
|
||||
haiku-rag --db-name medic list
|
||||
haiku-rag --db-name papers list
|
||||
haiku-rag add -h
|
||||
```
|
||||
|
||||
With `lancedb.databases` configured, `search`, `ask`, `analyze` and `chat`
|
||||
cover every database in it. `settings`, `init-config` and `download-models`
|
||||
open no database at all. Every other command works on one, named with
|
||||
`--db-name` or `--db`. See
|
||||
[Several Databases](configuration/storage.md#several-databases).
|
||||
With `lancedb.databases` configured, `search`, `ask`, `analyze`, and `chat`
|
||||
use the full set by default. Select one database for other commands with
|
||||
`--db-name` or `--db`. `settings`, `init-config`, and `download-models` do
|
||||
not open a database. See
|
||||
[Multiple Databases](configuration/storage.md#multiple-databases).
|
||||
|
||||
## Document Management
|
||||
|
||||
|
|
|
|||
|
|
@ -80,7 +80,7 @@ lancedb:
|
|||
An explicit `--db PATH` overrides `lancedb.uri` for that invocation.
|
||||
|
||||
This places one database without naming it. Its `source` is `None` in search
|
||||
results, citations and documents, since only [`lancedb.databases`](#several-databases)
|
||||
results, citations and documents, since only [`lancedb.databases`](#multiple-databases)
|
||||
assigns the names that carry provenance. A path here changes where the database
|
||||
lives, not what it is called.
|
||||
|
||||
|
|
@ -198,132 +198,129 @@ The recommended layout for production is "different buckets, same account, separ
|
|||
|
||||
Each process picks up its own credentials from the AWS default chain (env vars, IAM instance role, AWS profile), so no credentials are hard-coded in the configuration files.
|
||||
|
||||
## Several Databases
|
||||
## Multiple Databases
|
||||
|
||||
`lancedb.databases` maps a name to a location, for searching several databases at
|
||||
once:
|
||||
Use `lancedb.databases` to name local or remote databases that should be searched
|
||||
together:
|
||||
|
||||
```yaml
|
||||
lancedb:
|
||||
databases:
|
||||
medic: s3://my-bucket/medic.lancedb
|
||||
st: s3://my-bucket/st.lancedb
|
||||
local: /data/notes.lancedb
|
||||
papers: s3://my-bucket/papers.lancedb
|
||||
wiki: s3://my-bucket/wiki.lancedb
|
||||
notes: /data/notes.lancedb
|
||||
```
|
||||
|
||||
A location is a URI or a local path. `databases` and `uri` are mutually
|
||||
exclusive, and setting both fails validation.
|
||||
A location can be a URI or local path. `databases` and `uri` are mutually
|
||||
exclusive.
|
||||
|
||||
Results, citations, model input and errors opening a named database carry the
|
||||
configured name rather than the location, so a path or a bucket does not reach a
|
||||
trace or a model. Commands that report on a database — `info`, `init`, `tag` —
|
||||
print its location, as does an error about a path.
|
||||
Results, documents, citations, and model context use the configured name as
|
||||
`source`. Commands such as `info` and path-related errors still show locations.
|
||||
|
||||
Every database in the set is opened with the same embedding configuration. A
|
||||
different `vector_dim` raises `ConfigMismatchError` on open. A different provider
|
||||
or model name at the same dimension is a warning on a read-only open, since the
|
||||
same model served by another stack is spelled differently, and raises on a
|
||||
writable one.
|
||||
All databases in a search must use compatible embeddings because the query is
|
||||
embedded once. A dimension mismatch raises `ConfigMismatchError`. A provider or
|
||||
model-name mismatch at the same dimension warns in read-only mode and raises in
|
||||
writable mode.
|
||||
|
||||
Searching embeds the query once for the whole selection, so the databases
|
||||
searched together must have been written with the same embedder. Two that
|
||||
disagree with each other raise `ConfigMismatchError` naming both, whatever the
|
||||
configuration says.
|
||||
### Search and Provenance
|
||||
|
||||
### Searching a set
|
||||
|
||||
`search`, `ask` and `analyze` cover the whole set, or the subset named by
|
||||
`sources`:
|
||||
`search`, `ask`, and `analyze` use the full set by default. Pass `sources` to
|
||||
select a subset:
|
||||
|
||||
```python
|
||||
results = await client.search("query") # every database
|
||||
results = await client.search("query", sources=["medic"]) # one of them
|
||||
results = await client.search("query", sources=["papers"]) # one of them
|
||||
```
|
||||
|
||||
Candidates from each database are fused into one ranked list, by the configured
|
||||
reranker where there is one and by reciprocal rank fusion otherwise. Each result
|
||||
carries `source`, the name of the database it came from, and so does each
|
||||
citation. A document from `list_documents`, `get_document_by_id`,
|
||||
`get_document_by_uri` or `resolve_document` carries it too, and a database
|
||||
named in `lancedb.databases` keeps that name even when it is the only one a
|
||||
client covers. Only a database placed by `lancedb.uri`, which names none, has no
|
||||
name to carry. The CLI does not print the name of a single database it was told
|
||||
to use, since the caller just named it.
|
||||
Candidates are combined into one ranked list with the configured reranker, or
|
||||
with reciprocal rank fusion when reranking is disabled. `SearchResult.source`,
|
||||
`Citation.source`, and `Document.source` contain the configured database name.
|
||||
The name is retained when a client covers only one named database. Databases
|
||||
configured through `lancedb.uri` are unnamed, so their `source` is `None`.
|
||||
|
||||
Chunk ids are unique within a database and say nothing across them, so a database
|
||||
copied from another holds the same ids. Results are told apart by the database
|
||||
and the id together. A chunk id held by two of the databases searched cannot be
|
||||
cited: `resolve_citations` raises `AmbiguousCitationError`, and the capability
|
||||
asks the model for other evidence instead.
|
||||
The CLI labels results and citations only when the operation spans multiple
|
||||
databases. A command already narrowed with `--db-name` does not repeat the name
|
||||
on every result.
|
||||
|
||||
Document ids behave the same way, and two places treat a collision differently.
|
||||
The analysis sandbox mounts one document per id and refuses a set where two
|
||||
databases claim one, since the mount has a single path per id. The chat document
|
||||
filter does not: it selects by document id and applies `id IN (...)` to every
|
||||
covered database, so selecting an id that exists in copies of a database matches
|
||||
the document in each of them. Independently built databases use UUID document
|
||||
ids and do not collide.
|
||||
#### Duplicate IDs
|
||||
|
||||
**Configure a reranker when searching several databases.** Reciprocal rank fusion
|
||||
compares ranks, not scores, so every database contributes its own best matches
|
||||
whether or not they are relevant to the question, and results from databases
|
||||
holding nothing relevant displace better ones. A reranker scores the whole union
|
||||
instead, which removes the effect. Measured on one corpus split three ways, with
|
||||
the same queries: retrieval MAP 0.9914 with a reranker against 0.9918 for the
|
||||
same corpus in a single database, and 0.6044 without one against 0.9798. The cost
|
||||
is that a reranker scores candidates in proportion to the number of databases.
|
||||
IDs are unique within a database, not across databases. Copies of a database
|
||||
therefore retain the same IDs. Citation resolution raises
|
||||
`AmbiguousCitationError` when a cited chunk ID exists in more than one selected
|
||||
database; a shared ID that nothing cites is ignored. The analysis sandbox
|
||||
rejects shared document IDs because its mount path is `/documents/{id}/`.
|
||||
|
||||
Without one, consider raising `search.limit` with the number of databases
|
||||
searched. Each contributes its own best matches to a list that is then truncated
|
||||
at the limit, so with three full rankings and a limit of 5 any one database may
|
||||
contribute only one or two results. Raising the limit raises what the caller and
|
||||
the model receive, since without a reranker nothing is over-fetched to absorb
|
||||
it.
|
||||
The chat document filter selects by document ID and applies `id IN (...)` to
|
||||
every covered database, so selecting an ID that copies share matches the
|
||||
document in each of them.
|
||||
|
||||
Creating names a database: `create=True` on a client covering the set raises
|
||||
`AmbiguousDatabaseError`, and `HaikuRAG(config=config, create=True,
|
||||
sources=["name"])` creates that one.
|
||||
#### Ranking
|
||||
|
||||
Converting, chunking and title generation are functions of the configuration
|
||||
rather than of a database, so they work on a client covering the set. Writing,
|
||||
rebuilding and vacuuming name one database: asking a set-covering client raises
|
||||
`AmbiguousDatabaseError`, and `client.clients_for(["name"])` returns a client for
|
||||
one of them, writable when the covering client is.
|
||||
Configure a reranker when searching multiple databases. Reciprocal rank fusion
|
||||
compares positions rather than scores, so each database contributes top-ranked
|
||||
results even when another database has stronger matches. A reranker scores the
|
||||
combined candidate set directly.
|
||||
|
||||
A database that cannot be opened fails the whole query and is named in the error.
|
||||
A result set silently missing one of the databases asked for cannot be told apart
|
||||
from a complete one.
|
||||
In a 3,045-query evaluation over a corpus split across three databases,
|
||||
reranking produced retrieval MAP 0.9914, compared with 0.9918 for the same corpus
|
||||
in one database. Without a reranker, MAP was 0.6044, compared with 0.9798 in one
|
||||
database. Reranking cost grows with the number of databases because each
|
||||
contributes candidates.
|
||||
|
||||
### How commands treat the set
|
||||
Without a reranker, consider increasing `search.limit` with the number of
|
||||
databases. With three complete rankings and a limit of 5, a database may
|
||||
contribute only one or two results. A higher limit also sends more results to
|
||||
the caller and model.
|
||||
|
||||
Commands fall into three groups:
|
||||
If any selected database fails to open, the operation fails and identifies that
|
||||
database.
|
||||
|
||||
- **Set-capable**: `search`, `ask`, `analyze` and `chat` cover the whole
|
||||
### Python Operations
|
||||
|
||||
Creating, writing, rebuilding, and vacuuming require one database. Calling these
|
||||
operations on a client that covers several raises `AmbiguousDatabaseError`.
|
||||
Select one at creation time or obtain a single-database client:
|
||||
|
||||
```python
|
||||
async with HaikuRAG(config=config, create=True, sources=["papers"]) as papers:
|
||||
...
|
||||
|
||||
async with HaikuRAG(config=config) as client:
|
||||
papers = (await client.clients_for(["papers"]))[0]
|
||||
```
|
||||
|
||||
Conversion, chunking, and title generation do not access a database and remain
|
||||
available on a multi-database client.
|
||||
|
||||
### CLI Commands
|
||||
|
||||
Commands use database sets as follows:
|
||||
|
||||
- **Set-capable**: `search`, `ask`, `analyze`, and `chat` use the full
|
||||
configured set, or the subset named by `--db-name`.
|
||||
- **Config-only**: `settings`, `init-config` and `download-models` open no
|
||||
database, so the set is irrelevant to them.
|
||||
- **Config-only**: `settings`, `init-config`, and `download-models` do not open
|
||||
a database.
|
||||
- **Single-database**: everything else — document writes, `rebuild`, `vacuum`,
|
||||
`migrate`, `init`, `info`, `history`, `tag`, `doctor`, `list`, `inspect`,
|
||||
`visualize` and `mcp` — works on one database, named with the global
|
||||
`visualize`, and `mcp` — works on one database, selected with the global
|
||||
`--db-name` option.
|
||||
|
||||
```bash
|
||||
haiku-rag search "query" # every configured database
|
||||
haiku-rag --db-name medic list # one of them
|
||||
haiku-rag --db-name medic migrate
|
||||
haiku-rag search "query" # every configured database
|
||||
haiku-rag --db-name papers list # one of them
|
||||
haiku-rag --db-name papers migrate
|
||||
```
|
||||
|
||||
`--db-name` takes a name from `lancedb.databases`, which is how a database
|
||||
behind a URI is reached. `--db` takes a path, and overrides the configured
|
||||
location with that one database. A single-database command given neither fails
|
||||
rather than choosing for you, unless `lancedb.databases` names exactly one: a
|
||||
set of one is unambiguous and is used, keeping its configured name.
|
||||
`--db-name` selects an entry from `lancedb.databases`, including remote entries.
|
||||
`--db` selects a local path and overrides the configured location. A
|
||||
single-database command requires one of these options when multiple databases are
|
||||
configured. A configured set of one is selected automatically.
|
||||
|
||||
Each database is created, migrated and vacuumed on its own:
|
||||
|
||||
```bash
|
||||
haiku-rag --db-name medic init
|
||||
haiku-rag --db-name st init
|
||||
haiku-rag --db-name papers init
|
||||
haiku-rag --db-name wiki init
|
||||
```
|
||||
|
||||
## Vector Indexing
|
||||
|
|
|
|||
|
|
@ -230,56 +230,51 @@ for result in results:
|
|||
print(f"Document Title: {result.document_title}") # when available
|
||||
```
|
||||
|
||||
### Searching Several Databases
|
||||
### Searching Multiple Databases
|
||||
|
||||
With [`lancedb.databases`](configuration/storage.md#several-databases)
|
||||
configured, a client covers every database in it. `sources` narrows a call to
|
||||
some of them, and each result names the database it came from:
|
||||
With [`lancedb.databases`](configuration/storage.md#multiple-databases)
|
||||
configured, a client covers the full set. Use `sources` to select a subset.
|
||||
Each result includes its database name:
|
||||
|
||||
```python
|
||||
results = await client.search("machine learning") # all of them
|
||||
results = await client.search("machine learning", sources=["medic"]) # one of them
|
||||
results = await client.search("machine learning", sources=["papers"]) # one of them
|
||||
|
||||
for result in results:
|
||||
print(f"{result.source}: {result.content}")
|
||||
```
|
||||
|
||||
`ask` and `analyze` take `sources` too, and every citation carries the database
|
||||
it was drawn from:
|
||||
`ask` and `analyze` also accept `sources`. Citations include the database name:
|
||||
|
||||
```python
|
||||
answer, citations = await client.ask("What changed?", sources=["medic", "st"])
|
||||
answer, citations = await client.ask("What changed?", sources=["papers", "wiki"])
|
||||
for cite in citations:
|
||||
print(f"[{cite.source}] {cite.document_title or cite.document_uri}")
|
||||
|
||||
result = await client.analyze("How many documents mention it?", sources=["medic"])
|
||||
result = await client.analyze("How many documents mention it?", sources=["papers"])
|
||||
```
|
||||
|
||||
A question scoped to some databases can only cite those, and the analysis
|
||||
sandbox mounts only their documents.
|
||||
A scoped question can cite only the selected databases. Analysis mounts only
|
||||
their documents.
|
||||
|
||||
`sources=None` covers every database the client covers. `sources=[]` covers
|
||||
none: `search` returns no results, and `ask` and `analyze` run with no evidence
|
||||
from any database.
|
||||
|
||||
#### Asking a client what it covers
|
||||
#### Inspecting the client scope
|
||||
|
||||
```python
|
||||
client.covers_multiple # True while reading more than one database
|
||||
client.source_names # the configured names covered, in configured order
|
||||
client.source # the one name, or None while covering a set
|
||||
client.covers_multiple # whether the client covers more than one database
|
||||
client.source_names # configured names, in order
|
||||
client.source # one configured name, or None for a set or unnamed database
|
||||
|
||||
owner = await client.reader_for("medic") # the client reading that database
|
||||
medic, st = await client.clients_for(["medic", "st"])
|
||||
owner = await client.reader_for("papers") # the client reading that database
|
||||
papers, wiki = await client.clients_for(["papers", "wiki"])
|
||||
```
|
||||
|
||||
`clients_for` opens the databases it names, on first use rather than at entry,
|
||||
and returns a client for each. Those clients borrow their databases from the
|
||||
covering one: they are valid only while it is open, and closing or entering one
|
||||
leaves its database alone. The covering client closes them all on teardown.
|
||||
|
||||
A borrowed client reads one database, so it has the `store` and repositories a
|
||||
covering client cannot have, and it is writable when the covering client is.
|
||||
`reader_for` and `clients_for` open databases lazily and return borrowed clients.
|
||||
They remain valid while the covering client is open and inherit its read-only
|
||||
mode. The covering client owns and closes their database sessions.
|
||||
|
||||
### Filtering Search Results
|
||||
|
||||
|
|
|
|||
|
|
@ -90,20 +90,18 @@ By default, evaluation databases are stored in the haiku.rag data directory:
|
|||
|
||||
You can override this with the `--db` option.
|
||||
|
||||
### Evaluating over several databases
|
||||
### Evaluating over Multiple Databases
|
||||
|
||||
With [`lancedb.databases`](https://ggozad.github.io/haiku.rag/configuration/storage/#several-databases)
|
||||
configured, `evaluations run <dataset> --skip-db` benchmarks the configured set:
|
||||
retrieval, QA and live conversations all search every database in it, and each
|
||||
result and citation names the one it came from. A mapping of one follows the same
|
||||
path and keeps its configured name.
|
||||
With [`lancedb.databases`](https://ggozad.github.io/haiku.rag/configuration/storage/#multiple-databases)
|
||||
configured, `evaluations run <dataset> --skip-db` benchmarks the full set.
|
||||
Retrieval, QA, and live conversations preserve the database name on results and
|
||||
citations. A configured set of one follows the same path and retains its name.
|
||||
|
||||
Population is not part of that. It writes one database, so it needs a path:
|
||||
Population writes one database and therefore requires `--db`:
|
||||
|
||||
```bash
|
||||
evaluations run hotpotqa --db /path/to/one.lancedb # populate, then benchmark
|
||||
evaluations run hotpotqa --skip-db # benchmark the configured set
|
||||
```
|
||||
|
||||
A `--db` path names one database and overrides the configured set for the whole
|
||||
run, population and benchmarks alike.
|
||||
`--db` overrides the configured set for both population and benchmarks.
|
||||
|
|
|
|||
|
|
@ -25,8 +25,8 @@ STATE_NAMESPACE = "analysis"
|
|||
_CAPABILITY_ID = "haiku-rag-analysis"
|
||||
_TOOL_NAMES = frozenset({"analysis_search", "analysis_execute_code", "analysis_cite"})
|
||||
_instructions_path = Path(__file__).parent / "instructions" / "analysis.md"
|
||||
_several_databases_path = (
|
||||
Path(__file__).parent / "instructions" / "analysis_several_databases.md"
|
||||
_multiple_databases_path = (
|
||||
Path(__file__).parent / "instructions" / "analysis_multiple_databases.md"
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -44,10 +44,10 @@ def instructions() -> str:
|
|||
|
||||
|
||||
@cache
|
||||
def several_databases_instructions() -> str:
|
||||
def multiple_databases_instructions() -> str:
|
||||
"""Appended only where the capability covers several databases, so a single
|
||||
database is instructed exactly as it was before they could be named."""
|
||||
return _several_databases_path.read_text().rstrip()
|
||||
return _multiple_databases_path.read_text().rstrip()
|
||||
|
||||
|
||||
def _recovery_hint(stderr: str) -> str:
|
||||
|
|
@ -217,7 +217,7 @@ def create_capability(
|
|||
# A lent client covers what it covers; otherwise the scope says.
|
||||
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
||||
if several:
|
||||
instruction_text += several_databases_instructions()
|
||||
instruction_text += multiple_databases_instructions()
|
||||
return AnalysisCapability(
|
||||
scope=scope,
|
||||
config=config,
|
||||
|
|
|
|||
|
|
@ -1,5 +1,5 @@
|
|||
|
||||
## Several databases
|
||||
## Multiple databases
|
||||
|
||||
The corpus spans several databases, and each interface names them differently:
|
||||
|
||||
|
|
@ -1,5 +1,5 @@
|
|||
|
||||
## Several databases
|
||||
## Multiple databases
|
||||
|
||||
The corpus spans several databases. Each result carries a `Database:` line naming
|
||||
the configured database it came from. Say which database an answer draws on when
|
||||
|
|
@ -29,8 +29,8 @@ STATE_NAMESPACE = "rag"
|
|||
_CAPABILITY_ID = "haiku-rag"
|
||||
_TOOL_NAMES = frozenset({"rag_search", "rag_cite"})
|
||||
_instructions_path = Path(__file__).parent / "instructions" / "rag.md"
|
||||
_several_databases_path = (
|
||||
Path(__file__).parent / "instructions" / "rag_several_databases.md"
|
||||
_multiple_databases_path = (
|
||||
Path(__file__).parent / "instructions" / "rag_multiple_databases.md"
|
||||
)
|
||||
|
||||
|
||||
|
|
@ -44,10 +44,10 @@ def instructions() -> str:
|
|||
|
||||
|
||||
@cache
|
||||
def several_databases_instructions() -> str:
|
||||
def multiple_databases_instructions() -> str:
|
||||
"""Appended only where the capability covers several databases, so a single
|
||||
database is instructed exactly as it was before they could be named."""
|
||||
return _several_databases_path.read_text().rstrip()
|
||||
return _multiple_databases_path.read_text().rstrip()
|
||||
|
||||
|
||||
@dataclass
|
||||
|
|
@ -120,7 +120,7 @@ def create_capability(
|
|||
# A lent client covers what it covers; otherwise the scope says.
|
||||
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
||||
if several:
|
||||
instruction_text += several_databases_instructions()
|
||||
instruction_text += multiple_databases_instructions()
|
||||
return RAGCapability(
|
||||
scope=scope,
|
||||
config=config,
|
||||
|
|
|
|||
Loading…
Reference in a new issue