Correct the multi-database prose
`resolve_citations` documented the behaviour it now rejects: a shared chunk id resolving to whichever result came last. The storage page ran two embedding checks together as if the second explained the first. They are separate: each database against the configuration on open, and the databases in a selection against each other, which raises in read-only mode too and does not apply to full-text search. `several` becomes `multiple` where it names the feature, matching the docs and `covers_multiple`.
This commit is contained in:
parent
c8a9b5df50
commit
746b663a2f
9 changed files with 29 additions and 25 deletions
|
|
@ -16,7 +16,7 @@
|
||||||
|
|
||||||
### Fixed
|
### Fixed
|
||||||
|
|
||||||
- `client.chunk()` and `client.embedder` work when the client covers several
|
- `client.chunk()` and `client.embedder` work when the client covers multiple
|
||||||
databases. Operations that require one database raise `AmbiguousDatabaseError`.
|
databases. Operations that require one database raise `AmbiguousDatabaseError`.
|
||||||
- The chat document filter selects by document ID and shows each document's
|
- The chat document filter selects by document ID and shows each document's
|
||||||
database.
|
database.
|
||||||
|
|
@ -25,6 +25,7 @@
|
||||||
`lancedb.databases`.
|
`lancedb.databases`.
|
||||||
- A `lancedb.uri` without a scheme is treated as a local path. `--db PATH`
|
- A `lancedb.uri` without a scheme is treated as a local path. `--db PATH`
|
||||||
overrides it.
|
overrides it.
|
||||||
|
- Inspector search results mark truncated previews with an ellipsis.
|
||||||
|
|
||||||
## [0.78.0] - 2026-08-24
|
## [0.78.0] - 2026-08-24
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -88,7 +88,7 @@ lancedb:
|
||||||
uri: "" # Empty for local, or db://, s3://, az://, gs://
|
uri: "" # Empty for local, or db://, s3://, az://, gs://
|
||||||
api_key: ""
|
api_key: ""
|
||||||
region: ""
|
region: ""
|
||||||
databases: {} # Name-to-location map to search several at once, instead of uri
|
databases: {} # Name-to-location map to search multiple at once, instead of uri
|
||||||
|
|
||||||
embeddings:
|
embeddings:
|
||||||
model:
|
model:
|
||||||
|
|
|
||||||
|
|
@ -217,10 +217,18 @@ exclusive.
|
||||||
Results, documents, citations, and model context use the configured name as
|
Results, documents, citations, and model context use the configured name as
|
||||||
`source`. Commands such as `info` and path-related errors still show locations.
|
`source`. Commands such as `info` and path-related errors still show locations.
|
||||||
|
|
||||||
All databases in a search must use compatible embeddings because the query is
|
Embedding compatibility is checked against two different things.
|
||||||
embedded once. A dimension mismatch raises `ConfigMismatchError`. A provider or
|
|
||||||
model-name mismatch at the same dimension warns in read-only mode and raises in
|
On open, each database is compared with the current configuration. A dimension
|
||||||
writable mode.
|
mismatch raises `ConfigMismatchError`. A provider or model-name mismatch at the
|
||||||
|
same dimension warns in read-only mode and raises in writable mode.
|
||||||
|
|
||||||
|
Across a selection, the databases are compared with each other. Vector and
|
||||||
|
hybrid search embed the query once, so every database answering it must record
|
||||||
|
the same provider, model, and dimension. A disagreement raises
|
||||||
|
`ConfigMismatchError` in read-only mode as well. Only the databases searched
|
||||||
|
together have to agree, and full-text search embeds nothing, so it is
|
||||||
|
unaffected.
|
||||||
|
|
||||||
### Search and Provenance
|
### Search and Provenance
|
||||||
|
|
||||||
|
|
@ -278,7 +286,7 @@ database.
|
||||||
### Python Operations
|
### Python Operations
|
||||||
|
|
||||||
Creating, writing, rebuilding, and vacuuming require one database. Calling these
|
Creating, writing, rebuilding, and vacuuming require one database. Calling these
|
||||||
operations on a client that covers several raises `AmbiguousDatabaseError`.
|
operations on a client that covers multiple raises `AmbiguousDatabaseError`.
|
||||||
Select one at creation time or obtain a single-database client:
|
Select one at creation time or obtain a single-database client:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
|
|
|
||||||
|
|
@ -45,8 +45,7 @@ def instructions() -> str:
|
||||||
|
|
||||||
@cache
|
@cache
|
||||||
def multiple_databases_instructions() -> str:
|
def multiple_databases_instructions() -> str:
|
||||||
"""Appended only where the capability covers several databases, so a single
|
"""Appended only where the capability covers multiple databases."""
|
||||||
database is instructed exactly as it was before they could be named."""
|
|
||||||
return _multiple_databases_path.read_text().rstrip()
|
return _multiple_databases_path.read_text().rstrip()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -215,8 +214,8 @@ def create_capability(
|
||||||
scope = resolve_scope(db_path, config)
|
scope = resolve_scope(db_path, config)
|
||||||
instruction_text = instructions()
|
instruction_text = instructions()
|
||||||
# A lent client covers what it covers; otherwise the scope says.
|
# A lent client covers what it covers; otherwise the scope says.
|
||||||
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
covers_multiple = rag.covers_multiple if rag is not None else scope.covers_multiple
|
||||||
if several:
|
if covers_multiple:
|
||||||
instruction_text += multiple_databases_instructions()
|
instruction_text += multiple_databases_instructions()
|
||||||
return AnalysisCapability(
|
return AnalysisCapability(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
|
|
|
||||||
|
|
@ -45,8 +45,7 @@ def instructions() -> str:
|
||||||
|
|
||||||
@cache
|
@cache
|
||||||
def multiple_databases_instructions() -> str:
|
def multiple_databases_instructions() -> str:
|
||||||
"""Appended only where the capability covers several databases, so a single
|
"""Appended only where the capability covers multiple databases."""
|
||||||
database is instructed exactly as it was before they could be named."""
|
|
||||||
return _multiple_databases_path.read_text().rstrip()
|
return _multiple_databases_path.read_text().rstrip()
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -118,8 +117,8 @@ def create_capability(
|
||||||
scope = resolve_scope(db_path, config)
|
scope = resolve_scope(db_path, config)
|
||||||
instruction_text = instructions()
|
instruction_text = instructions()
|
||||||
# A lent client covers what it covers; otherwise the scope says.
|
# A lent client covers what it covers; otherwise the scope says.
|
||||||
several = rag.covers_multiple if rag is not None else scope.covers_multiple
|
covers_multiple = rag.covers_multiple if rag is not None else scope.covers_multiple
|
||||||
if several:
|
if covers_multiple:
|
||||||
instruction_text += multiple_databases_instructions()
|
instruction_text += multiple_databases_instructions()
|
||||||
return RAGCapability(
|
return RAGCapability(
|
||||||
scope=scope,
|
scope=scope,
|
||||||
|
|
|
||||||
|
|
@ -65,11 +65,11 @@ _db_name: str | None = None
|
||||||
def create_app(db: Path | None = None, *, covers_set: bool = False) -> "HaikuRAGApp":
|
def create_app(db: Path | None = None, *, covers_set: bool = False) -> "HaikuRAGApp":
|
||||||
"""The application for a command, on the database(s) it works on.
|
"""The application for a command, on the database(s) it works on.
|
||||||
|
|
||||||
`covers_set` is the command declaring that it can read several: `search`,
|
`covers_set` is the command declaring that it can read multiple: `search`,
|
||||||
`ask`, `analyze` and `chat` can, and everything else names one.
|
`ask`, `analyze` and `chat` can, and everything else names one.
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
AmbiguousDatabaseError: several databases are configured and this
|
AmbiguousDatabaseError: multiple databases are configured and this
|
||||||
command works on one, without `--db` or `--db-name` naming which.
|
command works on one, without `--db` or `--db-name` naming which.
|
||||||
"""
|
"""
|
||||||
from haiku.rag.app import HaikuRAGApp
|
from haiku.rag.app import HaikuRAGApp
|
||||||
|
|
|
||||||
|
|
@ -102,7 +102,7 @@ class LanceDBConfig(ConfigModel):
|
||||||
The cache sizes are per process, since the session is shared across
|
The cache sizes are per process, since the session is shared across
|
||||||
connections.
|
connections.
|
||||||
|
|
||||||
`databases` maps a name to a location, for searching several at once. The
|
`databases` maps a name to a location, for searching multiple at once. The
|
||||||
name is what results and citations carry, so a location never leaves the
|
name is what results and citations carry, so a location never leaves the
|
||||||
configuration. Mutually exclusive with `uri`.
|
configuration. Mutually exclusive with `uri`.
|
||||||
"""
|
"""
|
||||||
|
|
|
||||||
|
|
@ -75,8 +75,8 @@ def resolve_citations(
|
||||||
"""Resolve chunk IDs to full Citation objects with metadata.
|
"""Resolve chunk IDs to full Citation objects with metadata.
|
||||||
|
|
||||||
Raises ``AmbiguousCitationError`` when a cited id names a chunk in more than
|
Raises ``AmbiguousCitationError`` when a cited id names a chunk in more than
|
||||||
one of the databases searched. An id held by two of them, as after copying a
|
one of the databases searched, as after copying a database. A citation
|
||||||
database, resolves to whichever result came last, attributing the answer to a
|
records the id alone, so resolving one would attribute the answer to a
|
||||||
database it may not have come from.
|
database it may not have come from.
|
||||||
"""
|
"""
|
||||||
by_id: dict[str, SearchResult] = {}
|
by_id: dict[str, SearchResult] = {}
|
||||||
|
|
|
||||||
|
|
@ -363,11 +363,8 @@ def format_citations(citations: "list[Citation]") -> str:
|
||||||
|
|
||||||
|
|
||||||
def truncated(text: str, limit: int) -> str:
|
def truncated(text: str, limit: int) -> str:
|
||||||
"""`text` cut to `limit` characters, marked where anything was dropped.
|
"""The first `limit` characters of `text`, with `…` appended when anything
|
||||||
|
was dropped. A cut result is `limit` characters plus the mark."""
|
||||||
Without the mark a clipped value reads as the value: a sentence ending
|
|
||||||
"commissioned in 1991" becomes "commissioned in 1".
|
|
||||||
"""
|
|
||||||
if len(text) <= limit:
|
if len(text) <= limit:
|
||||||
return text
|
return text
|
||||||
return text[:limit].rstrip() + "…"
|
return text[:limit].rstrip() + "…"
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue