Commit graph

18 commits

Author SHA1 Message Date
Yiorgis Gozadinos
6f976ef2a9
Share the LanceDB session across connections
Every `Store` built its own connection with its own caches and discarded them on
close, so the index a vector query loads was refetched by the next connection.
On object storage that first fetch dominates: measured on a ~500k-chunk 2560-dim
corpus over a ~200ms link, the first query cost ~41s and the second ~3s, and a
new connection reusing the session cost ~7s instead of ~47s.

`connect_lancedb` now passes a process-wide session, keyed on the configured
cache sizes so a caller asking for different sizes gets its own.

Also sets `read_consistency_interval`, defaulting to 30s. It was None, meaning a
connection never re-checked for other processes' writes. Per-call connections hid
that; a shared session makes connections long-lived enough for a reader to go
stale against the ingester.

All three settings reject negatives at the config boundary. A negative cache size
raises OverflowError and a negative interval panics inside Lance, so neither is
catchable further in. Zero stays valid for both: no cache, and check on every
read.

The routing tests now assert the kwargs they care about rather than the full call
signature, since every connection carries the two new kwargs.
2026-08-18 14:58:17 +03:00
Yiorgis Gozadinos
749752820c
Document vacuum memory requirements 2026-07-30 14:32:40 +03:00
Yiorgis Gozadinos
6c04dd5881
docs 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
7ea61a7b10
drop the old monitor, rename serve→mcp, add e2e tests 2026-05-26 11:42:41 +03:00
Yiorgis Gozadinos
0ee00334c7
drop prose emdashes and semicolons; minor fixes 2026-05-20 14:27:16 +03:00
Yiorgis Gozadinos
19e2be003e
docs: add chat page, restructure nav, reorder sections 2026-05-20 14:27:16 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow 2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
6f6ebf27c8
Document s3 storage & monitoring 2026-05-11 11:29:56 +03:00
Yiorgis Gozadinos
ff8ad0879c
fix context expansion: respect section boundaries, remove max_context_items 2026-04-20 15:18:16 +03:00
Yiorgis Gozadinos
b6113bf8ab
replace fixed-radius expansion with section-bounded algorithm
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
367accb814
Replace MinIO with SeaweedFS for S3 integration tests 2026-04-08 14:19:32 +03:00
Yiorgis Gozadinos
5e7a4ebae5
Update docs 2026-04-08 14:19:32 +03:00
Yiorgis Gozadinos
8206498af8
Documentation updates 2025-12-10 13:02:49 +02:00
Yiorgis Gozadinos
2161727716
Add auto_vacuum config option to control automatic vacuuming 2025-12-10 11:42:14 +02:00
Yiorgis Gozadinos
52acdcb56a
Update docs, include tuning document 2025-12-09 12:55:28 +02:00
Yiorgis Gozadinos
802f058205
Move search related config under SearchConfig 2025-12-09 12:28:28 +02:00
Yiorgis Gozadinos
4da50873bd
Make search limit configurable 2025-12-09 11:46:43 +02:00
Yiorgis Gozadinos
5612ecaee7
Fix documentation configuration index 2025-11-26 10:24:53 +02:00
Renamed from docs/storage.md (Browse further)