Adds the 0.75.0 upgrade, which brings a pre-existing database up to the index
set `_init_tables` now creates. It rewrites no table data, so unlike the earlier
data migrations its cost is the index builds alone, each of which reads the
column it indexes.
`ensure_indexes` ensures an index of the declared *type* covers each declared
column, rather than checking that the column is indexed at all. The distinction
is what makes it safe to run against a database of unknown provenance:
- A wrong-typed index no longer satisfies the check. A BTree on `label` covers
the column while losing the low-cardinality equality lookup the Bitmap is for.
- Nothing is dropped or converted away from. Two index types over one column can
be deliberate, serving different query shapes, so an index this function did
not declare survives even on a column it does. The one thing it overwrites is
an index at LanceDB's default name, `{column}_idx`, which is the name it
creates itself.
- A column already carrying the declared type is skipped, so a database with the
full set migrates instantly rather than re-sorting every indexed column.
- Undeclared columns are untouched, so a vector index on `chunks` survives.
It returns the columns it acted on, because a change is not always visible from
outside: adding a Bitmap beside an existing BTree leaves the column indexed
before and after.
The version bump to 0.75.0 is required, not incidental: `_set_initial_version`
stamps a new database with the installed package version, so a migration
numbered above it would be pending the moment the database was created.
`test_client_update_document_replaces_rows_with_bounded_versions` turns
auto_vacuum off. Indexing `documents` means a background vacuum now has an index
to maintain on that table, so `optimize()` writes a version where it previously
had nothing to do, and it landed inside the window the test measures. The
document update itself is still one version, so the bound stays exact.
0.0.19 runs sandboxed code in a subprocess worker pool (AsyncMonty /
AsyncMontySession) and drops MontyRepl. The sandbox checks out a session,
drives it with feed_run, and closes it to return the worker; close() is
now async. Worker crashes surface as a failed SandboxResult. The document
VFS (OSAccess/CallbackFile/MemoryFile) is unchanged.
Migrate the ingester queue storage from raw aiosqlite to SQLAlchemy Core
async. The backend is chosen by ingester.queue.dburi: a SQLAlchemy async
URL points the queue at a database server, and SQLite remains the default
when unset. The Postgres path claims jobs with FOR UPDATE SKIP LOCKED so
multiple ingester processes can share one queue; SQLite caps the pool to a
single connection to keep the select-then-update claim atomic.
