Commit graph

141 commits

Author SHA1 Message Date
Yiorgis Gozadinos
73d04ddba5
vb 2026-08-24 16:06:03 +03:00
Yiorgis Gozadinos
152d57d6e2
Add frames evaluation dataset 2026-08-24 08:44:01 +03:00
Yiorgis Gozadinos
e9f6fea598
vb 2026-08-21 13:15:50 +03:00
Yiorgis Gozadinos
2a7c6510d0
vb 2026-08-20 15:48:23 +03:00
Yiorgis Gozadinos
8e93b639bc
Migrate existing databases to the full index set
Adds the 0.75.0 upgrade, which brings a pre-existing database up to the index
set `_init_tables` now creates. It rewrites no table data, so unlike the earlier
data migrations its cost is the index builds alone, each of which reads the
column it indexes.

`ensure_indexes` ensures an index of the declared *type* covers each declared
column, rather than checking that the column is indexed at all. The distinction
is what makes it safe to run against a database of unknown provenance:

- A wrong-typed index no longer satisfies the check. A BTree on `label` covers
  the column while losing the low-cardinality equality lookup the Bitmap is for.
- Nothing is dropped or converted away from. Two index types over one column can
  be deliberate, serving different query shapes, so an index this function did
  not declare survives even on a column it does. The one thing it overwrites is
  an index at LanceDB's default name, `{column}_idx`, which is the name it
  creates itself.
- A column already carrying the declared type is skipped, so a database with the
  full set migrates instantly rather than re-sorting every indexed column.
- Undeclared columns are untouched, so a vector index on `chunks` survives.

It returns the columns it acted on, because a change is not always visible from
outside: adding a Bitmap beside an existing BTree leaves the column indexed
before and after.

The version bump to 0.75.0 is required, not incidental: `_set_initial_version`
stamps a new database with the installed package version, so a migration
numbered above it would be pending the moment the database was created.

`test_client_update_document_replaces_rows_with_bounded_versions` turns
auto_vacuum off. Indexing `documents` means a background vacuum now has an index
to maintain on that table, so `optimize()` writes a version where it previously
had nothing to do, and it landed inside the window the test measures. The
document update itself is still one version, so the bound stays exact.
2026-08-17 16:34:58 +03:00
Yiorgis Gozadinos
d864cf8008
vb 2026-08-13 16:16:35 +03:00
Yiorgis Gozadinos
96475bba8c
vb 2026-08-06 13:55:57 +03:00
Yiorgis Gozadinos
5a023a24ad
vb 2026-07-31 15:25:28 +03:00
Yiorgis Gozadinos
adae9eff2e
vb 2026-07-30 19:37:45 +03:00
Yiorgis Gozadinos
ebd1fc7f3e
vb 2026-07-29 09:06:55 +03:00
Yiorgis Gozadinos
94e03d777a
vb 2026-07-25 15:11:54 +03:00
Yiorgis Gozadinos
acbd66afbd
vb 2026-07-24 17:32:20 +03:00
Yiorgis Gozadinos
ec78641cc4
vb 2026-07-24 15:15:20 +03:00
Yiorgis Gozadinos
9f49bdde6e
vb 2026-07-23 14:28:23 +03:00
Yiorgis Gozadinos
294ced57cd
vb 2026-07-23 10:45:48 +03:00
Yiorgis Gozadinos
4f0fb7322e
vb 2026-07-22 13:27:35 +03:00
Yiorgis Gozadinos
b5c0ea630c
vb 2026-07-16 13:50:40 +03:00
Yiorgis Gozadinos
5dd310a9b7
vb 2026-07-14 16:06:50 +03:00
Yiorgis Gozadinos
5c41f8a0ed
vb 2026-07-10 13:38:24 +03:00
Yiorgis Gozadinos
da0748916b
vb 2026-07-09 15:58:58 +03:00
Yiorgis Gozadinos
a1cf405cae
Rename document_meta identity column document_id to id 2026-07-08 16:32:55 +03:00
Yiorgis Gozadinos
9a9bea86d3
vb 2026-07-03 13:35:10 +03:00
Yiorgis Gozadinos
9a85632cf3
vb 2026-06-29 12:36:36 +03:00
Yiorgis Gozadinos
b2abdfaffb
vb 2026-06-28 10:57:26 +03:00
Yiorgis Gozadinos
c7c3f75508
vb 2026-06-27 09:59:21 +03:00
Yiorgis Gozadinos
cd1a73b389
vb 2026-06-26 17:13:03 +03:00
Yiorgis Gozadinos
197d58a602
vb 2026-06-24 18:41:05 +03:00
Yiorgis Gozadinos
ab1775dbc6
vb 2026-06-24 16:45:19 +03:00
Yiorgis Gozadinos
65f1339aa0
vb 2026-06-23 16:59:00 +03:00
Yiorgis Gozadinos
0fcf91fcf7
vb 2026-06-22 15:18:35 +03:00
Yiorgis Gozadinos
5c4b37df89
vb 2026-06-19 11:12:45 +03:00
Yiorgis Gozadinos
99011755fb
vb 2026-06-16 16:48:36 +03:00
Yiorgis Gozadinos
df8af54298
rebase from main 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
88696cc6f1
vb 2026-06-11 09:50:33 +03:00
Yiorgis Gozadinos
bdad908a60
vb 2026-06-09 15:39:14 +03:00
Yiorgis Gozadinos
46b8fa3fef
vb 2026-06-08 12:24:04 +03:00
Yiorgis Gozadinos
916b84bff4
vb 2026-06-05 12:09:57 +03:00
Yiorgis Gozadinos
e70f271ac7
vb 2026-06-04 14:13:55 +03:00
Yiorgis Gozadinos
4eae86225e
vb 2026-06-03 14:26:58 +03:00
Yiorgis Gozadinos
8dbcd4bd7a
vb, changelog 2026-06-01 18:54:22 +03:00
Yiorgis Gozadinos
62be23d130
vb 2026-05-29 11:47:12 +03:00
Yiorgis Gozadinos
3828a91c37
vb 2026-05-27 16:23:43 +03:00
Yiorgis Gozadinos
14d9951016
vb 2026-05-21 14:32:13 +03:00
Yiorgis Gozadinos
98b0c068e8
vb 2026-05-20 14:34:33 +03:00
Yiorgis Gozadinos
ceb645564c
Drop GEPA prompt optimization and --target qa from evaluations 2026-05-19 11:01:07 +03:00
Yiorgis Gozadinos
97a7eb246c
vb 2026-05-14 16:03:33 +03:00
Yiorgis Gozadinos
862a209137
vb 2026-05-13 16:53:35 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
29bc42514e
vb 2026-04-29 13:45:36 +03:00
Yiorgis Gozadinos
6807f082ec
vb 2026-04-25 09:35:47 +03:00