Commit graph

1 commit

Author SHA1 Message Date
Yiorgis Gozadinos
581ddf4f41
Bound vacuum compaction memory on the tables holding docling blobs
Vacuum compacts documents and document_items through lance directly
with a fragment target derived from storage.compaction_target_bytes
(default 2 GiB), instead of AsyncTable.optimize, whose default
1,048,576-row fragment target re-merges the whole table on every pass
and peaks at 5-6x the table's bytes. The target is sized from the
widest fragment's bytes per row, so peak memory is bounded by the
target instead of the corpus. lancedb's async API exposes no
compaction options (lancedb/lancedb#2325), hence the pylance
dependency.
2026-09-04 10:35:26 +03:00