Commit graph

65 commits

Author SHA1 Message Date
Yiorgis Gozadinos
ac9b2cbf81
Load document blobs only when asked 2026-08-06 13:04:51 +03:00
Yiorgis Gozadinos
f96a428ef1
Fix defects found reviewing the coverage work
check_source_accessible narrowed its handler to ValueError, but Path.exists
re-raises errno values outside its ignored set (EACCES, ENAMETOOLONG). Those
were swallowed before and now escaped into the rebuild sweep the guard exists
to protect. Catch OSError too.

Restore the arity guard in _common_path_prefix: without it an empty list
raises from min() and a single label yields a prefix covering the whole path.

Two tests would have hung rather than failed on regression (the vacuum skip
and the protected-wait cancellation); both are now bounded. The import
vacuum test raced against the done-callback that discards the task, and now
spies on the call instead, with a negative control.

Replace assertions that could not fail: blank-query search against an empty
corpus, a batch flush counted against an empty table, a picture description
asserting its own input state, and an FS scheme check with nothing on disk to
resolve. The get_model matrix asserted only the returned type across 26
cases and now pins the per-provider settings. The three batching tests now
count flushes, which revealed embed-only writes through chunks_table.add
rather than _flush_rebuild_batch.
2026-07-27 10:44:32 +03:00
Yiorgis Gozadinos
f6acb65e95
Reach and enforce 100% coverage
Cover the remaining paths in the client, context, downloads, title
generation, document tools and store models, and add fail_under=100 so
uncovered lines fail CI.

Six lines that no test can reach get a pragma with its reason: the docling
import guard, the nameless PDF attachment, the FS symlink OSError guard that
resolve(strict=False) absorbs, the docling bbox and LanceDB document-id
shape guards, the tag-retention branch vacuum makes unreachable, and Monty's
Rust-thread print callback.

Fix test_find_config_file_user_config, which wrote its config into the cwd it
had chdir'd to, so the cwd branch answered first and the user-directory
lookup it names was never exercised.
2026-07-26 20:11:02 +03:00
Yiorgis Gozadinos
2afe1bd28c
Cover store engine and repository paths
Add vector-index creation tests including the warned failure, chunk
repository get_by_id, list_all pagination, blank-query and precomputed-vector
search, the unknown-score-column guard, settings row recreation, and
replace_for_document with no items.
2026-07-26 19:36:37 +03:00
Yiorgis Gozadinos
0308220769
Read verbs skip the embeddings config-compatibility check 2026-07-24 12:14:31 +03:00
Yiorgis Gozadinos
75a21c82c2
Search results and citations carry document metadata 2026-07-23 10:23:48 +03:00
Yiorgis Gozadinos
f02cde5ddf
Fix same-tick cancellation losing recovery results in _wait_protected 2026-07-16 13:11:35 +03:00
Yiorgis Gozadinos
dd2817ff6d
Protect tag-creation cleanup from cancellation 2026-07-16 13:11:35 +03:00
Yiorgis Gozadinos
1f88944ada
Protect restore rollback from cancellation; report delete_tag listing failures 2026-07-16 13:11:34 +03:00
Yiorgis Gozadinos
5453c00c95
Test restore and migration separation 2026-07-16 13:11:34 +03:00
Yiorgis Gozadinos
f41ec379df
Add Store.restore_tag 2026-07-16 13:11:34 +03:00
Yiorgis Gozadinos
a1f3435df3
Vacuum suppresses OSError only 2026-07-16 13:11:34 +03:00
Yiorgis Gozadinos
f997d7adc9
Consolidate tag primitives 2026-07-16 13:11:34 +03:00
Yiorgis Gozadinos
0e271eaf4b
Remove --before/--at time travel 2026-07-16 13:11:33 +03:00
Yiorgis Gozadinos
0813a1c980
Add tag CLI commands and history tag annotations 2026-07-16 13:11:27 +03:00
Yiorgis Gozadinos
0cbde6b7a2
Serialize tag operations, vacuum, and metadata refresh with writes 2026-07-16 12:32:57 +03:00
Yiorgis Gozadinos
69319c9390
Grow vacuum retention to protect tagged versions 2026-07-16 12:32:57 +03:00
Yiorgis Gozadinos
ae603d9b4d
Add database-level tag primitives to Store 2026-07-16 12:32:56 +03:00
Yiorgis Gozadinos
0bcf34363a
Carry merged chunk ids on SearchResult and Citation 2026-07-09 11:05:41 +03:00
Yiorgis Gozadinos
a1cf405cae
Rename document_meta identity column document_id to id 2026-07-08 16:32:55 +03:00
Yiorgis Gozadinos
d2593304c5
Attach figure bytes when a result matches its caption 2026-07-03 13:22:02 +03:00
Yiorgis Gozadinos
9f4dda9254
Reuse one MarkdownDocSerializer per document in item extraction 2026-06-24 12:02:13 +03:00
Yiorgis Gozadinos
2b2b475279
Collapse docling compression to a single function 2026-06-24 10:43:27 +03:00
bryan davis
2ca12c0096
adjust zstd handling to avoid possible core dumps with concurrency 2026-06-23 15:26:33 -05:00
Yiorgis Gozadinos
885e7b7ce7
coverage 2026-06-12 10:17:19 +03:00
Yiorgis Gozadinos
2b8b9477b7
Serialize and roll back the multi-table document delete 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
df8af54298
rebase from main 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
3366a6d383
Split meta document attributes into a document_meta table 2026-06-12 10:17:17 +03:00
Yiorgis Gozadinos
0be2e5b24f
Extract gather_database_info shared by the info command 2026-06-09 11:09:28 +03:00
Yiorgis Gozadinos
414be551fb
Open read CLI verbs read-only 2026-06-05 10:48:49 +03:00
Yiorgis Gozadinos
b6bbea3d64
Stop writing the schema version on database open 2026-06-05 10:32:30 +03:00
Yiorgis Gozadinos
5affe70eae
Cover unparseable-metadata path 2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
c2f681dd78
constant-time auth, migration short-circuit 2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
13cbadeb6f
canonical metadata keys: source_revision + content_type, bump to 0.50.0 2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
d14bf88096
fix migration failures on pre-v0.48.0 document_items schemas 2026-05-21 10:53:21 +03:00
Yiorgis Gozadinos
3b6bef3bfb
Rename get_captions_for_chunk() to get_text_for_refs() 2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
514b0c51f8
surface figure captions in search results, lower search.limit to 5 2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
f13461cdca
route figures through cite, drop show_image 2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
1360303edf
persist docling heading hierarchy on document_items 2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
4478b7ce2b
test v0.45.0 migration 2026-05-05 16:06:14 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean 2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
aa3e9406cf
auto-append /v1 to per-model Ollama base_url, fix flaky tests 2026-05-04 13:12:37 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table 2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
35e6568f59
add a regression test pinning down extract_items precedence: when a PictureItem has both an inline image.uri AND a fallback in existing_picture_data, the live URI wins. 2026-04-30 15:35:13 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags 2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
dc16f74b58
add picture_data column to document_items 2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
758bd3ee72
Group relative-path test with the other connect_lancedb dispatch tests 2026-04-25 09:28:46 +03:00
Tres Seaver
0d70f4aec6
fix: pass absolute 'db_path' to 'lancedb.connect_async'
Closes #354.
2026-04-24 17:59:01 -04:00
Yiorgis Gozadinos
3224d1f20c
Extract document orchestration into client/documents.py 2026-04-24 14:43:59 +03:00