Commit graph

48 commits

Author SHA1 Message Date
Yiorgis Gozadinos
2310b7a8b3
Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
Yiorgis Gozadinos
72ef18e39d
Reject unknown and out-of-range configuration values
Every section inherited plain BaseModel, so unknown keys were dropped
silently: providers.docling_serve.timeout was documented for months while
being ignored, and a typo in any setting took the default. Sections now
derive from ConfigModel, which forbids extras, so a stale or misspelled key
fails with its path. This already found search.context_radius in a live app
config and providers.vllm in soliplex's example.

converter, chunker and chunker_type are Literals. Sizes, limits,
dimensions, token budgets, attempt counts and breaker thresholds must be
positive; retention, delays, intervals and cooldowns non-negative;
similarity_threshold within 0-1; port within 0-65535. port 0 keeps its
OS-assigned meaning and worker_count allows 0 for an API-and-reaper-only
process.

get_reranker caught ImportError and returned None, so a configured reranker
whose extra was missing silently disappeared. It now propagates.
raise_missing_extra names the install command and re-raises when the failure
came from inside an installed package, so a broken transitive import is not
reported as a missing one. zeroentropy imported bare and now guards like the
others.

The haiku.rag package declares the jina extra. jina-local already worked
there through cross-encoder's transitive transformers and torch; the
resolved package set is unchanged, but the support is now promised rather
than inherited.

Provider fields stay unconstrained: get_model ends in a pass-through to
pydantic-ai for any provider it supports, so a Literal there would reject
valid configurations.
2026-08-19 15:32:51 +03:00
Yiorgis Gozadinos
e522d8cfa8
Remove the wix evaluation dataset 2026-08-14 14:46:51 +03:00
Yiorgis Gozadinos
af77f836d5
Reuse the local docling converter across documents 2026-08-13 11:39:01 +03:00
Yiorgis Gozadinos
e2b930d469
Use a neutral dotfile in the docling name examples
.gitignore reads as this repository's own file in the helper docstring,
the tests and the changelog entry.
2026-07-30 14:13:23 +03:00
Yiorgis Gozadinos
8e7e494fd1
Strip leading dots from names handed to docling 2026-07-30 14:07:00 +03:00
Yiorgis Gozadinos
f6acb65e95
Reach and enforce 100% coverage
Cover the remaining paths in the client, context, downloads, title
generation, document tools and store models, and add fail_under=100 so
uncovered lines fail CI.

Six lines that no test can reach get a pragma with its reason: the docling
import guard, the nameless PDF attachment, the FS symlink OSError guard that
resolve(strict=False) absorbs, the docling bbox and LanceDB document-id
shape guards, the tag-retention branch vacuum makes unreachable, and Monty's
Rust-thread print callback.

Fix test_find_config_file_user_config, which wrote its config into the cwd it
had chdir'd to, so the cwd branch answered first and the user-directory
lookup it names was never exercised.
2026-07-26 20:11:02 +03:00
Yiorgis Gozadinos
1e8e5e9f6f
Cover MCP, ingester and converter error paths
Add tests for the MCP tools' degradation contracts, malformed WebDAV
multistatus bodies, dry-run poller sweeps including the circuit-open and
discover-failure paths, FS source scheme and symlink handling, docling-serve
zip parsing, and the remaining embedding and reranker helpers. Parametrize
_strip_etag.

Drop the misplaced pragma on the analyze handler, which sat on the return and
left the except uncovered. Add one on the FS symlink OSError guard, which
resolve(strict=False) absorbs for every real link.
2026-07-26 19:14:26 +03:00
Yiorgis Gozadinos
d2f5ecfc58
Prevent magic-byte sniffing from misrouting text conversion in docling-local
Docling guesses a DocumentStream's format from its first bytes before
considering the extension, so markdown/HTML content starting with a binary
magic signature ("BM" = BMP, "ID3" = MP3) was routed to an image or audio
backend and fell back to plain-text conversion. Prefix the encoded text
with a newline so the sniff finds nothing and the extension decides.
2026-07-15 19:20:54 +03:00
Yiorgis Gozadinos
afd6e3ae00
Share the VLM URL builder and centralize the embedder empty guard 2026-06-29 15:15:26 +03:00
Yiorgis Gozadinos
ad3b111cf1
test: clarify off-loop thread assertions 2026-06-22 10:51:35 +03:00
Yiorgis Gozadinos
c6514c9df4
compare off-loop work against actual event-loop thread, fix ty 2026-06-22 10:17:02 +03:00
bryan davis
fe954a090e
move cpu bound actions off of main loop 2026-06-18 16:35:48 -05:00
Yiorgis Gozadinos
e37d764ab2
Add a docker-compose for integration test services 2026-06-04 11:13:16 +03:00
Yiorgis Gozadinos
b491f767f3
Let httpx errors flow through DoclingServeClient 2026-05-26 17:38:49 +03:00
Yiorgis Gozadinos
9ed24ad53e
swap doclaynet.pdf to the full paper, add real-PDF split/merge test 2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
80e3ae0280
clean up deprecation warnings: docling annotations + haiku.skills 0.17.1 2026-05-21 13:45:40 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum 2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
df21286cc9
add tests for mixed <img> sources and IMAGE-format wiring 2026-05-12 15:37:50 +03:00
Yiorgis Gozadinos
afd4665517
Thread source_uri through URL ingest 2026-05-12 15:13:02 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local 2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean 2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
e88321767d
always store picture bytes; collapse converter to a single zip path 2026-05-05 10:40:40 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags 2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
191cdc636b
retrieve picture image bytes via referenced+zip path, fix docling-serve not returning images even when set to do so. Remove xfail from relevant test 2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
ce320ca772
Update docling, docling-core and move to version 1.10.0 of DoclingDocument. 2026-03-23 17:34:06 +02:00
Yiorgis Gozadinos
d4e34ab79f
Support for plantuml diagrams 2026-03-13 17:18:19 +02:00
Yiorgis Gozadinos
c7b03eb9ab
Strengthen weak assertions in context enhancement and converter tests
- Add doc_item_refs and headings assertions to test_expand_context_docling_merges_metadata
- Add page count and markdown content assertions to test_convert_pdf_with_ocr_engine

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-05 11:44:51 +02:00
Yiorgis Gozadinos
62d35d2907
Add ocr_engine field to ConversionOptions 2026-01-20 13:41:43 +02:00
Yiorgis Gozadinos
929c29250e
Additional tests for page & picture images for local and docling-serve 2026-01-14 12:48:13 +02:00
Yiorgis Gozadinos
447e94c5db
Config options for generating page images 2026-01-14 11:55:08 +02:00
Yiorgis Gozadinos
98b3d114b3
Update to docling-document 1.9.0 2026-01-13 19:20:27 +02:00
Yiorgis Gozadinos
c9cbf04cc8
Reverted docling from 2.67.0 to 2.65.0 to use docling-document 1.8.0 2026-01-13 18:22:58 +02:00
Yiorgis Gozadinos
ce326d80ff
Disable ocr in tests previously marked as integration 2026-01-12 16:19:25 +02:00
Yiorgis Gozadinos
52363639a6
Remove integration marker from tests that have vcr 2026-01-12 16:03:05 +02:00
Yiorgis Gozadinos
9c4fd08c47
Update client serialization for compressed docling documents 2026-01-11 09:23:56 +02:00
Yiorgis Gozadinos
f861664656
Extract shared DoclingServeClient for async workflow & use it for VLM support, chunking, converting 2026-01-07 11:09:23 +02:00
Yiorgis Gozadinos
3b7fd440cc
End-to-end test for VLM annotation 2026-01-07 11:09:23 +02:00
Yiorgis Gozadinos
769e0d4746
Update picture description prompt, move its config under prompts 2026-01-07 11:09:23 +02:00
Yiorgis Gozadinos
3c4116397d
Add VLM picture description support for image handling 2026-01-07 11:09:22 +02:00
Yiorgis Gozadinos
0da8ba97c4
Docling-serve cassettes 2025-12-26 18:59:46 +02:00
Yiorgis Gozadinos
3149552a61
Add 'plain' format and fallback for text conversion 2025-12-12 18:11:25 +02:00
Yiorgis Gozadinos
68457f251e
Do not generate base64 images by default 2025-12-09 18:42:56 +02:00
Yiorgis Gozadinos
06b4f6a224
Add format parameter for text-to-DoclingDocument conversion 2025-12-08 15:56:21 +02:00
Yiorgis Gozadinos
e40c352026
Switch from requests to httpx.AsyncClient for docling-serve 2025-11-26 18:00:29 +02:00
Yiorgis Gozadinos
14bdb7cb27
async versions for convert_file() convert_text() 2025-11-26 17:46:25 +02:00
Yiorgis Gozadinos
0bb3f4301c
Additional options for document conversion with docling 2025-11-18 15:44:04 +02:00
Yiorgis Gozadinos
fe7d0c5c24
Implement docling-serve converter 2025-11-17 13:05:05 +02:00