Yiorgis Gozadinos
cd77bd9889
Bound analysis execute_code calls to avoid request-limit nulls
2026-06-06 14:52:05 +03:00
Yiorgis Gozadinos
44089e5b1f
Support a dburi for the ingester queue (SQLite + Postgres)
...
Migrate the ingester queue storage from raw aiosqlite to SQLAlchemy Core
async. The backend is chosen by ingester.queue.dburi: a SQLAlchemy async
URL points the queue at a database server, and SQLite remains the default
when unset. The Postgres path claims jobs with FOR UPDATE SKIP LOCKED so
multiple ingester processes can share one queue; SQLite caps the pool to a
single connection to keep the select-then-update claim atomic.
2026-06-03 14:34:09 +03:00
Yiorgis Gozadinos
46747d369a
Add retention window to ingester queue, prune terminal job rows past retention window
2026-06-03 10:38:58 +03:00
Chris McDonough
7e4aa8c71f
Add configurable max_file_size to reject oversized files before ingestion
...
Large files buffered entirely in RAM can OOM workers. Add
max_file_size to source config (default None = no limit).
FS checks stat().st_size before read_bytes(). HTTP and WebDAV issue
a HEAD request before GET when a limit is configured. S3 checks the
size from the existing head_async() call before get_async().
FileTooLargeError is classified as PermanentError so oversized files
go straight to the DLQ instead of retrying.
2026-06-01 18:40:46 +03:00
Yiorgis Gozadinos
26bc71d6d8
Cascade delete_document to children via metadata.parent_uri
2026-05-28 15:36:00 +03:00
Yiorgis Gozadinos
1c710433c7
Collapse worker_count and max_concurrent into worker_count
2026-05-27 12:48:14 +03:00
Yiorgis Gozadinos
fdb73fc41f
Drop defensive branches
2026-05-26 16:56:32 +03:00
Yiorgis Gozadinos
6799a4a6d4
Document worker-pool settings
2026-05-26 14:13:16 +03:00
Yiorgis Gozadinos
6bcc2f6357
PDF split-convert-merge for memory-bound large PDFs
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
055fd23d5d
round-robin docling-serve across multiple base_urls
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
5ccbadde0a
Handle shutdown more gracefully, by stopping pollers and cancelling jobs after timeout. Skip periodic poll if a source has pending jobs
2026-05-26 11:44:03 +03:00
Yiorgis Gozadinos
65c309e12b
Handle WebDAV as source
2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
7ea61a7b10
drop the old monitor, rename serve→mcp, add e2e tests
2026-05-26 11:42:41 +03:00
Yiorgis Gozadinos
75c3896588
additional config, pollers, serve
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
4b3641244c
SQLite queue + haiku-ingester CLI skeleton
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
f9392cee46
evaluations: judge model moves to config.evaluations.judge
2026-05-20 12:56:22 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow
2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
6f95e2bc27
Delete the standalone QA and analysis agents
2026-05-19 11:39:20 +03:00
Yiorgis Gozadinos
74dea65f21
analysis.model inherits qa.model when unset; per-skill vision gate
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
514b0c51f8
surface figure captions in search results, lower search.limit to 5
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum
2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
8b135c4d4d
Expose embedding batch size as config
2026-05-13 14:09:47 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through
2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local
2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
d009da06d6
support s3:// document sources
2026-05-11 11:28:31 +03:00
Yiorgis Gozadinos
61a88edfd8
drop bogus deprecation warning for processing.pictures
2026-05-05 14:46:41 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean
2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
e8d89aa035
dedup picture-only chunks at the search-result layer
2026-05-04 13:31:15 +03:00
Yiorgis Gozadinos
21f12cd770
add vision: bool flag on ModelConfig (default False). Gate the agent search tool's BinaryContent attachment on qa.model.vision so picture bytes are only sent to vision-capable QA models.
2026-05-04 10:47:21 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags
2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
ff8ad0879c
fix context expansion: respect section boundaries, remove max_context_items
2026-04-20 15:18:16 +03:00
Yiorgis Gozadinos
d2b3ba1b59
rename RLM agent to analysis throughout the codebase
2026-04-17 18:32:01 +03:00
Yiorgis Gozadinos
75c84805b4
Improve judge prompt, QA prompt, and raise max_searches to 5
2026-04-16 12:12:23 +03:00
Yiorgis Gozadinos
b6113bf8ab
replace fixed-radius expansion with section-bounded algorithm
...
Context expansion is now automatic and structure-aware. For structured
documents, expands within the section containing the match. For sections
that exceed the budget or are too small, expands item-by-item outward
skipping noise labels. Unstructured documents use budget-based outward
expansion. Results sorted by relevance score.
2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
110accb8e7
Add ConnectionMode enum (LOCAL/CLOUD/OBJECT_STORAGE) and connect_lancedb() utility to support S3, GCS, Azure, and HDFS backends via storage_options.
2026-04-08 14:19:09 +03:00
Yiorgis Gozadinos
45e94ec67a
Set search limit to 10
2026-03-11 12:26:41 +02:00
Yiorgis Gozadinos
e2749ad2a6
Cap QA agent search iterations to reduce response time
2026-03-11 12:13:43 +02:00
Yiorgis Gozadinos
5a0430857f
Enable thinking by default for QA agent
2026-03-05 15:48:08 +02:00
Yiorgis Gozadinos
3cc2d5e19e
Set appropriate temperature and max_tokens defaults
2026-03-05 13:23:50 +02:00
Yiorgis Gozadinos
ae3469b282
add automatic title generation for documents
2026-02-25 17:12:24 +02:00
Yiorgis Gozadinos
c61ded1271
Replace Docker sandbox with pydantic-monty
2026-02-24 09:55:40 +02:00
Yiorgis Gozadinos
9685bd533b
Expand ~ in paths
2026-02-16 17:21:35 +02:00
Yiorgis Gozadinos
720f697a48
Cleanup
2026-02-06 12:07:21 +01:00
Yiorgis Gozadinos
b426827fc2
Docker-based sandbox. Reused within a single rlm() call for latency
2026-02-06 12:07:19 +01:00
Yiorgis Gozadinos
b68b2393e9
Integrate with client, cli, app, mcp
2026-02-06 12:07:03 +01:00
Yiorgis Gozadinos
ac76ecee86
Remove dead code from iterative research planning refactor
2026-01-31 10:51:34 +02:00
Yiorgis Gozadinos
62d35d2907
Add ocr_engine field to ConversionOptions
2026-01-20 13:41:43 +02:00
Yiorgis Gozadinos
447e94c5db
Config options for generating page images
2026-01-14 11:55:08 +02:00
Yiorgis Gozadinos
bb61e8fab6
Remove custom AG-UI infrastructure in favor of pydantic-ai native support
2026-01-12 12:37:07 +02:00
Yiorgis Gozadinos
f861664656
Extract shared DoclingServeClient for async workflow & use it for VLM support, chunking, converting
2026-01-07 11:09:23 +02:00
