Chris McDonough
720ff23357
Fix shutdown regression: notify idle workers on stop()
...
Workers parked on job_available.wait() were not woken by stop(),
causing them to sleep out the full poll_idle_interval_s before
noticing _stop. With the default 1.0s interval, stop() took ~0.8s
instead of ~0.007s.
Notify all waiters on the condition in stop() so idle workers exit
immediately. Add tests for fast job pickup via notification and
fast shutdown with idle workers.
2026-06-01 09:21:24 -04:00
Yiorgis Gozadinos
8bf1b6be1a
Merge pull request #403 from mcdonc/fix/watch-change-stat-race
...
fix: watch loop crash when file deleted before stat() in _handle_watch_change
2026-06-01 15:59:50 +03:00
Chris McDonough
3c7dbf0966
Classify PermissionError as PermanentError instead of TransientError
...
PermissionError is a subclass of OSError, so it was caught by the
broad timeout/io handler and classified as transient. An unreadable
file would retry 5 times then DLQ — permissions don't fix themselves
without operator intervention.
Add an explicit PermissionError check before the OSError catch so
unreadable files go straight to the DLQ.
2026-06-01 15:36:37 +03:00
Chris McDonough
f33a4629e1
Handle sync_state write failure after mark_succeeded without crashing
...
If sync.upsert() or sync.delete() raises after a job is already
marked succeeded (e.g. disk full, DB locked), the unhandled
exception crashes the worker. The job stays succeeded but sync_state
is stale, and the crashed worker stops processing other jobs.
Wrap the post-success sync_state writes in a try/except. On failure,
log the error and continue. The worst case is a redundant re-ingest
on the next sweep — better than killing the worker.
2026-06-01 08:11:02 -04:00
Chris McDonough
da8e7dc568
Fix run_batch hanging forever when all workers die
...
The drain loop in run_batch() polls counts_by_status() waiting for
queued and claimed counts to reach zero. If all worker tasks crash
(unhandled exception, OOM), claimed jobs stay claimed forever and
the loop never exits — the CLI command hangs.
Check live_workers during the drain loop. If claimed jobs exist but
no workers are alive to process them, log an error and break out.
The stranded jobs will be reaped on the next start.
2026-06-01 07:52:17 -04:00
Chris McDonough
42922cd2bd
Fix watch loop crash when file is deleted before stat() in _handle_watch_change
...
The expression `str(path.stat().st_mtime_ns) if path.exists() else None`
has a TOCTOU race: the file can be deleted between exists() and stat().
The resulting FileNotFoundError propagates up to _watch_loop's except
handler, which records a breaker failure and terminates the loop — no
more push events are processed until restart.
Replace with a try/except around stat() and return early on
FileNotFoundError. The deletion event from watchfiles will handle
cleanup.
2026-06-01 07:46:35 -04:00
Chris McDonough
c9c48bc814
Classify FileNotFoundError as PermanentError instead of TransientError
...
FileNotFoundError is a subclass of OSError, so it was caught by the
broad timeout/io handler and classified as transient. A file deleted
between discovery and fetch would retry 5 times on a file that's
permanently gone, then DLQ with a confusing error message.
Add an explicit FileNotFoundError check before the OSError catch so
deleted files go straight to the DLQ.
2026-06-01 07:32:09 -04:00
Chris McDonough
7d80eb4f83
Fix discover() crash when file is deleted during stat()
...
A file deleted between os.walk() and path.stat() raises
FileNotFoundError, which propagated uncaught and failed the entire
discover() sweep. With enough failures this trips the circuit
breaker, silencing the poller.
Catch FileNotFoundError around the stat() call and skip the file.
The next sweep (or watchfiles) will emit the DELETE event.
2026-06-01 07:29:51 -04:00
Yiorgis Gozadinos
339d231a4e
Remove repliqa from test setup
2026-06-01 10:40:52 +03:00
Yiorgis Gozadinos
5a23e4eda6
Surface failed discovery sweeps in run-batch
2026-06-01 10:27:31 +03:00
Yiorgis Gozadinos
6f2a40c676
cover run-batch, serve, and _stop_pool with tests
2026-05-29 17:43:58 +03:00
Yiorgis Gozadinos
5ba7838b71
Add haiku-ingester run-batch, remove run-once
2026-05-29 17:00:59 +03:00
Yiorgis Gozadinos
e7c7df2915
Always use the Store-owned embedder
2026-05-29 11:36:53 +03:00
Yiorgis Gozadinos
37a78a4e9b
Cache reranker on the client instead of rebuilding per search
2026-05-29 10:33:50 +03:00
Yiorgis Gozadinos
d8d6727227
improve coverage
2026-05-28 18:06:47 +03:00
Yiorgis Gozadinos
402957d3e5
Fix CI flakes for PDF attachment extraction
2026-05-28 17:51:02 +03:00
Yiorgis Gozadinos
6eee09743b
Cover PDF attachment ingest through the full create_document_from_source path
2026-05-28 16:07:12 +03:00
Yiorgis Gozadinos
8e8c4433bd
Extract PDF /EmbeddedFiles attachments as child documents
2026-05-28 15:51:32 +03:00
Yiorgis Gozadinos
26bc71d6d8
Cascade delete_document to children via metadata.parent_uri
2026-05-28 15:36:00 +03:00
Yiorgis Gozadinos
5400085147
Auto-prune dead jobs when a sibling DELETE succeeds
2026-05-27 17:17:01 +03:00
Yiorgis Gozadinos
1e6dc34871
Skip FS DELETE enqueue when the file is already back
2026-05-27 15:18:46 +03:00
Yiorgis Gozadinos
15d13cab04
Probe docling-serve only when converter or chunker uses it
2026-05-27 15:18:25 +03:00
Yiorgis Gozadinos
b0d0ac588d
Split snapshot APIs and resolve_fetcher by intent
2026-05-27 14:39:54 +03:00
Yiorgis Gozadinos
2c63ceda0c
Backfill ingester test gaps and drop unneeded retry clamp
2026-05-27 14:24:22 +03:00
Yiorgis Gozadinos
cfbd0b09d6
Tighten HTTP config-removal handling
2026-05-27 14:07:45 +03:00
Yiorgis Gozadinos
7466539b4f
Emit DELETE for HTTP URLs removed from config
2026-05-27 13:39:18 +03:00
Yiorgis Gozadinos
d7fdd61fed
Resolve worker source by source_id, not just supports(uri)
2026-05-27 13:36:03 +03:00
Yiorgis Gozadinos
922d1d567d
Prevent DELETE/UPSERT race for the same URI
2026-05-27 13:30:46 +03:00
Yiorgis Gozadinos
1c710433c7
Collapse worker_count and max_concurrent into worker_count
2026-05-27 12:48:14 +03:00
Yiorgis Gozadinos
87af5e5139
Boot-reap stale claims at WorkerPool start
2026-05-27 11:46:37 +03:00
Yiorgis Gozadinos
1b36452629
Surface provider reachability and per-job failure context on dashboard
2026-05-27 11:38:48 +03:00
Yiorgis Gozadinos
439307d5af
Add pool-wide circuit breaker to the worker pool
2026-05-27 10:52:48 +03:00
Yiorgis Gozadinos
b491f767f3
Let httpx errors flow through DoclingServeClient
2026-05-26 17:38:49 +03:00
Yiorgis Gozadinos
fdb73fc41f
Drop defensive branches
2026-05-26 16:56:32 +03:00
Yiorgis Gozadinos
29b3ccdbda
Drain orphan cancel-cleanup releases before closing the queue
2026-05-26 16:14:17 +03:00
Yiorgis Gozadinos
3c608046e2
Bound ?limit and ?offset on /jobs and /dlq
2026-05-26 16:05:47 +03:00
Yiorgis Gozadinos
761956eb70
Extend reaper-resurrection guard to reschedule and release_if_claimed
2026-05-26 16:04:24 +03:00
Yiorgis Gozadinos
a822b754b7
Guard mark_succeeded/mark_dead against reaper resurrection
2026-05-26 14:07:44 +03:00
Yiorgis Gozadinos
9cb3ce40ad
Shield cancel-cleanup release in worker pool
2026-05-26 13:57:35 +03:00
Yiorgis Gozadinos
78fc5d0e05
Discover within-root symlinks in FSSource
2026-05-26 13:54:33 +03:00
Yiorgis Gozadinos
e5e0df6ade
Unlink PDF slice tempfiles when slice write fails
2026-05-26 13:46:24 +03:00
Yiorgis Gozadinos
98efd73177
Serialize JobRepo and SyncStateRepo on one shared lock
2026-05-26 13:11:01 +03:00
Yiorgis Gozadinos
5f31a6b12f
Fix round-robin docling-serve
2026-05-26 13:07:21 +03:00
Yiorgis Gozadinos
cf6caf14fe
Tighten retry/reap_stale guards; fix SourceSummary.type leak
2026-05-26 12:38:55 +03:00
Yiorgis Gozadinos
f89cc998eb
add sources_provider to WorkerPool. workers now resolve extra info through these sources
2026-05-26 11:45:35 +03:00
Yiorgis Gozadinos
410279cd6c
Preserve sync_state revision when poller bumps last_seen_at
2026-05-26 11:45:35 +03:00
Yiorgis Gozadinos
ef9cacf981
Harden FS source against symlink escape; surface pool/poller liveness in /health, additional auth tests
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
4aee18dcbe
Operator dashboard at GET /; tighten Logfire span shape
...
Self-contained HTML status page served from the ingester's FastAPI app.
Polls /health, /sources, /stats, /jobs?status={claimed,dead,succeeded}
every 3s from the browser and renders queue chips, sources with
last-poll/skip-reason/circuit state, active jobs with cancel, recent
failures with retry, and recently-completed feed with op badges so
DELETE rows are visually distinct from UPSERTs. Zero external deps —
single static HTML, no CDN, no fonts, no images. Works offline.
To support the dashboard:
- New /stats endpoint exposing rolling throughput (5m/30m/1h), worker
occupancy, oldest-queued age, and per-source DLQ + queue-depth
breakdowns. Each field is a single SQL aggregation against the queue.
- JobRepo gains count_succeeded_since, oldest_queued_age_seconds,
counts_by_source.
- SourceSummary gains last_skip_reason. BasePoller now records the
reason the most recent sweep attempt was skipped ("pending_work" /
"circuit_open"), cleared on the next successful poll. Closes the
gap where operators couldn't tell from /sources alone why a source
wasn't picking up new work.
Auth: dashboard route is unauthenticated (markup only). The JS attaches
the bearer to its own JSON fetches; on 401 it prompts once and stashes
the token in localStorage.
Two Logfire fixes that landed alongside:
- Drop logfire.instrument_fastapi() and the [fastapi] extra. The control
plane is polled frequently (dashboard + docker healthcheck), so every
endpoint became a span and drowned the useful traces. logfire itself
stays — pulled in transitively via pydantic-ai-slim[logfire] — so
ingester.poller.* / ingester.job / document.* spans keep emitting.
- Wrap FSPoller._handle_watch_change in an ingester.poller.watch_event
span and pass _enqueue_extra. Without this, the watchfiles callback
ran with no active context, the _otel carrier in job.extra was empty,
and the worker's ingester.job span surfaced as an orphan trace root
instead of nesting under the FS event that caused it.
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
5affe70eae
Cover unparseable-metadata path
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
9ca06df745
Bound pdf_split memory + release lock between slices
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
c2f681dd78
constant-time auth, migration short-circuit
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
da3cfe1a58
Release claim on jobs when cancelled
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
710276ffd8
record cassettes against new fixtures, fix stale assertions
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
ed36cc2230
typed UnsupportedSourceError replaces pipeline string-marker matching
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
57e89426ea
regex-based _strip_etag with weak-marker test
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
9ed24ad53e
swap doclaynet.pdf to the full paper, add real-PDF split/merge test
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
6bcc2f6357
PDF split-convert-merge for memory-bound large PDFs
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
ec94426556
HTTPSource.head() returns ETag for the revision short-circuit
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
055fd23d5d
round-robin docling-serve across multiple base_urls
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
5ccbadde0a
Handle shutdown more gracefully, by stopping pollers and cancelling jobs after timeout. Skip periodic poll if a source has pending jobs
2026-05-26 11:44:03 +03:00
Yiorgis Gozadinos
65c309e12b
Handle WebDAV as source
2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
13cbadeb6f
canonical metadata keys: source_revision + content_type, bump to 0.50.0
2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
6d15f1f247
fix URL-decode file:// paths before stat/exists checks
2026-05-26 11:42:42 +03:00
Yiorgis Gozadinos
7ea61a7b10
drop the old monitor, rename serve→mcp, add e2e tests
2026-05-26 11:42:41 +03:00
Yiorgis Gozadinos
de3b3fa1c9
HTTP control plane
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
75c3896588
additional config, pollers, serve
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
bdcc1caa49
worker pool, rety policy, pipeline
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
4b3641244c
SQLite queue + haiku-ingester CLI skeleton
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
717fc3320b
route create_document_from_source through source adapters
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
b9637cd625
add HTTP and S3 source adapters + resolve_fetcher
2026-05-26 11:41:53 +03:00
Yiorgis Gozadinos
d0a730ef60
ingester: add source-adapter scaffolding + FS source
2026-05-26 11:41:53 +03:00
Yiorgis Gozadinos
8e9e84132e
drop list_documents/get_document from default RAG skill
2026-05-26 11:35:43 +03:00
Yiorgis Gozadinos
260b8e8b09
Merge pull request #382 from ggozad/chore/updates
...
Prepare for pydantic-ai 2.0, update haiku.skills
2026-05-21 14:30:55 +03:00
Yiorgis Gozadinos
188d35023f
rotate chat TUI conversation id per launch / clear-chat
2026-05-21 14:21:51 +03:00
Yiorgis Gozadinos
80e3ae0280
clean up deprecation warnings: docling annotations + haiku.skills 0.17.1
2026-05-21 13:45:40 +03:00
Yiorgis Gozadinos
d14bf88096
fix migration failures on pre-v0.48.0 document_items schemas
2026-05-21 10:53:21 +03:00
Yiorgis Gozadinos
b48ac2fdd0
sandbox + cite: guard sort precondition; require rag in cite fallback
2026-05-20 13:02:25 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow
2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
a317a951d9
CLI citations: compact panel, inline figures, doc/chunk IDs in footer
2026-05-20 12:00:24 +03:00
Yiorgis Gozadinos
3858ab905a
toc.json nodes carry chunk_ids; fix cite to accept DB-resolvable chunk_ids
2026-05-20 11:37:35 +03:00
Yiorgis Gozadinos
04eeec77d2
per-doc lazy items/toc cache; index document_items for fast per-doc lookup
2026-05-20 10:08:25 +03:00
Yiorgis Gozadinos
e3314e36fd
search: only attach picture bytes from pre-expansion chunks
2026-05-19 15:03:22 +03:00
Yiorgis Gozadinos
e8c61ad0c6
items.jsonl exposes chunk_ids per row; drops position and tree_depth
2026-05-19 14:20:11 +03:00
Yiorgis Gozadinos
6f95e2bc27
Delete the standalone QA and analysis agents
2026-05-19 11:39:20 +03:00
Yiorgis Gozadinos
947a26b391
client.analyze routes through the rag-analysis skill; drop documents= and AnalysisResult.program. Re-record all cassettes that are relevant
2026-05-19 10:46:32 +03:00
Yiorgis Gozadinos
d96d2eeb0f
client.ask routes through the rag skill; always show citations in CLI
2026-05-19 10:19:39 +03:00
Yiorgis Gozadinos
57d077e2c3
cite raises ModelRetry on unresolved chunk_ids
2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
84e91097a8
Stop hitting the embedder in test_sandbox_toc fixtures
2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
3b6bef3bfb
Rename get_captions_for_chunk() to get_text_for_refs()
2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
74dea65f21
analysis.model inherits qa.model when unset; per-skill vision gate
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
514b0c51f8
surface figure captions in search results, lower search.limit to 5
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
e51841b924
Drop list_documents from analysis skill
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
f13461cdca
route figures through cite, drop show_image
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
387c4f12a9
multimodal sandbox: show_image + picture_refs, drop llm
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
2831030f9b
expose toc.json + heading_level/tree_depth in analysis VFS
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
1360303edf
persist docling heading hierarchy on document_items
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
5de4d50622
coverage
2026-05-14 15:54:43 +03:00
Yiorgis Gozadinos
57f273e000
Minor fixes, CI should build
2026-05-14 15:43:36 +03:00
Yiorgis Gozadinos
519afe6709
Add cross-encoder reranking provider
2026-05-14 15:43:35 +03:00
Yiorgis Gozadinos
dbb87e091f
Ensure schema parity for _StagingChunkRecord
2026-05-14 15:11:18 +03:00
Yiorgis Gozadinos
849eca94d0
Make rebuild --embed-only idempotent across crashes
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
a035b0f9e4
Stream rebuild --embed-only through a staging table to bound memory
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
704bea6f4a
Stabilize test_client_search ranking assertion
2026-05-14 14:37:27 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum
2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
bdab201c42
Merge pull request #371 from ggozad/fix/image-results
...
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:19:09 +03:00
Yiorgis Gozadinos
a6e5664c78
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:01:39 +03:00
Yiorgis Gozadinos
072edad62f
Merge pull request #368 from ggozad/feat/relax-compat-check
...
Relax embedding compatibility check to vector_dim only
2026-05-13 15:43:37 +03:00
Yiorgis Gozadinos
701e9d3632
Relax embedding compat check to vector_dim only
2026-05-13 14:44:18 +03:00
Yiorgis Gozadinos
8b135c4d4d
Expose embedding batch size as config
2026-05-13 14:09:47 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through
2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
0f4c4af495
Cache the chunking tokenizer to avoid HF Hub 429
2026-05-13 13:32:24 +03:00
Yiorgis Gozadinos
df21286cc9
add tests for mixed <img> sources and IMAGE-format wiring
2026-05-12 15:37:50 +03:00
Yiorgis Gozadinos
afd4665517
Thread source_uri through URL ingest
2026-05-12 15:13:02 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local
2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
9f8c9fe3b1
cover s3 uri override, unsupported extension, watcher error path
2026-05-11 11:29:56 +03:00
Yiorgis Gozadinos
6631e339ee
record VCR cassettes for s3 source tests
2026-05-11 11:29:56 +03:00
Yiorgis Gozadinos
ef82e6eb08
swap aioboto3 for obstore in the S3 client path
2026-05-11 11:29:42 +03:00
Yiorgis Gozadinos
f87d894b01
add SeaweedFS integration tests for S3Watcher
2026-05-11 11:28:40 +03:00
Yiorgis Gozadinos
37099988a6
poll S3 prefixes with S3Watcher and wire into serve
2026-05-11 11:28:40 +03:00
Yiorgis Gozadinos
d009da06d6
support s3:// document sources
2026-05-11 11:28:31 +03:00
Yiorgis Gozadinos
6af463699d
record VCR cassette for vllm embedder end-to-end test
2026-05-06 20:01:22 +03:00
Yiorgis Gozadinos
e55e3d13c8
fix _patch_picture_descriptions silently dropping docling_pages
2026-05-06 12:57:41 +03:00
Yiorgis Gozadinos
75f75ec505
render attached pictures in inspector context modal under qa.model.vision
2026-05-06 12:00:12 +03:00
Yiorgis Gozadinos
7fac35d2af
rename embed_image_query to embed_image; run description check on text path
2026-05-05 16:15:47 +03:00
Yiorgis Gozadinos
1ccb5b5fad
auto-append /v1 to vllm base_url, matching ollama behavior
2026-05-05 16:08:14 +03:00
Yiorgis Gozadinos
4478b7ce2b
test v0.45.0 migration
2026-05-05 16:06:14 +03:00
Yiorgis Gozadinos
c96894a0f0
fix cross-document picture dedup in agent and skill search tools
2026-05-05 15:56:10 +03:00
Yiorgis Gozadinos
61a88edfd8
drop bogus deprecation warning for processing.pictures
2026-05-05 14:46:41 +03:00
Yiorgis Gozadinos
08696773fe
Fix vcr test
2026-05-05 14:38:03 +03:00
Yiorgis Gozadinos
49274ab51b
attach picture bytes through skill search and document the schema
2026-05-05 14:33:57 +03:00
Yiorgis Gozadinos
46b81f3fa1
cover picture-description provider and rebuild edge cases
2026-05-05 14:20:37 +03:00
Yiorgis Gozadinos
a45d82f206
Update benchmark
2026-05-05 13:58:48 +03:00
Yiorgis Gozadinos
ff82d36c2f
add rebuild --descriptions: run VLM over stored picture bytes only
2026-05-05 12:02:46 +03:00
Yiorgis Gozadinos
5fad0aa5c6
warn when picture descriptions silently fail to come back
2026-05-05 11:44:18 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean
2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
e88321767d
always store picture bytes; collapse converter to a single zip path
2026-05-05 10:40:40 +03:00
Yiorgis Gozadinos
35d5f4416e
fill in vLLM error-path and MCP image-query coverage
2026-05-05 09:39:20 +03:00
Yiorgis Gozadinos
ee41bc676f
remove defensive code
2026-05-04 15:12:27 +03:00
Yiorgis Gozadinos
e8d89aa035
dedup picture-only chunks at the search-result layer
2026-05-04 13:31:15 +03:00
Yiorgis Gozadinos
33e462d987
add real vLLM integration test
2026-05-04 13:16:31 +03:00
Yiorgis Gozadinos
aa3e9406cf
auto-append /v1 to per-model Ollama base_url, fix flaky tests
2026-05-04 13:12:37 +03:00
Yiorgis Gozadinos
65d9c74224
expose image-as-query through MCP and the CLI.
2026-05-04 11:46:45 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table
2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
21f12cd770
add vision: bool flag on ModelConfig (default False). Gate the agent search tool's BinaryContent attachment on qa.model.vision so picture bytes are only sent to vision-capable QA models.
2026-05-04 10:47:21 +03:00
Yiorgis Gozadinos
6037f26673
drop mlx
2026-05-03 19:05:49 +03:00
Yiorgis Gozadinos
c9fdad15a9
bump docling-core to >=2.74.1 so that local docling-local and docling-serve chunkers run the same MarkdownTableSerializer
2026-05-03 18:40:44 +03:00
Yiorgis Gozadinos
c9227c649f
emit synthetic picture chunks at ingest under multimodal embedders.
...
processing.chunk() merges text chunks with one synthetic Chunk per PictureItem-with-bytes,
sorted by iterate_items() position so chunk.order is structural.
embed_chunks dispatches on a Chunk._picture_data PrivateAttr
(text through embed_documents, picture through embed_image_query)
2026-05-03 17:04:43 +03:00
Yiorgis Gozadinos
a36f920f5d
drop the batched embed_images method from multi-modal embedders
2026-05-03 16:21:47 +03:00
Yiorgis Gozadinos
37215f1a00
add multimodal embedder support for vllm and mlx
2026-05-03 10:57:56 +03:00
Yiorgis Gozadinos
75c8e83515
Allow uri override on create_document_from_source
2026-04-30 16:36:35 +03:00
