Yiorgis Gozadinos
6799a4a6d4
Document worker-pool settings
2026-05-26 14:13:16 +03:00
Yiorgis Gozadinos
a822b754b7
Guard mark_succeeded/mark_dead against reaper resurrection
2026-05-26 14:07:44 +03:00
Yiorgis Gozadinos
9cb3ce40ad
Shield cancel-cleanup release in worker pool
2026-05-26 13:57:35 +03:00
Yiorgis Gozadinos
78fc5d0e05
Discover within-root symlinks in FSSource
2026-05-26 13:54:33 +03:00
Yiorgis Gozadinos
e5e0df6ade
Unlink PDF slice tempfiles when slice write fails
2026-05-26 13:46:24 +03:00
Yiorgis Gozadinos
98efd73177
Serialize JobRepo and SyncStateRepo on one shared lock
2026-05-26 13:11:01 +03:00
Yiorgis Gozadinos
5f31a6b12f
Fix round-robin docling-serve
2026-05-26 13:07:21 +03:00
Yiorgis Gozadinos
cf6caf14fe
Tighten retry/reap_stale guards; fix SourceSummary.type leak
2026-05-26 12:38:55 +03:00
Yiorgis Gozadinos
f89cc998eb
add sources_provider to WorkerPool. workers now resolve extra info through these sources
2026-05-26 11:45:35 +03:00
Yiorgis Gozadinos
410279cd6c
Preserve sync_state revision when poller bumps last_seen_at
2026-05-26 11:45:35 +03:00
Yiorgis Gozadinos
083ff401d5
vacuum db after migration
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
73f8349a00
Use haiku.rag as telemtry scope
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
d900651b92
Render "queue busy" badge on sources skipped due to pending_work
...
Replaces the confusing em-dash + raw `pending_work` text when no successful sweep has run yet.
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
ef9cacf981
Harden FS source against symlink escape; surface pool/poller liveness in /health, additional auth tests
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
4aee18dcbe
Operator dashboard at GET /; tighten Logfire span shape
...
Self-contained HTML status page served from the ingester's FastAPI app.
Polls /health, /sources, /stats, /jobs?status={claimed,dead,succeeded}
every 3s from the browser and renders queue chips, sources with
last-poll/skip-reason/circuit state, active jobs with cancel, recent
failures with retry, and recently-completed feed with op badges so
DELETE rows are visually distinct from UPSERTs. Zero external deps —
single static HTML, no CDN, no fonts, no images. Works offline.
To support the dashboard:
- New /stats endpoint exposing rolling throughput (5m/30m/1h), worker
occupancy, oldest-queued age, and per-source DLQ + queue-depth
breakdowns. Each field is a single SQL aggregation against the queue.
- JobRepo gains count_succeeded_since, oldest_queued_age_seconds,
counts_by_source.
- SourceSummary gains last_skip_reason. BasePoller now records the
reason the most recent sweep attempt was skipped ("pending_work" /
"circuit_open"), cleared on the next successful poll. Closes the
gap where operators couldn't tell from /sources alone why a source
wasn't picking up new work.
Auth: dashboard route is unauthenticated (markup only). The JS attaches
the bearer to its own JSON fetches; on 401 it prompts once and stashes
the token in localStorage.
Two Logfire fixes that landed alongside:
- Drop logfire.instrument_fastapi() and the [fastapi] extra. The control
plane is polled frequently (dashboard + docker healthcheck), so every
endpoint became a span and drowned the useful traces. logfire itself
stays — pulled in transitively via pydantic-ai-slim[logfire] — so
ingester.poller.* / ingester.job / document.* spans keep emitting.
- Wrap FSPoller._handle_watch_change in an ingester.poller.watch_event
span and pass _enqueue_extra. Without this, the watchfiles callback
ran with no active context, the _otel carrier in job.extra was empty,
and the worker's ingester.job span surfaced as an orphan trace root
instead of nesting under the FS event that caused it.
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
5affe70eae
Cover unparseable-metadata path
2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
9ca06df745
Bound pdf_split memory + release lock between slices
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
c2f681dd78
constant-time auth, migration short-circuit
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
da3cfe1a58
Release claim on jobs when cancelled
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
14beba6786
Lock pypdfium2 under concurrent workers; per-slice convert spans
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
ba623b0862
mark event-loop glue + defensive guards as no-cover
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
ed36cc2230
typed UnsupportedSourceError replaces pipeline string-marker matching
2026-05-26 11:44:46 +03:00
Yiorgis Gozadinos
57e89426ea
regex-based _strip_etag with weak-marker test
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
6bcc2f6357
PDF split-convert-merge for memory-bound large PDFs
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
f4468b65ee
Add --host, --port to ingester cli, update docker compose example
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
ec94426556
HTTPSource.head() returns ETag for the revision short-circuit
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
055fd23d5d
round-robin docling-serve across multiple base_urls
2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
5ccbadde0a
Handle shutdown more gracefully, by stopping pollers and cancelling jobs after timeout. Skip periodic poll if a source has pending jobs
2026-05-26 11:44:03 +03:00
Yiorgis Gozadinos
65c309e12b
Handle WebDAV as source
2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
13cbadeb6f
canonical metadata keys: source_revision + content_type, bump to 0.50.0
2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
fa672c298a
logfire spans for the ingester, nested poller/job/document traces
2026-05-26 11:42:42 +03:00
Yiorgis Gozadinos
6d15f1f247
fix URL-decode file:// paths before stat/exists checks
2026-05-26 11:42:42 +03:00
Yiorgis Gozadinos
7ea61a7b10
drop the old monitor, rename serve→mcp, add e2e tests
2026-05-26 11:42:41 +03:00
Yiorgis Gozadinos
1ca3c25a83
logging, sqlite cursors
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
de3b3fa1c9
HTTP control plane
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
75c3896588
additional config, pollers, serve
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
bdcc1caa49
worker pool, rety policy, pipeline
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
4b3641244c
SQLite queue + haiku-ingester CLI skeleton
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
717fc3320b
route create_document_from_source through source adapters
2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
b9637cd625
add HTTP and S3 source adapters + resolve_fetcher
2026-05-26 11:41:53 +03:00
Yiorgis Gozadinos
d0a730ef60
ingester: add source-adapter scaffolding + FS source
2026-05-26 11:41:53 +03:00
Yiorgis Gozadinos
8e9e84132e
drop list_documents/get_document from default RAG skill
2026-05-26 11:35:43 +03:00
Yiorgis Gozadinos
14d9951016
vb
2026-05-21 14:32:13 +03:00
Yiorgis Gozadinos
260b8e8b09
Merge pull request #382 from ggozad/chore/updates
...
Prepare for pydantic-ai 2.0, update haiku.skills
2026-05-21 14:30:55 +03:00
Yiorgis Gozadinos
188d35023f
rotate chat TUI conversation id per launch / clear-chat
2026-05-21 14:21:51 +03:00
Yiorgis Gozadinos
80e3ae0280
clean up deprecation warnings: docling annotations + haiku.skills 0.17.1
2026-05-21 13:45:40 +03:00
Yiorgis Gozadinos
392c74039b
bump pydantic-ai-slim to 1.100 and haiku.skills to 0.17
...
Migrate off two APIs slated for removal in pydantic-ai 2.0:
- Agent(tool_retries=, output_retries=) -> Agent(retries={"tools": ...,
"output": ...}) in the LLM-as-judge evaluator.
- Evaluator.evaluation_name class attribute -> overriding
get_default_evaluation_name() on the citation MRR / MAP evaluators.
haiku.skills 0.17.0 already migrated its internal AGUIAdapter,
MCPToolset and ProcessEventStream usage; no further changes needed on
our side beyond the pin bumps.
2026-05-21 13:20:54 +03:00
Yiorgis Gozadinos
d14bf88096
fix migration failures on pre-v0.48.0 document_items schemas
2026-05-21 10:53:21 +03:00
Yiorgis Gozadinos
6690953480
fix mxbai reranker crash in chat TUI
2026-05-20 17:18:57 +03:00
Yiorgis Gozadinos
b48ac2fdd0
sandbox + cite: guard sort precondition; require rag in cite fallback
2026-05-20 13:02:25 +03:00
Yiorgis Gozadinos
f9392cee46
evaluations: judge model moves to config.evaluations.judge
2026-05-20 12:56:22 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow
2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
a317a951d9
CLI citations: compact panel, inline figures, doc/chunk IDs in footer
2026-05-20 12:00:24 +03:00
Yiorgis Gozadinos
3858ab905a
toc.json nodes carry chunk_ids; fix cite to accept DB-resolvable chunk_ids
2026-05-20 11:37:35 +03:00
Yiorgis Gozadinos
04eeec77d2
per-doc lazy items/toc cache; index document_items for fast per-doc lookup
2026-05-20 10:08:25 +03:00
Yiorgis Gozadinos
e3314e36fd
search: only attach picture bytes from pre-expansion chunks
2026-05-19 15:03:22 +03:00
Yiorgis Gozadinos
e8c61ad0c6
items.jsonl exposes chunk_ids per row; drops position and tree_depth
2026-05-19 14:20:11 +03:00
Yiorgis Gozadinos
6f95e2bc27
Delete the standalone QA and analysis agents
2026-05-19 11:39:20 +03:00
Yiorgis Gozadinos
947a26b391
client.analyze routes through the rag-analysis skill; drop documents= and AnalysisResult.program. Re-record all cassettes that are relevant
2026-05-19 10:46:32 +03:00
Yiorgis Gozadinos
d96d2eeb0f
client.ask routes through the rag skill; always show citations in CLI
2026-05-19 10:19:39 +03:00
Yiorgis Gozadinos
57d077e2c3
cite raises ModelRetry on unresolved chunk_ids
2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
87c551c892
Cover 0.48.0 migration
2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
3b6bef3bfb
Rename get_captions_for_chunk() to get_text_for_refs()
2026-05-18 16:49:44 +03:00
Yiorgis Gozadinos
74dea65f21
analysis.model inherits qa.model when unset; per-skill vision gate
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
52e93d08eb
Explain why no images in sandbox search(). We might want to add show_image() back in the future.
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
c562c397f9
move analysis skill toward search-first; document VFS doc_id vs uri
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
514b0c51f8
surface figure captions in search results, lower search.limit to 5
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
7d99be471f
Add tree-sitter-json on tui
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
e51841b924
Drop list_documents from analysis skill
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
68388032be
consolidate #/pictures/ prefix, tighten CHANGELOG, log migration exc_info
2026-05-18 16:49:43 +03:00
Yiorgis Gozadinos
f13461cdca
route figures through cite, drop show_image
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
387c4f12a9
multimodal sandbox: show_image + picture_refs, drop llm
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
2831030f9b
expose toc.json + heading_level/tree_depth in analysis VFS
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
1360303edf
persist docling heading hierarchy on document_items
2026-05-18 16:48:34 +03:00
Yiorgis Gozadinos
f208730a5b
Merge pull request #374 from tseaver/feat-cli-search_type
...
feat: add '--search-type' option to CLI 'search'
2026-05-18 16:39:28 +03:00
Tres Seaver
8026693450
chore: ruff format
2026-05-18 09:31:38 -04:00
Tres Seaver
ad0161e6da
chore: isort
2026-05-18 09:31:12 -04:00
Tres Seaver
f9ce3bc401
refactor: move 'SearchType' to 'store.models.chunk'
2026-05-18 09:27:34 -04:00
Tres Seaver
d7e96f2658
fix: typo
2026-05-18 09:17:53 -04:00
Tres Seaver
4feac9683a
fix: default 'search_type' to None
2026-05-18 09:17:09 -04:00
Yiorgis Gozadinos
04415a9d09
bump pydantic-monty, refresh deps, cap transformers<5
2026-05-18 16:02:55 +03:00
Yiorgis Gozadinos
1df6530d03
Bump pydantic-ai, prepare for 2.*
2026-05-18 15:14:29 +03:00
Yiorgis Gozadinos
b1e8bdc839
Bump docling
2026-05-18 15:14:29 +03:00
Tres Seaver
b331dd0b76
fix: default 'search_type' to 'None' in 'client.search.search'
...
Apply 'hybrid' default only for text queries.
2026-05-18 06:52:03 -04:00
Tres Seaver
5079205d3d
fix: use 'SearchType' for 'ChunkRepository.search'
2026-05-18 06:50:49 -04:00
Tres Seaver
7953de46b9
fix: declare 'SearchType' for 'search_type' arg
...
fix: allow 'search_type' only for text searches
2026-05-18 06:35:51 -04:00
Tres Seaver
b2ffc50168
WIP: add '--search-type' option to CLI 'search'
2026-05-18 06:35:50 -04:00
Yiorgis Gozadinos
97a7eb246c
vb
2026-05-14 16:03:33 +03:00
Yiorgis Gozadinos
5de4d50622
coverage
2026-05-14 15:54:43 +03:00
Yiorgis Gozadinos
57f273e000
Minor fixes, CI should build
2026-05-14 15:43:36 +03:00
Yiorgis Gozadinos
519afe6709
Add cross-encoder reranking provider
2026-05-14 15:43:35 +03:00
Yiorgis Gozadinos
dbb87e091f
Ensure schema parity for _StagingChunkRecord
2026-05-14 15:11:18 +03:00
Yiorgis Gozadinos
849eca94d0
Make rebuild --embed-only idempotent across crashes
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
a035b0f9e4
Stream rebuild --embed-only through a staging table to bound memory
2026-05-14 15:02:12 +03:00
Yiorgis Gozadinos
862a209137
vb
2026-05-13 16:53:35 +03:00
Yiorgis Gozadinos
a98ddc14b8
Replace picture_description.enabled with processing.pictures enum
2026-05-13 16:22:44 +03:00
Yiorgis Gozadinos
bdab201c42
Merge pull request #371 from ggozad/fix/image-results
...
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:19:09 +03:00
Yiorgis Gozadinos
a6e5664c78
Verify picture bytes before attaching to multimodal tool returns
2026-05-13 16:01:39 +03:00
Yiorgis Gozadinos
072edad62f
Merge pull request #368 from ggozad/feat/relax-compat-check
...
Relax embedding compatibility check to vector_dim only
2026-05-13 15:43:37 +03:00
Yiorgis Gozadinos
701e9d3632
Relax embedding compat check to vector_dim only
2026-05-13 14:44:18 +03:00
Yiorgis Gozadinos
8b135c4d4d
Expose embedding batch size as config
2026-05-13 14:09:47 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through
2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
0f4c4af495
Cache the chunking tokenizer to avoid HF Hub 429
2026-05-13 13:32:24 +03:00
Yiorgis Gozadinos
d512f9fbd1
Disable Typer locals in CLI tracebacks
2026-05-13 13:24:05 +03:00
Yiorgis Gozadinos
bf1cd68f64
Preserve cause chain on docling wrapper exceptions
2026-05-13 12:18:53 +03:00
Yiorgis Gozadinos
674b80e8c9
Support streaming for markdown
2026-05-12 16:01:42 +03:00
Yiorgis Gozadinos
afd4665517
Thread source_uri through URL ingest
2026-05-12 15:13:02 +03:00
Yiorgis Gozadinos
a25d9699ee
Wire per-format options across docling-local
2026-05-12 15:02:59 +03:00
Yiorgis Gozadinos
ef82e6eb08
swap aioboto3 for obstore in the S3 client path
2026-05-11 11:29:42 +03:00
Yiorgis Gozadinos
37099988a6
poll S3 prefixes with S3Watcher and wire into serve
2026-05-11 11:28:40 +03:00
Yiorgis Gozadinos
d009da06d6
support s3:// document sources
2026-05-11 11:28:31 +03:00
Yiorgis Gozadinos
387032511e
Simplify
2026-05-08 12:11:25 +03:00
Yiorgis Gozadinos
788fac422e
Clean up migration
2026-05-08 11:52:57 +03:00
Yiorgis Gozadinos
e55e3d13c8
fix _patch_picture_descriptions silently dropping docling_pages
2026-05-06 12:57:41 +03:00
Yiorgis Gozadinos
75f75ec505
render attached pictures in inspector context modal under qa.model.vision
2026-05-06 12:00:12 +03:00
Yiorgis Gozadinos
7fac35d2af
rename embed_image_query to embed_image; run description check on text path
2026-05-05 16:15:47 +03:00
Yiorgis Gozadinos
1ccb5b5fad
auto-append /v1 to vllm base_url, matching ollama behavior
2026-05-05 16:08:14 +03:00
Yiorgis Gozadinos
4478b7ce2b
test v0.45.0 migration
2026-05-05 16:06:14 +03:00
Yiorgis Gozadinos
c96894a0f0
fix cross-document picture dedup in agent and skill search tools
2026-05-05 15:56:10 +03:00
Yiorgis Gozadinos
61a88edfd8
drop bogus deprecation warning for processing.pictures
2026-05-05 14:46:41 +03:00
Yiorgis Gozadinos
49274ab51b
attach picture bytes through skill search and document the schema
2026-05-05 14:33:57 +03:00
Yiorgis Gozadinos
ffc7b95375
yield per document during rebuild for live progress reporting
2026-05-05 14:04:37 +03:00
Yiorgis Gozadinos
a45d82f206
Update benchmark
2026-05-05 13:58:48 +03:00
Yiorgis Gozadinos
dca3188a48
hydrate documents lazily during rebuild
2026-05-05 12:47:45 +03:00
Yiorgis Gozadinos
ff82d36c2f
add rebuild --descriptions: run VLM over stored picture bytes only
2026-05-05 12:02:46 +03:00
Yiorgis Gozadinos
5fad0aa5c6
warn when picture descriptions silently fail to come back
2026-05-05 11:44:18 +03:00
Yiorgis Gozadinos
2038d43435
collapse pictures enum to picture_description.enabled boolean
2026-05-05 11:16:08 +03:00
Yiorgis Gozadinos
e88321767d
always store picture bytes; collapse converter to a single zip path
2026-05-05 10:40:40 +03:00
Yiorgis Gozadinos
ee41bc676f
remove defensive code
2026-05-04 15:12:27 +03:00
Yiorgis Gozadinos
e8d89aa035
dedup picture-only chunks at the search-result layer
2026-05-04 13:31:15 +03:00
Yiorgis Gozadinos
aa3e9406cf
auto-append /v1 to per-model Ollama base_url, fix flaky tests
2026-05-04 13:12:37 +03:00
Yiorgis Gozadinos
65d9c74224
expose image-as-query through MCP and the CLI.
2026-05-04 11:46:45 +03:00
Yiorgis Gozadinos
ff656504d3
make client.search() polymorphic on query type: str | bytes | PIL.Image.Image. Bytes/PIL queries embed via embed_image_query and run vector-only against the chunks table
2026-05-04 11:22:22 +03:00
Yiorgis Gozadinos
21f12cd770
add vision: bool flag on ModelConfig (default False). Gate the agent search tool's BinaryContent attachment on qa.model.vision so picture bytes are only sent to vision-capable QA models.
2026-05-04 10:47:21 +03:00
Yiorgis Gozadinos
6037f26673
drop mlx
2026-05-03 19:05:49 +03:00
Yiorgis Gozadinos
c9fdad15a9
bump docling-core to >=2.74.1 so that local docling-local and docling-serve chunkers run the same MarkdownTableSerializer
2026-05-03 18:40:44 +03:00
Yiorgis Gozadinos
c9227c649f
emit synthetic picture chunks at ingest under multimodal embedders.
...
processing.chunk() merges text chunks with one synthetic Chunk per PictureItem-with-bytes,
sorted by iterate_items() position so chunk.order is structural.
embed_chunks dispatches on a Chunk._picture_data PrivateAttr
(text through embed_documents, picture through embed_image_query)
2026-05-03 17:04:43 +03:00
Yiorgis Gozadinos
a36f920f5d
drop the batched embed_images method from multi-modal embedders
2026-05-03 16:21:47 +03:00
Yiorgis Gozadinos
37215f1a00
add multimodal embedder support for vllm and mlx
2026-05-03 10:57:56 +03:00
Yiorgis Gozadinos
75c8e83515
Allow uri override on create_document_from_source
2026-04-30 16:36:35 +03:00
Yiorgis Gozadinos
4c9ef81fc6
drop the existing-row-not-found guard from the 0.45.0 picture-bytes migration.
2026-04-30 15:48:03 +03:00
Yiorgis Gozadinos
b290bbe82d
rename document_items repository's _LIGHT_COLUMNS to _METADATA_COLUMNS
2026-04-30 15:43:03 +03:00
Yiorgis Gozadinos
875c635a77
have rebuild --rechunk re-chunk from the stored docling blob instead of re-converting from the markdown export
2026-04-30 15:30:24 +03:00
Yiorgis Gozadinos
7831057339
add processing.pictures enum (none|description|image), replacing the implicit pair of generate_picture_images+picture_description.enabled flags
2026-04-30 14:26:14 +03:00
Yiorgis Gozadinos
b01c649684
surface picture image bytes in SearchResult and emit multimodal ToolReturn from the agent search tool
2026-04-30 12:43:05 +03:00
Yiorgis Gozadinos
6a77ce92a9
extract picture bytes to document_items.picture_data at ingest, strip them from the docling_document blob, and add 0.45.0 migration to backfill existing
...
databases
2026-04-30 11:26:37 +03:00
Yiorgis Gozadinos
191cdc636b
retrieve picture image bytes via referenced+zip path, fix docling-serve not returning images even when set to do so. Remove xfail from relevant test
2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
dc16f74b58
add picture_data column to document_items
2026-04-30 11:23:29 +03:00
Yiorgis Gozadinos
29bc42514e
vb
2026-04-29 13:45:36 +03:00
Yiorgis Gozadinos
5f08a2a8ae
Adapt rag SKILL.md in order to improve citation rate
2026-04-28 14:51:28 +03:00
Yiorgis Gozadinos
7685a592c2
bump haiku.skills to 0.16.0
2026-04-28 12:03:14 +03:00
Yiorgis Gozadinos
6807f082ec
vb
2026-04-25 09:35:47 +03:00
Tres Seaver
0d70f4aec6
fix: pass absolute 'db_path' to 'lancedb.connect_async'
...
Closes #354 .
2026-04-24 17:59:01 -04:00
Yiorgis Gozadinos
81ba248483
vb
2026-04-24 17:47:57 +03:00
Yiorgis Gozadinos
821b7361e9
Push document-id filter into chunk search query
2026-04-24 16:07:07 +03:00
Yiorgis Gozadinos
31fc22ce9d
List tables consistently
2026-04-24 15:57:07 +03:00
Yiorgis Gozadinos
cd4224a727
drop private title/embedding forwarders from HaikuRAG
2026-04-24 15:52:50 +03:00
Yiorgis Gozadinos
b23bad32c3
Check table_names() before dropping chunks in recreate_embeddings_table
2026-04-24 15:32:06 +03:00
Yiorgis Gozadinos
729899299b
Log post-rebuild vacuum failures and clarify vacuum-await scope
2026-04-24 15:31:36 +03:00
Yiorgis Gozadinos
258ffa41fa
Improve test coverage
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
aca3ccc29f
Close connections if __aenter__ fails mid-initialization
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
3224d1f20c
Extract document orchestration into client/documents.py
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
cb02a2c3d8
Extract rebuild machinery into client/rebuild.py
2026-04-24 14:43:59 +03:00
Yiorgis Gozadinos
528acb79a3
Extract LLM agents into client/agents.py
2026-04-24 14:43:58 +03:00
Yiorgis Gozadinos
12939f1444
Extract search, expand_context, visualize_chunk into client/search.py
2026-04-24 14:43:58 +03:00
Yiorgis Gozadinos
d4ecebf896
Extract processing primitives into client/processing.py
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
b5e480bfa1
Extract title generation into client/titles.py
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
e814459e1f
Extract download_models into client/downloads.py
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
43f2774c14
Turn client.py into a client/ package
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
d0ed9e213f
Pin FTS search column to content_fts
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
a74c3a6f9e
Satisfy ty by adding a typed to_pydantic helper
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
b6ea07d2de
Track all in-flight background vacuum tasks, not just the last one
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
57385506c1
Fix is_new_db detection when create=True is passed on an existing DB
2026-04-24 14:42:53 +03:00
Yiorgis Gozadinos
82fd10e0ee
Migrate LanceDB to native async API
...
Convert all LanceDB operations from sync calls wrapped in async
functions to the native async API (connect_async, AsyncConnection,
AsyncTable, AsyncQuery). Database I/O no longer blocks the event loop.
- Store and HaikuRAG use async context managers (async with). Store
initialization is deferred to __aenter__; direct construction
without async with is no longer supported.
- Index creation uses config objects (FTS, BTree, IvfPq) instead of
string-based index_type parameter.
- Upgrade callbacks are async.
- HaikuRAG tracks background vacuum tasks and awaits them in __aexit__
and before destructive rebuild operations to avoid races with
concurrent table mutations.
- temp_db_path fixture uses pytest's tmp_path for reliable async
cleanup.
2026-04-24 14:42:52 +03:00
Yiorgis Gozadinos
84c5391259
Merge pull request #353 from ggozad/feat/better-visual-highlight
...
Tone down the yellow colour in visual highlights
2026-04-24 14:41:27 +03:00
Yiorgis Gozadinos
cb9360ef7b
fix chat TUI citation rendering after state flattening
2026-04-24 13:48:57 +03:00
Yiorgis Gozadinos
a833e58d54
Tone down the yellow colour in visual highlights
2026-04-24 10:31:43 +03:00
Yiorgis Gozadinos
4f058bf484
vb
2026-04-22 17:00:03 +03:00
Yiorgis Gozadinos
ae2b8461be
flatten skill state citations from list[list[str]] to list[str]
2026-04-22 16:50:13 +03:00
Yiorgis Gozadinos
bb9e7b28ca
vb
2026-04-22 16:18:44 +03:00
Yiorgis Gozadinos
206c4ffcf2
address review: top-level AnalysisRunDeps import, asserts, e2e tests
...
- Move AnalysisRunDeps to the module-level import in _tools.py.
- Swap the RuntimeError guards in _require_rag and execute_code for
plain asserts with the same diagnostic messages. Apply the same
treatment to the ty-reachability guard in Sandbox._ensure_initialized.
- Add end-to-end lifespan tests for both skills that drive _run_skill
with TestModel, covering the sub-agent entry path that the existing
tool-level tests bypass.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 15:04:24 +03:00
Yiorgis Gozadinos
9067b89d2f
fix convert() misreading text content that starts with a URL
2026-04-22 14:47:23 +03:00
Yiorgis Gozadinos
f0016ebcd2
scope citations, searches, and executions to the current invocation
2026-04-22 13:53:39 +03:00
Yiorgis Gozadinos
4a9dd9b49a
persist sandbox variables across execute_code calls within one invocation
2026-04-22 13:30:38 +03:00
Yiorgis Gozadinos
4e9c02afc2
open one HaikuRAG client per skill invocation via lifespan
2026-04-22 13:13:42 +03:00
Yiorgis Gozadinos
8a23108fbd
vb
2026-04-20 16:36:32 +03:00
Yiorgis Gozadinos
eb4e1721ce
remove dead code, add tool descriptions in frontend, update changelog
2026-04-20 15:38:17 +03:00
Yiorgis Gozadinos
ff8ad0879c
fix context expansion: respect section boundaries, remove max_context_items
2026-04-20 15:18:16 +03:00
Yiorgis Gozadinos
af7731f4e1
bulk-fetch items.jsonl via lazy cache to avoid per-document query timeout
2026-04-20 14:25:16 +03:00
Yiorgis Gozadinos
cd0c21c996
remove cited_chunks from analysis agent, hoist _deny_write out of loop
2026-04-20 14:05:10 +03:00
Yiorgis Gozadinos
579e609ae1
revert REPL persistence: fresh sandbox per execute_code call
2026-04-20 12:48:05 +03:00
Yiorgis Gozadinos
9d921b13fe
reset sandbox per skill invocation to prevent state leaks
2026-04-20 12:18:14 +03:00
Yiorgis Gozadinos
167c837514
repository methods, module-level executor, read-only VFS
2026-04-20 11:57:39 +03:00
Yiorgis Gozadinos
27c5defdbb
Improve analysis SKILL.md
2026-04-20 11:38:09 +03:00
Yiorgis Gozadinos
4d75943df5
use MontyRepl for persistent variables across execute_code calls
2026-04-20 10:47:50 +03:00
Yiorgis Gozadinos
68a9f191d2
Remove documents from state, fix tests
2026-04-20 09:48:18 +03:00
Yiorgis Gozadinos
d52f453c44
flatten skill architecture: replace ask/analyze/research with direct tools
2026-04-17 18:35:17 +03:00
Yiorgis Gozadinos
20e40d75f9
add collapsible program display to chat TUI
...
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 18:35:17 +03:00
Yiorgis Gozadinos
7d98d0ec57
add citation support to analysis agent
2026-04-17 18:35:16 +03:00
Yiorgis Gozadinos
cf89ff55cd
When setting --model, set all subagents as well
2026-04-17 18:35:16 +03:00
