Commit graph

1934 commits

Author SHA1 Message Date
Yiorgis Gozadinos
515f42dd5a
Add custom ingester sources via entry points 2026-06-16 10:54:24 +03:00
Yiorgis Gozadinos
2ce7d10c51
Merge pull request #441 from ggozad/fix/sqlite-pool
Widen SQLite ingester queue pool to serve concurrent connections
2026-06-16 09:41:11 +03:00
Yiorgis Gozadinos
7e20b47e98
Widen SQLite ingester queue pool to serve concurrent connections 2026-06-16 09:25:01 +03:00
Yiorgis Gozadinos
f94c419c31
cl 2026-06-15 10:21:09 +03:00
Yiorgis Gozadinos
6440698b80
Merge pull request #440 from ggozad/fix/webdav-follow-redirects
Follow HTTP redirects in the WebDAV source
2026-06-15 10:10:44 +03:00
Yiorgis Gozadinos
fac62cb347
Follow HTTP redirects in the WebDAV source 2026-06-15 09:59:13 +03:00
Yiorgis Gozadinos
84b778b4ea
Merge pull request #439 from ggozad/feat/ingestor-meta
Ingester metadata providers
2026-06-15 08:51:52 +03:00
Yiorgis Gozadinos
cc73a8629a
Wire metadata providers into ingester ingestion 2026-06-15 08:41:26 +03:00
Yiorgis Gozadinos
5722260857
Add metadata-provider discovery for the ingester 2026-06-15 08:05:45 +03:00
Yiorgis Gozadinos
61f27085f9
Merge pull request #438 from ggozad/fix/pdfium-concurrency
Serialize all in-process pdfium access under a shared lock
2026-06-14 11:08:26 +03:00
Yiorgis Gozadinos
70c9c2778f
Serialize all in-process pdfium access under a shared lock 2026-06-14 10:50:42 +03:00
Yiorgis Gozadinos
8d9dbd0abc
Merge pull request #435 from ggozad/feat/ingestor-vacuuming
Fix ingestion disk bloat: split mutable document  attributes into a document_meta table
2026-06-12 10:29:38 +03:00
Yiorgis Gozadinos
885e7b7ce7
coverage 2026-06-12 10:17:19 +03:00
Yiorgis Gozadinos
6c04dd5881
docs 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
bfc51b8b28
urface document_meta in info, history, and the inspector 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
2b8b9477b7
Serialize and roll back the multi-table document delete 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
f2a5ac4246
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
5d5d87d44c
Throttle background auto-vacuum to at most once per 5 minutes 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
df8af54298
rebase from main 2026-06-12 10:17:18 +03:00
Yiorgis Gozadinos
3366a6d383
Split meta document attributes into a document_meta table 2026-06-12 10:17:17 +03:00
Yiorgis Gozadinos
4bc52f0710
Serialize multi-table document writes and bound update version churn 2026-06-12 10:16:39 +03:00
Yiorgis Gozadinos
ab9fea4833
Merge pull request #437 from bd-mkt/invalid-attachments
fix for issue with parsing pdf attachment such as *.joboptions files
2026-06-12 09:53:23 +03:00
Yiorgis Gozadinos
8ac670714d
Tidy PDF attachment extension fix 2026-06-12 09:44:11 +03:00
bryan davis
bce24e84a0
fix for issue with parsing pdf attachment such as *.joboptions files 2026-06-11 16:52:07 -05:00
Yiorgis Gozadinos
88696cc6f1
vb 2026-06-11 09:50:33 +03:00
Yiorgis Gozadinos
77eb2fc2f9
Merge pull request #434 from ggozad/feat/secrets-from-env
Expand ${VAR} environment references in YAML config
2026-06-11 09:48:49 +03:00
Yiorgis Gozadinos
ceae56562f
Expand ${VAR} environment references in YAML config 2026-06-11 09:39:54 +03:00
Yiorgis Gozadinos
a64524d5ba
Merge pull request #432 from tseaver/feat-431-control-plane-behind-proxy
feat(ingester): serve control plane under a configurable base path
2026-06-11 09:38:43 +03:00
Yiorgis Gozadinos
90e9a6bc17
changelog entry + bare-prefix redirect for ingester root_path 2026-06-11 09:30:22 +03:00
Tres Seaver
b78f0ae9ed
feat(ingester): serve control plane under a configurable base path
Add `ingester.api.root_path` so the HTTP control plane (dashboard + API)
can be reverse-proxied behind a sub-path (e.g. /ingester/) on a shared
origin, instead of needing nginx sub_filter URL-rewriting.

- APIConfig.root_path: normalized ('', or single leading slash, no trailing
  slash) via a field_validator; validate_assignment so CLI overrides
  normalize the same way as config-file values.
- Forwarded to FastAPI(root_path=) and uvicorn.Config(root_path=) so
  OpenAPI/docs links are prefix-aware.
- Dashboard route injects a <base href> matching root_path; all dashboard
  fetches are now base-relative, so they resolve under the prefix while
  staying identical at the root.
- `serve --root-path` CLI flag.
- Docs: "Behind a reverse proxy" section with an nginx example.

Closes #431

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:02:43 -04:00
Yiorgis Gozadinos
bdad908a60
vb 2026-06-09 15:39:14 +03:00
Yiorgis Gozadinos
2aa3528cd0
Merge pull request #430 from ggozad/feat/ingestor-ui-info
Ingester dashboard: Database info and full-config panels
2026-06-09 12:58:08 +03:00
Yiorgis Gozadinos
54524a4ea3
Gate ingester control-plane access logs to DEBUG 2026-06-09 12:46:17 +03:00
Yiorgis Gozadinos
4c186b01c7
Add Database and Configuration panels to the ingester dashboard 2026-06-09 12:28:13 +03:00
Yiorgis Gozadinos
c62166b26e
Add /database and /config endpoints to the ingester API 2026-06-09 12:11:33 +03:00
Yiorgis Gozadinos
0be2e5b24f
Extract gather_database_info shared by the info command 2026-06-09 11:09:28 +03:00
Yiorgis Gozadinos
d45f8f5de8
Merge pull request #429 from ggozad/feat/batch-document-creation
Batch document import (import_documents)
2026-06-09 10:44:01 +03:00
Yiorgis Gozadinos
a7ed405d75
Add HaikuRAG.import_documents for batch document import 2026-06-09 10:23:37 +03:00
Yiorgis Gozadinos
b43843f862
Batch-capable document and document-item repositories 2026-06-09 10:01:20 +03:00
Yiorgis Gozadinos
f83aad3cc4
Merge pull request #428 from ggozad/fix/per-source-circuit-breaker
Make the ingester worker circuit breaker per-source
2026-06-09 10:00:35 +03:00
Yiorgis Gozadinos
bd548837e5
Make the ingester worker circuit breaker per-source 2026-06-09 09:41:54 +03:00
Yiorgis Gozadinos
46b8fa3fef
vb 2026-06-08 12:24:04 +03:00
Yiorgis Gozadinos
7941c6738c
Merge pull request #427 from ggozad/feat/t2-ragbench
Add T²-RAGBench evaluation
2026-06-08 12:15:06 +03:00
Yiorgis Gozadinos
9ef25e2e53
Order benchmarks docs ORB, T², Wix 2026-06-08 11:51:21 +03:00
Yiorgis Gozadinos
295570797c
Disable Logfire scrubbing in evaluations 2026-06-08 11:51:13 +03:00
Yiorgis Gozadinos
e9bd56467c
Add T²-RAGBench leaderboard submission exporter 2026-06-08 11:51:02 +03:00
Yiorgis Gozadinos
a3a73f1331
Add --filter-ids to run QA on a case-id subset 2026-06-08 09:36:01 +03:00
Yiorgis Gozadinos
d386d7f900
Bump haiku.skills to 0.17.2 2026-06-06 14:52:05 +03:00
Yiorgis Gozadinos
cd77bd9889
Bound analysis execute_code calls to avoid request-limit nulls 2026-06-06 14:52:05 +03:00
Yiorgis Gozadinos
1d7d540270
Match the numeric scale convention in Number-Match 2026-06-06 14:52:05 +03:00