Commit graph

26 commits

Author SHA1 Message Date
Yiorgis Gozadinos
3814b2bc78
Report a stemless --db path as a usage error and separate the two lancedb.uri migrations
An empty lancedb.uri, which init-config used to emit, is told to remove the
key; only a non-empty value is told the databases spelling. haiku-rag --db
and haiku-ingester --db turn a path with no stem into typer.BadParameter.
2026-09-03 15:12:09 +03:00
Yiorgis Gozadinos
34180a0fd1
One reference and one placement for a database
DatabaseRef is a name and a location. The configuration places databases
through lancedb.databases alone; with none configured the default is the
entry haiku.rag under storage.data_dir, selectable like any other.
lancedb.uri is removed, and a config carrying it fails to load with the
replacement spelled out. A path passed from Python is valid where the
configuration places nothing and raises AmbiguousDatabaseError beside
lancedb.databases; haiku-rag --db and haiku-ingester --db construct the
scope directly, so a human's override keeps working. Every database
answers to a name, and a database given as a path keeps its own errors.
2026-09-03 15:12:08 +03:00
Yiorgis Gozadinos
dafc15978e
Translate every database-opening failure in haiku-ingester
`ConfigMismatchError` and `SourceUnavailableError` escaped the entry point as
tracebacks rather than a message and exit code 1. The entry-point test is
parametrized over the translated types, so the two CLIs' lists cannot drift
apart silently again.
2026-08-28 09:59:37 +03:00
Yiorgis Gozadinos
b6bc54c69e
Answer for an unknown database name with one type
A name nothing covers raised `KeyError` in four places and
`AmbiguousDatabaseError` in a fifth, so a caller had to catch both and neither
name said what happened. `UnknownDatabaseError` is all of them, exported from
`haiku.rag.store` beside the other errors.

It subclasses `KeyError`, since selecting by name is a lookup, and prints its
message plainly rather than quoted as a missing key. Both CLIs turn it into the
same clean exit they already gave the others.
2026-08-27 18:04:27 +03:00
Yiorgis Gozadinos
206d29b74a
Let the configuration place the ingester's database
The ingester manufactured `data_dir / haiku.rag.lancedb` whenever `--db` was
absent and passed it to HaikuRAG. A path is now an explicit override that
clears the configured URI, so a `lancedb.uri` deployment wrote to local disk
while the control plane reported on the remote.

`IngesterApp` resolves the databases it works on once, in its constructor, and
both the client it opens and the control plane read that scope. `--db` names
one directly and nothing stands in for it, so a configured set is refused
rather than guessed, and `AmbiguousDatabaseError` joins the errors `cli()`
turns into a clean exit.
2026-08-27 14:00:21 +03:00
Yiorgis Gozadinos
6186dae83f
Move ingester telemetry setup behind argument parsing 2026-08-13 11:37:36 +03:00
Lawrence Akka
d4d6414bc4 Similar fix for haiku.rag.ingester, and formatting
- Add regression tests
- Update Changelog
2026-08-12 15:20:11 +01:00
Yiorgis Gozadinos
463c55673e
improve coverage 2026-06-22 12:59:44 +03:00
Yiorgis Gozadinos
baf8decb27
Show progress for run-batch drains 2026-06-22 12:51:38 +03:00
Yiorgis Gozadinos
6e85bbbafe
Make run-batch CLI validation output stable 2026-06-22 12:32:26 +03:00
Yiorgis Gozadinos
68bbf94577
tighten run-batch manifest replay validation 2026-06-22 11:58:32 +03:00
Yiorgis Gozadinos
7251d104c4
Add run-batch manifest replay 2026-06-22 11:39:12 +03:00
Yiorgis Gozadinos
a3b542dba8
Add run-batch dry-run manifest output 2026-06-22 11:33:26 +03:00
Tres Seaver
b78f0ae9ed
feat(ingester): serve control plane under a configurable base path
Add `ingester.api.root_path` so the HTTP control plane (dashboard + API)
can be reverse-proxied behind a sub-path (e.g. /ingester/) on a shared
origin, instead of needing nginx sub_filter URL-rewriting.

- APIConfig.root_path: normalized ('', or single leading slash, no trailing
  slash) via a field_validator; validate_assignment so CLI overrides
  normalize the same way as config-file values.
- Forwarded to FastAPI(root_path=) and uvicorn.Config(root_path=) so
  OpenAPI/docs links are prefix-aware.
- Dashboard route injects a <base href> matching root_path; all dashboard
  fetches are now base-relative, so they resolve under the prefix while
  staying identical at the root.
- `serve --root-path` CLI flag.
- Docs: "Behind a reverse proxy" section with an nginx example.

Closes #431

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-09 23:02:43 -04:00
Yiorgis Gozadinos
5cc32f111e
Mask the dburi password in queue init/migrate output 2026-06-03 14:42:32 +03:00
Yiorgis Gozadinos
44089e5b1f
Support a dburi for the ingester queue (SQLite + Postgres)
Migrate the ingester queue storage from raw aiosqlite to SQLAlchemy Core
async. The backend is chosen by ingester.queue.dburi: a SQLAlchemy async
URL points the queue at a database server, and SQLite remains the default
when unset. The Postgres path claims jobs with FOR UPDATE SKIP LOCKED so
multiple ingester processes can share one queue; SQLite caps the pool to a
single connection to keep the select-then-update claim atomic.
2026-06-03 14:34:09 +03:00
Yiorgis Gozadinos
5a23e4eda6
Surface failed discovery sweeps in run-batch 2026-06-01 10:27:31 +03:00
Yiorgis Gozadinos
5ba7838b71
Add haiku-ingester run-batch, remove run-once 2026-05-29 17:00:59 +03:00
Yiorgis Gozadinos
73f8349a00
Use haiku.rag as telemtry scope 2026-05-26 11:44:47 +03:00
Yiorgis Gozadinos
f4468b65ee
Add --host, --port to ingester cli, update docker compose example 2026-05-26 11:44:45 +03:00
Yiorgis Gozadinos
fa672c298a
logfire spans for the ingester, nested poller/job/document traces 2026-05-26 11:42:42 +03:00
Yiorgis Gozadinos
1ca3c25a83
logging, sqlite cursors 2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
de3b3fa1c9
HTTP control plane 2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
75c3896588
additional config, pollers, serve 2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
bdcc1caa49
worker pool, rety policy, pipeline 2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
4b3641244c
SQLite queue + haiku-ingester CLI skeleton 2026-05-26 11:41:54 +03:00