Commit graph

60 commits

Author SHA1 Message Date
Yiorgis Gozadinos
34180a0fd1
One reference and one placement for a database
DatabaseRef is a name and a location. The configuration places databases
through lancedb.databases alone; with none configured the default is the
entry haiku.rag under storage.data_dir, selectable like any other.
lancedb.uri is removed, and a config carrying it fails to load with the
replacement spelled out. A path passed from Python is valid where the
configuration places nothing and raises AmbiguousDatabaseError beside
lancedb.databases; haiku-rag --db and haiku-ingester --db construct the
scope directly, so a human's override keeps working. Every database
answers to a name, and a database given as a path keeps its own errors.
2026-09-03 15:12:08 +03:00
Yiorgis Gozadinos
22a5b7d17c
Add --full-citations to ask and analyze 2026-08-31 16:31:09 +03:00
Yiorgis Gozadinos
0c67db4459
Accept a /v1 suffix on the vLLM reranker base_url
vllm_base_url moves to utils.py and is shared with the embedder, so the same
endpoint works written either way. Writing /v1 posted to /v1/v1/rerank.
2026-08-28 15:46:42 +03:00
Yiorgis Gozadinos
ad100ecd4d
Reject unknown chat model providers and rename gemini to google 2026-08-28 15:46:20 +03:00
Yiorgis Gozadinos
09a7076b7e
State what the code does, not what it replaced
Comments and docstrings across the branch narrated rejected
alternatives, consequences and history; each now states the current
contract. Renames test_a_legacy_uri_client_keeps_its_error to
test_an_unnamed_database_keeps_its_error. Documents the Sandbox
connection paths, the citation header's database segment, both
AmbiguousDatabaseError conditions on create_app, and run_inspector's
scope parameter. Doc paragraphs added by the branch in python.md,
storage.md and cli.md are one physical line each.
2026-08-28 15:13:52 +03:00
Yiorgis Gozadinos
688d7e708e
State what comments guarantee, not what would go wrong without them
Comments that narrated a failure mode across three or four lines say the
invariant they protect instead: the repository's late embedding, one database
keeping its hybrid scores, one over-fetch decision for a selection, and a cite
fallback that covers exactly what the question covers.
2026-08-28 14:03:05 +03:00
Yiorgis Gozadinos
bfd09f6880
Leave no database reading when a federated fan-out fails
`asyncio.gather` propagates the first failure while its siblings run on, and the
caller unwinding from that closes the set through `async with` — so a sibling
still reading reads through a closed session. Seven fan-outs were affected:
lookup, search, image enrichment, multimodal picture loading, context expansion,
document listing and counting, and the sandbox's document load.

`gather_all` cancels and drains the rest, then re-raises the original exception.
A `TaskGroup` would drain them too but raise an `ExceptionGroup`, which every
caller and both CLIs' exception handlers would have to unwrap. The two
`return_exceptions=True` gathers in session opening and teardown already drain
their children and are left alone.
2026-08-28 12:02:07 +03:00
Yiorgis Gozadinos
64b1096865
Mark a picture no database claims
`format_citations_rich` asked a client covering a set for a picture whose
citation named no database, which raises, losing the whole answer to one figure.
Citations recorded before databases could be named carry no source, so the
figure marker the caller already renders stands in.
2026-08-27 16:01:30 +03:00
Yiorgis Gozadinos
746b663a2f
Correct the multi-database prose
`resolve_citations` documented the behaviour it now rejects: a shared chunk id
resolving to whichever result came last.

The storage page ran two embedding checks together as if the second explained
the first. They are separate: each database against the configuration on open,
and the databases in a selection against each other, which raises in read-only
mode too and does not apply to full-text search.

`several` becomes `multiple` where it names the feature, matching the docs and
`covers_multiple`.
2026-08-27 11:40:52 +03:00
Yiorgis Gozadinos
ea6b864f6e
Mark a truncated preview instead of cutting it silently 2026-08-27 10:31:49 +03:00
Yiorgis Gozadinos
8db447e095
Ask the client what it covers
covers_multiple, source_names, source and reader_for replace the private
state seven modules were reading to work out how many databases they had.
The configured selection is kept intact, so entering a client twice derives
the same database rather than the last derivation.
2026-08-25 14:16:22 +03:00
Yiorgis Gozadinos
fdb5710491
Ask and analyze across several databases
Chunk 2 gave search a configured set to fan out over. ask and analyze
covered one database still: the RAG capability had no way to be told which
databases a question spanned, and the analysis sandbox mounted one
document tree.

The selection travels as sources on EvidenceState, beside the filter it
scopes with, so both capabilities read it the same way. clients_covering
is the one rule that turns a selection into clients, used by search, the
sandbox mount and the cite fallback, so a question scoped to some
databases cannot search, mount or cite another. Citations carry the
database they came from, and format_for_agent names it, so the model can
attribute evidence while it answers rather than only afterwards.

The sandbox keeps one flat /documents/{id}/ namespace and resolves each id
to the client holding it, which rests on ids being UUID4. A database
copied from another breaks that, so an id held twice is refused rather
than resolved to whichever arrived last.

On the CLI, search, ask and analyze cover the configured set and label
each result with its database. Every other command works on one, named
with --database NAME (a name reaches a database behind a URI, which --db
cannot) or --db PATH, and refuses a set it cannot choose from instead of
silently reading the default database. Cold databases open together, so a
first query costs the slowest open rather than their sum.
2026-08-24 10:03:46 +03:00
Yiorgis Gozadinos
2310b7a8b3
Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
Yiorgis Gozadinos
72ef18e39d
Reject unknown and out-of-range configuration values
Every section inherited plain BaseModel, so unknown keys were dropped
silently: providers.docling_serve.timeout was documented for months while
being ignored, and a typo in any setting took the default. Sections now
derive from ConfigModel, which forbids extras, so a stale or misspelled key
fails with its path. This already found search.context_radius in a live app
config and providers.vllm in soliplex's example.

converter, chunker and chunker_type are Literals. Sizes, limits,
dimensions, token budgets, attempt counts and breaker thresholds must be
positive; retention, delays, intervals and cooldowns non-negative;
similarity_threshold within 0-1; port within 0-65535. port 0 keeps its
OS-assigned meaning and worker_count allows 0 for an API-and-reaper-only
process.

get_reranker caught ImportError and returned None, so a configured reranker
whose extra was missing silently disappeared. It now propagates.
raise_missing_extra names the install command and re-raises when the failure
came from inside an installed package, so a broken transitive import is not
reported as a missing one. zeroentropy imported bare and now guards like the
others.

The haiku.rag package declares the jina extra. jina-local already worked
there through cross-encoder's transitive transformers and torch; the
resolved package set is unchanged, but the support is now promised rather
than inherited.

Provider fields stay unconstrained: get_model ends in a pass-through to
pydantic-ai for any provider it supports, so a Literal there would reject
valid configurations.
2026-08-19 15:32:51 +03:00
Yiorgis Gozadinos
e8f00fcff4
Make get_config the only configuration lookup
haiku.rag.config exported two configuration instances: the lazy _config
behind get_config/set_config, and Config, loaded at import time. Nothing
linked them, and eleven signatures captured Config as a default argument,
so set_config could not reach the factories, the client, the store or the
MCP server. reranking/base.py went further and snapshotted the configured
reranker name into a class attribute at import.

Config is removed. Internal defaults are config: AppConfig | None = None,
resolved through get_config() per call. RerankerBase._model is None and
CohereReranker takes its model name as an argument, like every other
reranker.

The suite patched attributes on Config while production read the instance
get_config() returns, a different object, so those patches were no-ops
waiting to happen. They now go through get_config().
2026-08-19 14:43:40 +03:00
Yiorgis Gozadinos
ae345cc39f
Map thinking onto Pydantic AI's unified setting 2026-07-27 18:26:42 +03:00
Yiorgis Gozadinos
4c5050d161
Accept images on ask/analyze 2026-07-25 10:23:55 +03:00
Yiorgis Gozadinos
55351a8829
Merge leading system messages for OpenAI-compatible endpoints 2026-07-24 16:57:55 +03:00
Yiorgis Gozadinos
9deb1f2bd4
replace haiku.skills with native Pydantic AI capabilities 2026-07-24 15:26:17 +03:00
Yiorgis Gozadinos
0e271eaf4b
Remove --before/--at time travel 2026-07-16 13:11:33 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow 2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
a317a951d9
CLI citations: compact panel, inline figures, doc/chunk IDs in footer 2026-05-20 12:00:24 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through 2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
aa3e9406cf
auto-append /v1 to per-model Ollama base_url, fix flaky tests 2026-05-04 13:12:37 +03:00
Yiorgis Gozadinos
e2bac1e887
deduplicate escape_sql_string into utils 2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
15d94bfe7f
replace UUIDs with readable identifiers in citation text 2026-04-01 11:44:17 +03:00
Yiorgis Gozadinos
aa9fdf051d
Remove support for structured output 2026-03-24 14:11:43 +02:00
Yiorgis Gozadinos
fd327996b8
Configurable judge and reflect models for evaluations 2026-03-18 17:14:11 +02:00
Yiorgis Gozadinos
28e90e5a87
Auto-detect structured output mode from model profile. Remove the structured_output config field from ModelConfig. 2026-03-04 14:51:09 +02:00
Yiorgis Gozadinos
fdd7c21757
Add configurable structured output mode (tool vs native) 2026-03-04 13:35:32 +02:00
Yiorgis Gozadinos
524647c501
Add unified RAG skill with session context and reuse existing tools; 2026-02-20 16:36:47 +02:00
Yiorgis Gozadinos
72c8f6e1b1
Additional tests 2026-02-13 18:08:46 +02:00
Yiorgis Gozadinos
f05159859c
Additional fixes to remove type ignores 2026-01-19 16:00:18 +02:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker 2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
affffc1013
Extract get_package_versions() for info & modals 2026-01-14 18:24:41 +02:00
Yiorgis Gozadinos
7cc561d1db
Refactor to make agents a top-level module. Bring in the conversational agent from the app 2026-01-12 12:37:08 +02:00
Yiorgis Gozadinos
1616891c40
Fix non-reasoning OpenAI models that do not support reasoning config 2026-01-07 18:47:31 +02:00
Yiorgis Gozadinos
08fb99c6f8
Add prompt customization and domain preable 2025-12-26 13:34:51 +02:00
Yiorgis Gozadinos
3141c91023
Add base_url to ModelConfig and EmbeddingModelConfig. Deprecate vllm and lm_studio configs, now through open ai 2025-12-24 12:33:44 +02:00
Yiorgis Gozadinos
1e5eebfbf0
Checkout version logic and datetime utils 2025-12-19 12:03:16 +02:00
Yiorgis Gozadinos
e8e67d336c
Fix get_model() uses where default Config is forced 2025-12-18 15:10:31 +02:00
Yiorgis Gozadinos
45de9bf0d5
Flatten graph structure, simplify 2025-12-18 11:41:57 +02:00
Yiorgis Gozadinos
011f527579
Use rich rendering for citations in cli 2025-12-12 10:02:30 +02:00
Yiorgis Gozadinos
17c7147a49
Remove pre-processor, no longer needed 2025-12-08 15:56:01 +02:00
Yiorgis Gozadinos
75a086bae0
Visualize chunk command, format citations to show chunk id 2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
d808c6c425
Simplify citations in qa & graph agents 2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
80447c9024
Make download_models() a HaikuRAG client method. Update app to show progress of what happens. 2025-12-04 16:30:26 +02:00
Yiorgis Gozadinos
3525fae625
Use EmbeddingModelConfig similar to ModelConfig for embeddings 2025-12-02 11:55:31 +02:00
Yiorgis Gozadinos
ed53ec461a
Convert prefetch_models() to async 2025-11-26 18:08:48 +02:00
Yiorgis Gozadinos
0303e6a8cd
Support LMStudio 2025-11-26 10:31:40 +02:00