Commit graph

49 commits

Author SHA1 Message Date
Yiorgis Gozadinos
fdb5710491
Ask and analyze across several databases
Chunk 2 gave search a configured set to fan out over. ask and analyze
covered one database still: the RAG capability had no way to be told which
databases a question spanned, and the analysis sandbox mounted one
document tree.

The selection travels as sources on EvidenceState, beside the filter it
scopes with, so both capabilities read it the same way. clients_covering
is the one rule that turns a selection into clients, used by search, the
sandbox mount and the cite fallback, so a question scoped to some
databases cannot search, mount or cite another. Citations carry the
database they came from, and format_for_agent names it, so the model can
attribute evidence while it answers rather than only afterwards.

The sandbox keeps one flat /documents/{id}/ namespace and resolves each id
to the client holding it, which rests on ids being UUID4. A database
copied from another breaks that, so an id held twice is refused rather
than resolved to whichever arrived last.

On the CLI, search, ask and analyze cover the configured set and label
each result with its database. Every other command works on one, named
with --database NAME (a name reaches a database behind a URI, which --db
cannot) or --db PATH, and refuses a set it cannot choose from instead of
silently reading the default database. Cold databases open together, so a
first query costs the slowest open rather than their sum.
2026-08-24 10:03:46 +03:00
Yiorgis Gozadinos
2310b7a8b3
Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
Yiorgis Gozadinos
72ef18e39d
Reject unknown and out-of-range configuration values
Every section inherited plain BaseModel, so unknown keys were dropped
silently: providers.docling_serve.timeout was documented for months while
being ignored, and a typo in any setting took the default. Sections now
derive from ConfigModel, which forbids extras, so a stale or misspelled key
fails with its path. This already found search.context_radius in a live app
config and providers.vllm in soliplex's example.

converter, chunker and chunker_type are Literals. Sizes, limits,
dimensions, token budgets, attempt counts and breaker thresholds must be
positive; retention, delays, intervals and cooldowns non-negative;
similarity_threshold within 0-1; port within 0-65535. port 0 keeps its
OS-assigned meaning and worker_count allows 0 for an API-and-reaper-only
process.

get_reranker caught ImportError and returned None, so a configured reranker
whose extra was missing silently disappeared. It now propagates.
raise_missing_extra names the install command and re-raises when the failure
came from inside an installed package, so a broken transitive import is not
reported as a missing one. zeroentropy imported bare and now guards like the
others.

The haiku.rag package declares the jina extra. jina-local already worked
there through cross-encoder's transitive transformers and torch; the
resolved package set is unchanged, but the support is now promised rather
than inherited.

Provider fields stay unconstrained: get_model ends in a pass-through to
pydantic-ai for any provider it supports, so a Literal there would reject
valid configurations.
2026-08-19 15:32:51 +03:00
Yiorgis Gozadinos
e8f00fcff4
Make get_config the only configuration lookup
haiku.rag.config exported two configuration instances: the lazy _config
behind get_config/set_config, and Config, loaded at import time. Nothing
linked them, and eleven signatures captured Config as a default argument,
so set_config could not reach the factories, the client, the store or the
MCP server. reranking/base.py went further and snapshotted the configured
reranker name into a class attribute at import.

Config is removed. Internal defaults are config: AppConfig | None = None,
resolved through get_config() per call. RerankerBase._model is None and
CohereReranker takes its model name as an argument, like every other
reranker.

The suite patched attributes on Config while production read the instance
get_config() returns, a different object, so those patches were no-ops
waiting to happen. They now go through get_config().
2026-08-19 14:43:40 +03:00
Yiorgis Gozadinos
ae345cc39f
Map thinking onto Pydantic AI's unified setting 2026-07-27 18:26:42 +03:00
Yiorgis Gozadinos
4c5050d161
Accept images on ask/analyze 2026-07-25 10:23:55 +03:00
Yiorgis Gozadinos
55351a8829
Merge leading system messages for OpenAI-compatible endpoints 2026-07-24 16:57:55 +03:00
Yiorgis Gozadinos
9deb1f2bd4
replace haiku.skills with native Pydantic AI capabilities 2026-07-24 15:26:17 +03:00
Yiorgis Gozadinos
0e271eaf4b
Remove --before/--at time travel 2026-07-16 13:11:33 +03:00
Yiorgis Gozadinos
8c57a3ca99
Drop the multi-agent research workflow 2026-05-20 12:46:48 +03:00
Yiorgis Gozadinos
a317a951d9
CLI citations: compact panel, inline figures, doc/chunk IDs in footer 2026-05-20 12:00:24 +03:00
Yiorgis Gozadinos
c6cd847299
Add ModelConfig.extra_body for raw provider pass-through 2026-05-13 13:45:09 +03:00
Yiorgis Gozadinos
aa3e9406cf
auto-append /v1 to per-model Ollama base_url, fix flaky tests 2026-05-04 13:12:37 +03:00
Yiorgis Gozadinos
e2bac1e887
deduplicate escape_sql_string into utils 2026-04-16 12:11:53 +03:00
Yiorgis Gozadinos
15d94bfe7f
replace UUIDs with readable identifiers in citation text 2026-04-01 11:44:17 +03:00
Yiorgis Gozadinos
aa9fdf051d
Remove support for structured output 2026-03-24 14:11:43 +02:00
Yiorgis Gozadinos
fd327996b8
Configurable judge and reflect models for evaluations 2026-03-18 17:14:11 +02:00
Yiorgis Gozadinos
28e90e5a87
Auto-detect structured output mode from model profile. Remove the structured_output config field from ModelConfig. 2026-03-04 14:51:09 +02:00
Yiorgis Gozadinos
fdd7c21757
Add configurable structured output mode (tool vs native) 2026-03-04 13:35:32 +02:00
Yiorgis Gozadinos
524647c501
Add unified RAG skill with session context and reuse existing tools; 2026-02-20 16:36:47 +02:00
Yiorgis Gozadinos
72c8f6e1b1
Additional tests 2026-02-13 18:08:46 +02:00
Yiorgis Gozadinos
f05159859c
Additional fixes to remove type ignores 2026-01-19 16:00:18 +02:00
Yiorgis Gozadinos
5e55c0df54
replace pyright with ty type checker 2026-01-19 15:50:14 +02:00
Yiorgis Gozadinos
affffc1013
Extract get_package_versions() for info & modals 2026-01-14 18:24:41 +02:00
Yiorgis Gozadinos
7cc561d1db
Refactor to make agents a top-level module. Bring in the conversational agent from the app 2026-01-12 12:37:08 +02:00
Yiorgis Gozadinos
1616891c40
Fix non-reasoning OpenAI models that do not support reasoning config 2026-01-07 18:47:31 +02:00
Yiorgis Gozadinos
08fb99c6f8
Add prompt customization and domain preable 2025-12-26 13:34:51 +02:00
Yiorgis Gozadinos
3141c91023
Add base_url to ModelConfig and EmbeddingModelConfig. Deprecate vllm and lm_studio configs, now through open ai 2025-12-24 12:33:44 +02:00
Yiorgis Gozadinos
1e5eebfbf0
Checkout version logic and datetime utils 2025-12-19 12:03:16 +02:00
Yiorgis Gozadinos
e8e67d336c
Fix get_model() uses where default Config is forced 2025-12-18 15:10:31 +02:00
Yiorgis Gozadinos
45de9bf0d5
Flatten graph structure, simplify 2025-12-18 11:41:57 +02:00
Yiorgis Gozadinos
011f527579
Use rich rendering for citations in cli 2025-12-12 10:02:30 +02:00
Yiorgis Gozadinos
17c7147a49
Remove pre-processor, no longer needed 2025-12-08 15:56:01 +02:00
Yiorgis Gozadinos
75a086bae0
Visualize chunk command, format citations to show chunk id 2025-12-08 15:55:28 +02:00
Yiorgis Gozadinos
d808c6c425
Simplify citations in qa & graph agents 2025-12-08 15:55:27 +02:00
Yiorgis Gozadinos
80447c9024
Make download_models() a HaikuRAG client method. Update app to show progress of what happens. 2025-12-04 16:30:26 +02:00
Yiorgis Gozadinos
3525fae625
Use EmbeddingModelConfig similar to ModelConfig for embeddings 2025-12-02 11:55:31 +02:00
Yiorgis Gozadinos
ed53ec461a
Convert prefetch_models() to async 2025-11-26 18:08:48 +02:00
Yiorgis Gozadinos
0303e6a8cd
Support LMStudio 2025-11-26 10:31:40 +02:00
Yiorgis Gozadinos
e83978f88b
Revert embeddings to use flat config 2025-11-25 12:51:58 +02:00
Yiorgis Gozadinos
a9701616c8
Rename model to name under model 2025-11-25 12:28:43 +02:00
Yiorgis Gozadinos
1bb7b6d5bd
Add support for per-model configuration settings including thinking, temperature and max_tokens 2025-11-25 12:14:24 +02:00
Yiorgis Gozadinos
fdcf9a1a12
Show index stats in cli info command 2025-11-21 13:54:04 +02:00
Yiorgis Gozadinos
541552215e
Switch to using HuggingFace tokenizers 2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
7ba1c2376e
Introduce converters for supporting more than local docling document conversion. Transform existing FileReader and utils to "docling-local" converter 2025-11-17 13:05:05 +02:00
Yiorgis Gozadinos
b8bc6d9b68
Let ruff know about our package structure 2025-11-05 17:47:27 +02:00
Yiorgis Gozadinos
5cd951a5f3
When checking for updates, use haiku.rag-slim as the package, since it is always installed 2025-11-05 10:30:59 +02:00
Yiorgis Gozadinos
b02514dc82
Make docling optional, depend on docling-core 2025-11-04 18:15:04 +02:00
Yiorgis Gozadinos
2f9c907031
Restructure into uv workspace to support minimal and full installations 2025-11-04 17:59:12 +02:00
Renamed from src/haiku/rag/utils.py (Browse further)