Commit graph

20 commits

Author SHA1 Message Date
Yiorgis Gozadinos
7f7223e0ac
Default to ollama:qwen3.8
Replaces gpt-oss on ModelConfig, qa.model and processing.title_model, and
ministral-3 on the picture-description model. qa.model.vision follows the
model and is now true.

enable_thinking was gated on the gpt-oss name, so it did nothing for
qwen3.8. With title_model's max_tokens of 100 the reasoning consumed the
whole budget and title generation returned an empty string. The mapping
now applies to any ollama model via reasoning_effort(): false sends
"none", true sends "high". Measured on qwen3.8:27b-mlx, "low" does not
disable thinking and "none" does; gpt-oss is the inverse, its template
has no "none" level, so it keeps "low".

Picture description bypasses get_model -- docling posts the request
itself from a params dict -- so the flag was inert on that path too.
vlm_api_params() carries reasoning_effort into both converters' request
bodies. At max_tokens 200 the description survived either way, but the
switch cut completion tokens from 141 to 45.

test_search_tool_skips_binary_content_when_qa_model_is_text_only asserted
the vision default rather than setting it; it now configures vision=False
itself.

docs/benchmarks.md keeps ministral-3: those are recorded measurements.
2026-09-04 12:36:36 +03:00
Yiorgis Gozadinos
61df6b65a6
Guard against an FTS index that covers no rows
lance serves unsorted results with matching rows dropped when a
declared FTS index has indexed nothing, or when a populated table has
no FTS index at all. doctor fails on both: vacuum remediation for an
existing index, rebuild --embed-only for a missing one, since optimize
never creates an index. The chunk repository warns once per repository
on the first FTS or hybrid search against either state; a failing
coverage check is logged and never fails the search. Removes
_ensure_fts_index, which had no callers.
2026-08-31 18:42:31 +03:00
Yiorgis Gozadinos
2310b7a8b3
Add per-endpoint api_key to model and embedding config 2026-08-23 23:54:40 +03:00
Yiorgis Gozadinos
629e1ba4ea
Split the store module by responsibility
engine.py held four unrelated things: what the tables are, how to open a
connection, how to read a database's state, and the Store that coordinates
writes. At 1240 lines the Store's own concerns — locks, migrations, vacuum,
tags — were hard to find among them.

Table records, Arrow schemas, index_specs, ensure_indexes, REQUIRED_TABLES
and query_to_pydantic move to store/schema.py, which imports nothing from
haiku.rag: it describes the tables and never opens or mutates one.

gather_database_info, get_database_stats, DatabaseInfo and its result models
move to store/info.py. Nothing in Store calls them — they are read paths for
the CLI, doctor, inspector and ingester API — so info depends on engine and
not the reverse.

engine.py keeps the Store, ConnectionMode, connect_lancedb, the tag helpers
and the restore-order and retention constants. No re-exports: importers
point at the new modules.

test_app_info_uses_connect_lancedb_for_remote patched
haiku.rag.store.engine.connect_lancedb; gather_database_info now binds that
name in info.py, so the patch targets where the call is looked up.
2026-08-20 12:13:51 +03:00
Yiorgis Gozadinos
f6acb65e95
Reach and enforce 100% coverage
Cover the remaining paths in the client, context, downloads, title
generation, document tools and store models, and add fail_under=100 so
uncovered lines fail CI.

Six lines that no test can reach get a pragma with its reason: the docling
import guard, the nameless PDF attachment, the FS symlink OSError guard that
resolve(strict=False) absorbs, the docling bbox and LanceDB document-id
shape guards, the tag-retention branch vacuum makes unreachable, and Monty's
Rust-thread print callback.

Fix test_find_config_file_user_config, which wrote its config into the cwd it
had chdir'd to, so the cwd branch answered first and the user-directory
lookup it names was never exercised.
2026-07-26 20:11:02 +03:00
Yiorgis Gozadinos
a1cf405cae
Rename document_meta identity column document_id to id 2026-07-08 16:32:55 +03:00
Yiorgis Gozadinos
4f8cc4eb30
Show a progress spinner while doctor runs 2026-06-28 10:44:30 +03:00
Yiorgis Gozadinos
e7b4988d82
Compute document centroids during the vector scan 2026-06-28 10:23:16 +03:00
Yiorgis Gozadinos
15aae0f242
Simplify duplicate detection to whole-document centroid similarity 2026-06-28 10:23:16 +03:00
Yiorgis Gozadinos
73f62ab290
Enforce the duplicate candidate cap during row collection 2026-06-26 16:57:51 +03:00
Yiorgis Gozadinos
f1e1a16f9b
Add doctor --duplicates-out YAML export; summarize terminal report 2026-06-26 16:52:22 +03:00
Yiorgis Gozadinos
044ac62e49
Drop boilerplate handling; make duplicate report readable 2026-06-26 16:52:22 +03:00
Yiorgis Gozadinos
961913dde4
Ignore boilerplate chunks in duplicate-document detection 2026-06-26 16:52:21 +03:00
Yiorgis Gozadinos
43f2130b66
Add near-duplicate document detection to doctor 2026-06-26 16:52:21 +03:00
Yiorgis Gozadinos
f22341f003
Fix doctor's vector-index check to match documented guidance 2026-06-23 15:39:44 +03:00
Yiorgis Gozadinos
0ea251219c
Add explicit multimodal embedding flag, decoupled from provider name 2026-06-23 15:23:10 +03:00
Yiorgis Gozadinos
4fae733a50
Fix doctor provider checks: custom endpoints and processing models 2026-06-23 11:48:21 +03:00
Yiorgis Gozadinos
eb855f827a
Refine doctor's content and coverage checks 2026-06-23 11:31:20 +03:00
Yiorgis Gozadinos
cc1d8d1e4c
Add provider connectivity probes to haiku-rag doctor 2026-06-23 10:34:27 +03:00
Yiorgis Gozadinos
c2cb3cedf3
Add haiku-rag doctor database health check 2026-06-23 10:03:27 +03:00