From fe551066d83b24c775a5266d11ce5fbfb6f05beb Mon Sep 17 00:00:00 2001 From: Yiorgis Gozadinos Date: Thu, 4 Dec 2025 11:23:07 +0200 Subject: [PATCH] Visual grounding docs --- CHANGELOG.md | 6 ++++++ docs/cli.md | 13 ++++++++++++- docs/inspector.md | 11 +++++++++++ 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19f6f1f4..3cd2bf45 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,9 @@ - Keyboard navigation between pages (←/→ arrows) - Access from both main detail view and search results - Requires `textual-image` dependency +- **Visual Grounding CLI**: New `haiku-rag visualize ` command + - Displays page images with highlighted bounding boxes for a chunk + - Requires terminal with image support (iTerm2, Kitty, etc.) ### Changed @@ -41,6 +44,9 @@ - Headings remain available in `ChunkMetadata` for display and citations - Local and serve chunkers now produce identical output - **QA Prompts**: Updated to use page numbers and section headings in citations when available +- **Citation Models**: Introduced `RawSearchAnswer` for LLM output, `SearchAnswer` extends it with resolved citations + - Cleaner separation: LLM outputs chunk IDs, citations resolved programmatically + - `Citation` fields are now required (no defaults) for type safety ### Migration diff --git a/docs/cli.md b/docs/cli.md index da3f6493..1d432f7d 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -90,7 +90,18 @@ haiku-rag delete 3f4a... # document ID haiku-rag rm 3f4a... # alias ``` -Use this when you want to change things like the embedding model or chunk size for example. +## Visualize Chunk + +Display visual grounding for a chunk - shows page images with highlighted bounding boxes: + +```bash +haiku-rag visualize +``` + +This renders the source document pages with the chunk's location highlighted. Useful for verifying chunk boundaries and understanding document structure. + +!!! note + Requires a terminal with image support (iTerm2, Kitty, WezTerm, etc.) and documents processed with docling that have page images stored. ## Search diff --git a/docs/inspector.md b/docs/inspector.md index 7c250fb7..389c1d2b 100644 --- a/docs/inspector.md +++ b/docs/inspector.md @@ -52,3 +52,14 @@ Press `/` to open the full-screen search modal: - Press `Esc` to close search without selecting Search uses hybrid (vector + full-text) search across all chunks. Content is rendered as markdown with syntax highlighting. + +## Visual Grounding + +Press `v` while viewing a chunk to open the visual grounding modal: + +- Shows page images from the source document with the chunk's location highlighted in yellow/orange +- Use `←` / `→` arrow keys to navigate between pages (when chunk spans multiple pages) +- Press `Esc` to close the modal + +!!! note + Visual grounding requires documents processed with docling that have page images stored. Text-only documents or documents imported without DoclingDocument won't have visual grounding available.