Visual grounding docs

This commit is contained in:
Yiorgis Gozadinos 2025-12-04 11:23:07 +02:00
parent de14bb9f90
commit fe551066d8
No known key found for this signature in database
3 changed files with 29 additions and 1 deletions

View file

@ -26,6 +26,9 @@
- Keyboard navigation between pages (←/→ arrows)
- Access from both main detail view and search results
- Requires `textual-image` dependency
- **Visual Grounding CLI**: New `haiku-rag visualize <chunk_id>` command
- Displays page images with highlighted bounding boxes for a chunk
- Requires terminal with image support (iTerm2, Kitty, etc.)
### Changed
@ -41,6 +44,9 @@
- Headings remain available in `ChunkMetadata` for display and citations
- Local and serve chunkers now produce identical output
- **QA Prompts**: Updated to use page numbers and section headings in citations when available
- **Citation Models**: Introduced `RawSearchAnswer` for LLM output, `SearchAnswer` extends it with resolved citations
- Cleaner separation: LLM outputs chunk IDs, citations resolved programmatically
- `Citation` fields are now required (no defaults) for type safety
### Migration

View file

@ -90,7 +90,18 @@ haiku-rag delete 3f4a... # document ID
haiku-rag rm 3f4a... # alias
```
Use this when you want to change things like the embedding model or chunk size for example.
## Visualize Chunk
Display visual grounding for a chunk - shows page images with highlighted bounding boxes:
```bash
haiku-rag visualize <chunk_id>
```
This renders the source document pages with the chunk's location highlighted. Useful for verifying chunk boundaries and understanding document structure.
!!! note
Requires a terminal with image support (iTerm2, Kitty, WezTerm, etc.) and documents processed with docling that have page images stored.
## Search

View file

@ -52,3 +52,14 @@ Press `/` to open the full-screen search modal:
- Press `Esc` to close search without selecting
Search uses hybrid (vector + full-text) search across all chunks. Content is rendered as markdown with syntax highlighting.
## Visual Grounding
Press `v` while viewing a chunk to open the visual grounding modal:
- Shows page images from the source document with the chunk's location highlighted in yellow/orange
- Use `←` / `→` arrow keys to navigate between pages (when chunk spans multiple pages)
- Press `Esc` to close the modal
!!! note
Visual grounding requires documents processed with docling that have page images stored. Text-only documents or documents imported without DoclingDocument won't have visual grounding available.