Update docs
This commit is contained in:
parent
2afb10f6a4
commit
49efe3eede
6 changed files with 355 additions and 23 deletions
27
README.md
27
README.md
|
|
@ -8,19 +8,19 @@ Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.p
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
||||||
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
|
||||||
- **Question answering** — QA agents with citations (page numbers, section headings)
|
- **Question answering** — QA agents with citations (page numbers, section headings)
|
||||||
|
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
||||||
- **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize
|
- **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize
|
||||||
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
||||||
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
||||||
- **Visual grounding** — View chunks highlighted on original page images
|
|
||||||
- **Time travel** — Query the database at any historical point with `--before`
|
|
||||||
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI
|
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI
|
||||||
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
||||||
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
|
||||||
- **File monitoring** — Watch directories and auto-index on changes
|
|
||||||
- **Inspector** — TUI for browsing documents, chunks, and search results
|
|
||||||
- **CLI & Python API** — Full functionality from command line or code
|
- **CLI & Python API** — Full functionality from command line or code
|
||||||
|
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
||||||
|
- **Visual grounding** — View chunks highlighted on original page images
|
||||||
|
- **File monitoring** — Watch directories and auto-index on changes
|
||||||
|
- **Time travel** — Query the database at any historical point with `--before`
|
||||||
|
- **Inspector** — TUI for browsing documents, chunks, and search results
|
||||||
|
|
||||||
## Installation
|
## Installation
|
||||||
|
|
||||||
|
|
@ -29,21 +29,25 @@ Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.p
|
||||||
### Full Package (Recommended)
|
### Full Package (Recommended)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv pip install haiku.rag
|
pip install haiku.rag
|
||||||
```
|
```
|
||||||
|
|
||||||
Includes all features: document processing, all embedding providers, and rerankers.
|
Includes all features: document processing, all embedding providers, and rerankers.
|
||||||
|
|
||||||
|
Using [uv](https://docs.astral.sh/uv/)? `uv pip install haiku.rag`
|
||||||
|
|
||||||
### Slim Package (Minimal Dependencies)
|
### Slim Package (Minimal Dependencies)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv pip install haiku.rag-slim
|
pip install haiku.rag-slim
|
||||||
```
|
```
|
||||||
|
|
||||||
Install only the extras you need. See the [Installation](https://ggozad.github.io/haiku.rag/installation/) documentation for available options
|
Install only the extras you need. See the [Installation](https://ggozad.github.io/haiku.rag/installation/) documentation for available options.
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
|
> **Note**: Requires an embedding provider (Ollama, OpenAI, etc.). See the [Tutorial](https://ggozad.github.io/haiku.rag/tutorial/) for setup instructions.
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Index a PDF
|
# Index a PDF
|
||||||
haiku-rag add-src paper.pdf
|
haiku-rag add-src paper.pdf
|
||||||
|
|
@ -139,4 +143,9 @@ Full documentation at: https://ggozad.github.io/haiku.rag/
|
||||||
- [Benchmarks](https://ggozad.github.io/haiku.rag/benchmarks/) - Performance benchmarks
|
- [Benchmarks](https://ggozad.github.io/haiku.rag/benchmarks/) - Performance benchmarks
|
||||||
- [Changelog](https://ggozad.github.io/haiku.rag/changelog/) - Version history
|
- [Changelog](https://ggozad.github.io/haiku.rag/changelog/) - Version history
|
||||||
|
|
||||||
|
## License
|
||||||
|
|
||||||
|
This project is licensed under the [MIT License](LICENSE).
|
||||||
|
|
||||||
|
<!-- mcp-name is used by the MCP registry to identify this server -->
|
||||||
mcp-name: io.github.ggozad/haiku-rag
|
mcp-name: io.github.ggozad/haiku-rag
|
||||||
|
|
|
||||||
23
docs/apps.md
23
docs/apps.md
|
|
@ -25,6 +25,8 @@ The chat interface provides:
|
||||||
- Session memory for context-aware follow-up questions
|
- Session memory for context-aware follow-up questions
|
||||||
- Visual grounding to inspect chunk source locations
|
- Visual grounding to inspect chunk source locations
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
### Command Palette
|
### Command Palette
|
||||||
|
|
||||||
Press `Ctrl+P` to open the command palette:
|
Press `Ctrl+P` to open the command palette:
|
||||||
|
|
@ -146,6 +148,8 @@ Three panels display your data:
|
||||||
- **Chunks** (top right) - Chunks for the selected document
|
- **Chunks** (top right) - Chunks for the selected document
|
||||||
- **Detail View** (bottom right) - Full content and metadata
|
- **Detail View** (bottom right) - Full content and metadata
|
||||||
|
|
||||||
|

|
||||||
|
|
||||||
### Navigation
|
### Navigation
|
||||||
|
|
||||||
| Key | Action |
|
| Key | Action |
|
||||||
|
|
@ -182,10 +186,21 @@ Press `c` while viewing a chunk to see the expanded context that would be provid
|
||||||
|
|
||||||
### Visual Grounding
|
### Visual Grounding
|
||||||
|
|
||||||
|
Visual grounding shows exactly where a chunk appears in the original document by highlighting its bounding box on the page image. This helps verify chunk boundaries and understand how content was extracted.
|
||||||
|
|
||||||
Press `v` while viewing a chunk to see page images with the chunk's location highlighted:
|
Press `v` while viewing a chunk to see page images with the chunk's location highlighted:
|
||||||
|
|
||||||
- Use `←` / `→` arrow keys to navigate between pages
|
- Bounding boxes show the exact region of the page that maps to the chunk
|
||||||
- Requires documents processed with Docling that include page images
|
- Use `←` / `→` arrow keys to navigate between pages when a chunk spans multiple pages
|
||||||
|
- Press `Esc` to close the modal
|
||||||
|
|
||||||
!!! note
|

|
||||||
Visual grounding requires documents with a stored DoclingDocument that includes page images. Text-only documents won't have visual grounding available.
|
|
||||||
|
#### Requirements
|
||||||
|
|
||||||
|
- **Page images**: Documents must be processed with Docling's page image extraction enabled (default for PDFs)
|
||||||
|
- **Terminal image support**: Your terminal must support inline images (e.g., iTerm2, WezTerm, Kitty). Terminals without image support will show a placeholder message.
|
||||||
|
- **DoclingDocument storage**: Text-only documents (plain text, markdown added via `add`) don't have visual grounding available
|
||||||
|
|
||||||
|
!!! tip
|
||||||
|
You can also view visual grounding from the command line with `haiku-rag visualize <chunk_id>`. See [CLI documentation](cli.md#visualize-chunk) for details.
|
||||||
|
|
|
||||||
306
docs/img/tui-qa.svg
Normal file
306
docs/img/tui-qa.svg
Normal file
File diff suppressed because one or more lines are too long
|
After Width: | Height: | Size: 64 KiB |
BIN
docs/img/tui-visual-grounding.png
Normal file
BIN
docs/img/tui-visual-grounding.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 640 KiB |
|
|
@ -5,17 +5,19 @@ Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.p
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
||||||
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
|
||||||
- **Question answering** — QA agents with citations (page numbers, section headings)
|
- **Question answering** — QA agents with citations (page numbers, section headings)
|
||||||
|
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
||||||
- **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize
|
- **Research agents** — Multi-agent workflows via pydantic-graph: plan, search, evaluate, synthesize
|
||||||
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
||||||
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion and visual grounding
|
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
||||||
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI
|
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM. QA/Research: any model supported by Pydantic AI
|
||||||
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
||||||
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
|
||||||
- **File monitoring** — Watch directories and auto-index on changes
|
|
||||||
- **Inspector** — TUI for browsing documents, chunks, and search results
|
|
||||||
- **CLI & Python API** — Full functionality from command line or code
|
- **CLI & Python API** — Full functionality from command line or code
|
||||||
|
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
||||||
|
- **Visual grounding** — View chunks highlighted on original page images
|
||||||
|
- **File monitoring** — Watch directories and auto-index on changes
|
||||||
|
- **Time travel** — Query the database at any historical point with `--before`
|
||||||
|
- **Inspector** — TUI for browsing documents, chunks, and search results
|
||||||
|
|
||||||
## Quick Start
|
## Quick Start
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -86,21 +86,21 @@ Now you can view your [LanceDB](https://lancedb.com/) database, and the embeddin
|
||||||
haiku-rag info
|
haiku-rag info
|
||||||
```
|
```
|
||||||
|
|
||||||
You should see output like:
|
You should see output similar to:
|
||||||
|
|
||||||
```
|
```
|
||||||
haiku.rag database info
|
haiku.rag database info
|
||||||
path: /Users/moo/Library/Application Support/haiku.rag/haiku.rag.lancedb
|
path: /Users/moo/Library/Application Support/haiku.rag/haiku.rag.lancedb
|
||||||
haiku.rag version (db): 0.20.0
|
haiku.rag version (db): x.y.z
|
||||||
embeddings: openai/text-embedding-3-small (dim: 1536)
|
embeddings: openai/text-embedding-3-small (dim: 1536)
|
||||||
documents: 3 (storage: 48.0 KB)
|
documents: 3 (storage: 48.0 KB)
|
||||||
chunks: 3 (storage: 52.0 KB)
|
chunks: 3 (storage: 52.0 KB)
|
||||||
vector index: not created
|
vector index: not created
|
||||||
──────────────────────────────────────────────────────────────────────────────────
|
──────────────────────────────────────────────────────────────────────────────────
|
||||||
Versions
|
Versions
|
||||||
haiku.rag: 0.20.0
|
haiku.rag: x.y.z
|
||||||
lancedb: 0.25.2
|
lancedb: ...
|
||||||
docling: 2.58.0
|
docling: ...
|
||||||
```
|
```
|
||||||
|
|
||||||
## Asking questions and retrieving information
|
## Asking questions and retrieving information
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue