index, tutorial
This commit is contained in:
Yiorgis Gozadinos
parent
ae17e53f97
commit
76804b682f
4 changed files with 74 additions and 226 deletions
|
|
@ -53,6 +53,10 @@
|
|||
|
||||
- Chat TUI's state-edit screen syntax-highlights JSON instead of falling back to plain text. Adds `tree-sitter` + `tree-sitter-json` to the `[tui]` extra.
|
||||
|
||||
### Documentation
|
||||
|
||||
- Rework documentation
|
||||
|
||||
## [0.47.0] - 2026-05-14
|
||||
|
||||
### Added
|
||||
|
|
|
|||
|
|
@ -1,78 +1,47 @@
|
|||
# haiku.rag
|
||||
|
||||
Agentic RAG built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/), and [Docling](https://docling-project.github.io/docling/).
|
||||
haiku.rag is an agentic RAG that runs locally and scales to production. Index PDFs, web pages, or whole directories. Ask questions and get cited answers. Build agents, skills, and MCP integrations on top.
|
||||
|
||||
> **New: vision and multimodal search.** Picture-aware ingestion captures embedded figure bytes; vision-capable QA models receive them alongside text. Multimodal embedders (vLLM with `Qwen3-VL-Embedding-8B` or `jinaai/jina-embeddings-v4`) put picture vectors in the same space as text, enabling text-as-query → figure hits and image-as-query retrieval.
|
||||
haiku.rag is open-source first. The defaults run open models through [Ollama](https://ollama.com/) so the full pipeline works without external API keys. Any provider Pydantic AI supports works in its place.
|
||||
|
||||
## Features
|
||||
Built on [LanceDB](https://lancedb.com/), [Pydantic AI](https://ai.pydantic.dev/), and [Docling](https://docling-project.github.io/docling/). Embedded database, no servers required.
|
||||
|
||||
- **Hybrid search** — Vector + full-text with Reciprocal Rank Fusion
|
||||
- **Multimodal & cross-modal search** — Multimodal embedders (vLLM) put picture vectors in the same space as text; supports text-as-query → figure hits and image-as-query
|
||||
- **Question answering** — RAG skill with citations (page numbers, section headings)
|
||||
- **Vision QA** — Vision-capable models receive figure bytes alongside chunk text via pydantic-ai `BinaryContent` when `qa.model.vision = true`
|
||||
- **Reranking** — MxBAI, Cohere, Zero Entropy, or vLLM
|
||||
- **Analysis skill** — Complex analytical tasks via sandboxed Python code execution (aggregation, computation, multi-document analysis)
|
||||
- **Conversational RAG** — Chat TUI and web application for multi-turn conversations with session memory
|
||||
- **Document structure** — Stores full [DoclingDocument](https://docling-project.github.io/docling/concepts/docling_document/), enabling structure-aware context expansion
|
||||
- **Multiple providers** — Embeddings: Ollama, OpenAI, VoyageAI, LM Studio, vLLM (multimodal). QA: any model supported by Pydantic AI
|
||||
- **Local-first** — Embedded LanceDB, no servers required. Also supports S3, GCS, Azure, and LanceDB Cloud
|
||||
- **CLI & Python API** — Full functionality from command line or code
|
||||
- **MCP server** — Expose as tools for AI assistants (Claude Desktop, etc.)
|
||||
- **Visual grounding** — View chunks highlighted on original page images
|
||||
- **File monitoring** — Watch directories and auto-index on changes
|
||||
- **Time travel** — Query the database at any historical point with `--before`
|
||||
- **Inspector** — TUI for browsing documents, chunks, and search results
|
||||
|
||||
## Quick Start
|
||||
|
||||
Install haiku.rag:
|
||||
## See it work
|
||||
|
||||
```bash
|
||||
uv pip install haiku.rag
|
||||
|
||||
ollama pull qwen3-embedding:4b
|
||||
ollama pull gpt-oss
|
||||
|
||||
haiku-rag init
|
||||
haiku-rag add-src ~/Documents/some-paper.pdf
|
||||
haiku-rag chat
|
||||
```
|
||||
|
||||
Use from Python:
|
||||
The chat TUI is the fastest way to test retrieval and answer quality. `haiku-rag ask` and `haiku-rag search` cover one-shot CLI usage. Beyond that, the same database backs Python integrations, agents, skills, and the MCP server.
|
||||
|
||||
```python
|
||||
from haiku.rag.client import HaikuRAG
|
||||
## What it does
|
||||
|
||||
async with HaikuRAG("database.lancedb", create=True) as client:
|
||||
# Add a document
|
||||
doc = await client.create_document("Your content here")
|
||||
**Ingest.** PDFs, DOCX, HTML, images, and 40+ formats via Docling. Add files, URLs, or whole directories. Monitor folders and reindex on change.
|
||||
|
||||
# Search documents
|
||||
results = await client.search("query")
|
||||
**Search.** Hybrid retrieval (vector + full-text with reciprocal rank fusion), optional cross-encoder reranking, structure-aware context expansion. Image-as-query and cross-modal retrieval when configured with a multimodal embedder.
|
||||
|
||||
# Ask questions (returns answer and citations)
|
||||
answer, citations = await client.ask("Who is the author of haiku.rag?")
|
||||
```
|
||||
**Answer.** RAG skill with citations including page numbers, section headings, and visual grounding. Vision-capable models receive figure bytes alongside chunk text. Analysis skill with a sandboxed Python interpreter for aggregation and computation across documents.
|
||||
|
||||
Or use the CLI:
|
||||
**Integrate.** Use it from Python, the CLI, the [MCP server](mcp.md), or as composable [skills](skills/index.md) built on haiku.skills. Skills bundle tools, prompts, and state for use inside any Pydantic AI agent.
|
||||
|
||||
```bash
|
||||
haiku-rag add "Your document content"
|
||||
haiku-rag add "Your document content" --meta author=alice
|
||||
haiku-rag add-src /path/to/document.pdf --title "Q3 Financial Report" --meta source=manual
|
||||
haiku-rag search "query"
|
||||
haiku-rag ask "Who is the author of haiku.rag?"
|
||||
haiku-rag chat # Interactive conversation mode
|
||||
```
|
||||
**Operate.** Embedded LanceDB by default. Also runs on S3, GCS, Azure, or LanceDB Cloud. Time-travel queries via LanceDB versioning. File-monitoring mode for production deployments.
|
||||
|
||||
## Documentation
|
||||
## Where to go next
|
||||
|
||||
- [Getting started](tutorial.md) - Tutorial
|
||||
- [Installation](installation.md) - Install haiku.rag with different providers
|
||||
- [Configuration](configuration/index.md) - Environment variables and settings
|
||||
- [CLI](cli.md) - Command line interface usage
|
||||
- [Python](python.md) - Python API reference
|
||||
- [Custom Pipelines](custom-pipelines.md) - Build custom processing workflows
|
||||
- [Skills](skills/index.md) - The RAG and analysis skills the client wraps
|
||||
- [Analysis](agents/analysis.md) - Complex analytical tasks via code execution
|
||||
- [Applications](apps.md) - Chat TUI, web app, and inspector
|
||||
- [Server](server.md) - File monitoring and server mode
|
||||
- [MCP](mcp.md) - Model Context Protocol integration
|
||||
- [Remote processing](remote-processing.md) - Remote document processing with docling-serve
|
||||
- [Quickstart](tutorial.md): install through first chat in five minutes.
|
||||
- [Skills](skills/index.md): the rag and rag-analysis skills you compose into Pydantic AI agents.
|
||||
- [Python API](python.md): use haiku.rag from code.
|
||||
- [MCP server](mcp.md): expose haiku.rag to Claude Desktop or other AI assistants.
|
||||
- [Tuning](tuning.md): improve retrieval quality.
|
||||
- [Configuration](configuration/index.md): every setting.
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the [MIT License](https://raw.githubusercontent.com/ggozad/haiku.rag/main/LICENSE).
|
||||
MIT. Source on [GitHub](https://github.com/ggozad/haiku.rag).
|
||||
|
|
|
|||
211
docs/tutorial.md
211
docs/tutorial.md
|
|
@ -1,211 +1,86 @@
|
|||
# Tutorial
|
||||
# Quickstart
|
||||
|
||||
This tutorial provides quickstart instructions for getting familiar with `haiku.rag`. This tutorial is intended for people who are familiar with command line and Python, but not different AI ecosystem tools.
|
||||
Goal: install haiku.rag, index a document, and chat with it. Five minutes if you already have Ollama.
|
||||
|
||||
The tutorial covers:
|
||||
|
||||
- RAG and embeddings basics
|
||||
- Installing `haiku.rag` Python package
|
||||
- Configuring `haiku.rag` with YAML
|
||||
- Adding and retrieving items
|
||||
- Inspecting the database
|
||||
|
||||
The tutorial uses OpenAI API service - no local installation needed and will work on computers with any amount of RAM and GPU. The OpenAI API is pay-as-you-go, so you need to top it up with at least ~$5 when creating the API key.
|
||||
|
||||
## Introduction
|
||||
|
||||
Retrieval-Augmented Generation (RAG) lets you give AI models access to your own documents and data. Instead of relying solely on the model's training data, RAG finds relevant information from your documents and includes it in the AI's responses.
|
||||
|
||||
`haiku.rag` handles the mechanics: it converts your documents into searchable embeddings, stores them locally, and retrieves relevant chunks when you ask questions. You provide the documents and questions, and it coordinates between the embedding service (like OpenAI) and the AI model to give you accurate, grounded answers.
|
||||
|
||||
## Setup
|
||||
|
||||
First, [get an OpenAI API key](https://platform.openai.com/api-keys).
|
||||
|
||||
Install `haiku.rag` Python package using [uv](https://docs.astral.sh/uv/getting-started/installation/) or your favourite Python package manager:
|
||||
## Install
|
||||
|
||||
```bash
|
||||
# Python 3.12+ needed
|
||||
uv pip install haiku.rag
|
||||
```
|
||||
|
||||
Configure haiku.rag to use OpenAI. Create a `haiku.rag.yaml` file:
|
||||
|
||||
```yaml
|
||||
embeddings:
|
||||
model:
|
||||
provider: openai
|
||||
name: text-embedding-3-small # or text-embedding-3-large
|
||||
vector_dim: 1536
|
||||
|
||||
qa:
|
||||
model:
|
||||
provider: openai
|
||||
name: gpt-4o-mini # or gpt-4o, gpt-4, etc.
|
||||
```
|
||||
|
||||
Set your OpenAI API key as an environment variable (API keys should not be stored in the YAML file):
|
||||
You also need [Ollama](https://ollama.com/) for the default embedding and answering models:
|
||||
|
||||
```bash
|
||||
export OPENAI_API_KEY="<your OpenAI API key>"
|
||||
ollama pull qwen3-embedding:4b
|
||||
ollama pull gpt-oss
|
||||
```
|
||||
|
||||
For the list of available OpenAI models and their vector dimensions, see the [OpenAI documentation](https://platform.openai.com/docs/guides/embeddings).
|
||||
!!! note "Prefer OpenAI?"
|
||||
Drop this into a `haiku.rag.yaml` next to where you'll run the CLI:
|
||||
|
||||
See [Configuration](configuration/index.md) for all available options.
|
||||
```yaml
|
||||
embeddings:
|
||||
model:
|
||||
provider: openai
|
||||
name: text-embedding-3-small
|
||||
vector_dim: 1536
|
||||
|
||||
## Initialize the database
|
||||
qa:
|
||||
model:
|
||||
provider: openai
|
||||
name: gpt-4o-mini
|
||||
```
|
||||
|
||||
Before adding documents, initialize the database:
|
||||
Then `export OPENAI_API_KEY="sk-..."` and continue with the rest of this page. Any provider Pydantic AI supports works the same way. See [Providers](configuration/providers.md).
|
||||
|
||||
## Initialize
|
||||
|
||||
```bash
|
||||
haiku-rag init
|
||||
```
|
||||
|
||||
This creates an empty database with the configured settings.
|
||||
|
||||
## Adding the first documents
|
||||
|
||||
Now you can add some pieces of text in the database:
|
||||
This creates a LanceDB database in your platform's user directory. Pass `--db` to any subcommand to use a different path:
|
||||
|
||||
```bash
|
||||
haiku-rag add "Python is the best programming language in the world, because it is flexible, with robust ecosystem, open source licensing and thousands of contributors"
|
||||
haiku-rag add "JavaScript is a popular programming language, but has a lot of warts"
|
||||
haiku-rag add "PHP is a bad programming language, because of spotted security history, horrible syntax and declining popularity"
|
||||
haiku-rag init --db /tmp/test.lancedb
|
||||
```
|
||||
|
||||
What will happen:
|
||||
## Add a document
|
||||
|
||||
- The piece of text is sent to OpenAI `/embeddings` API service
|
||||
- OpenAI translates the free form text to RAG embedding vectors needed for the retrieval
|
||||
- The vector values will be stored in a local database
|
||||
|
||||
Now you can view your [LanceDB](https://lancedb.com/) database, and the embeddings it is configured for:
|
||||
Add a file, a URL, or a whole folder:
|
||||
|
||||
```bash
|
||||
haiku-rag info
|
||||
haiku-rag add-src https://arxiv.org/pdf/2408.09134
|
||||
haiku-rag add-src ~/Documents/papers/
|
||||
```
|
||||
|
||||
You should see output similar to:
|
||||
|
||||
```
|
||||
haiku.rag database info
|
||||
path: /Users/moo/Library/Application Support/haiku.rag/haiku.rag.lancedb
|
||||
haiku.rag version (db): x.y.z
|
||||
embeddings: openai/text-embedding-3-small (dim: 1536)
|
||||
documents: 3 (storage: 48.0 KB)
|
||||
chunks: 3 (storage: 52.0 KB)
|
||||
vector index: not created
|
||||
──────────────────────────────────────────────────────────────────────────────────
|
||||
Versions
|
||||
haiku.rag: x.y.z
|
||||
lancedb: ...
|
||||
docling: ...
|
||||
```
|
||||
|
||||
## Asking questions and retrieving information
|
||||
|
||||
Now we can use OpenAI LLMs to retrieve information from our embeddings database.
|
||||
|
||||
In this example, we connect to a remote OpenAI API.
|
||||
|
||||
Behind the scenes [pydantic-ai](https://ai.pydantic.dev/) query is created
|
||||
using `OpenAIChatModel.request()`.
|
||||
|
||||
The easiest way to do this is `ask` CLI command:
|
||||
Or paste text inline:
|
||||
|
||||
```bash
|
||||
haiku-rag ask "What is the best programming language in the world"
|
||||
haiku-rag add "Yiorgis wrote haiku.rag in 2025."
|
||||
```
|
||||
|
||||
```
|
||||
Question: What is the best programming language in the world
|
||||
Each `add-src` call converts the file with Docling, splits it into chunks, embeds them, and writes everything to LanceDB. Run `haiku-rag list` to see what you've added, `haiku-rag info` for a database summary.
|
||||
|
||||
Answer:
|
||||
According to the document, Python is considered the best programming language in the world due to its flexibility, robust ecosystem, open-source licensing, and thousands of contributors.
|
||||
```
|
||||
|
||||
## Programmatic interaction in Python
|
||||
|
||||
You can interact with haiku.rag from Python. Since the API is async, we'll use IPython which supports async/await directly.
|
||||
## Chat
|
||||
|
||||
```bash
|
||||
uv pip install ipython
|
||||
ipython
|
||||
haiku-rag chat
|
||||
```
|
||||
|
||||
Then run:
|
||||
Ask a question. The agent searches your documents, expands context around the hits, and answers with citations pointing back to the source page and section. Citations are expandable, with visual grounding so you can see the chunk highlighted on the original page. Follow-ups continue within the same session. Start a new session when you switch topics.
|
||||
|
||||
```python
|
||||
from haiku.rag.client import HaikuRAG
|
||||
|
||||
# Uses database from default location (must be initialized first)
|
||||
async with HaikuRAG() as client:
|
||||
answer, citations = await client.ask("What is the best programming language in the world?")
|
||||
print(answer)
|
||||
```
|
||||
|
||||
You should see:
|
||||
|
||||
```
|
||||
According to the document, Python is considered the best programming language in the world due to its flexibility, robust ecosystem, open-source licensing, and support from thousands of contributors.
|
||||
```
|
||||
|
||||
## Complex documents
|
||||
|
||||
Haiku RAG can also handle types beyond plain text, including PDF, DOCX, HTML, and 40+ other file formats.
|
||||
|
||||
Here we add research papers about Python from [arxiv](https://arxiv.org/search/?query=python&searchtype=all&source=header) using URL retriever.
|
||||
You can also ask a single question directly from the CLI without launching the TUI:
|
||||
|
||||
```bash
|
||||
# Better Python Programming for all: With the focus on Maintainability
|
||||
haiku-rag add-src --meta collection="Interesting Python papers" "https://arxiv.org/pdf/2408.09134"
|
||||
|
||||
# Interoperability From OpenTelemetry to Kieker: Demonstrated as Export from the Astronomy Shop
|
||||
haiku-rag add-src --meta collection="Interesting Python papers" "https://arxiv.org/pdf/2510.11179"
|
||||
haiku-rag ask "Who wrote haiku.rag?"
|
||||
```
|
||||
|
||||
Then we can query this:
|
||||
## Where to go next
|
||||
|
||||
```bash
|
||||
haiku-rag ask "Who wrote a paper about OpenTelemetry interoperability, and what was his take"
|
||||
```
|
||||
|
||||
We should get something along the lines:
|
||||
|
||||
```
|
||||
Answer:
|
||||
David Georg Reichelt from Lancaster University wrote a paper titled "Interoperability From OpenTelemetry to Kieker: Demonstrated as Export from the Astronomy Shop." In his work, he indicates that there is a structural difference between Kieker’s synchronous traces and OpenTelemetry’s asynchronous traces, leading to limited compatibility between the two systems. This highlights the challenges of interoperability in observability frameworks.
|
||||
```
|
||||
|
||||
We can also add offline files, like PDFs. Here we add a local file to ensure OpenAI does not cheat - a file we know that should not be very well known in Internet:
|
||||
|
||||
```bash
|
||||
# This static file is supplied in haiku.rag repo
|
||||
haiku-rag add-src "examples/samples/PyCon Finland 2025 Schedule.html"
|
||||
```
|
||||
|
||||
And then:
|
||||
|
||||
```bash
|
||||
haiku-rag ask "Who were presenting talks in Pycon Finland 2025? Can you give at least five different people."
|
||||
```
|
||||
|
||||
```
|
||||
The following people are presenting talks at PyCon Finland 2025:
|
||||
|
||||
1 Jeremy Mayeres - Talk: The Limits of Imagination: An Open Source Journey
|
||||
2 Aroma Rodrigues - Talk: Python and Rust, a Perfect Pairing
|
||||
3 Andreas Jung - Talk: Guillotina Volto: A New Backend for Volto
|
||||
4 Daniel Vahla - Talk: Experiences with AI in Software Projects
|
||||
5 Andreas Jung (also presenting another talk) - Talk: Debugging Python
|
||||
```
|
||||
|
||||
## Next Steps
|
||||
|
||||
- **[Chat](apps.md#chat-tui)** - Interactive conversations with `haiku-rag chat`
|
||||
- **[CLI Reference](cli.md)** - All available commands and options
|
||||
- **[Python API](python.md)** - Use haiku.rag in your Python applications
|
||||
- **[Skills](skills/index.md)** - The RAG and analysis skills the client wraps
|
||||
- **[Configuration](configuration/index.md)** - Complete YAML configuration reference
|
||||
- **[Server Mode](server.md)** - File monitoring and MCP server
|
||||
- [Chat](apps.md#chat-tui): sessions, citations, and the full TUI.
|
||||
- [CLI reference](cli.md): every command.
|
||||
- [Python API](python.md): use haiku.rag in your own code.
|
||||
- [Skills](skills/index.md): the rag and rag-analysis skills the client wraps.
|
||||
- [Tuning](tuning.md): better retrieval.
|
||||
- [Configuration](configuration/index.md): every setting.
|
||||
|
|
|
|||
|
|
@ -94,7 +94,7 @@ markdown_extensions:
|
|||
use_pygments: true
|
||||
- pymdownx.inlinehilite
|
||||
- pymdownx.snippets:
|
||||
base_path: ['.']
|
||||
base_path: ["."]
|
||||
- pymdownx.superfences:
|
||||
custom_fences:
|
||||
- name: mermaid
|
||||
|
|
|
|||
Loading…
Reference in a new issue