History

Yiorgis Gozadinos d63199d96d Keep cross-encoder rerank scores apart when they saturate mxbai-rerank-base-v2 ships a Sigmoid activation and evaluates it in bf16, so every strongly-relevant candidate rounds to exactly 1.0. Ties then leave the order to the stable sort, which preserves the incoming hybrid ranking: on 100 t2_finqa retrieval cases the reranker scored MAP 0.661 against 0.659 with no reranker at all, and 0.742 once the scores separate. Ask the model for logits and apply the sigmoid here, where it runs in float64. Scores stay 0-1, matching the cohere, vllm and zeroentropy rerankers. Also drop the remaining pyright references; the project type-checks with ty.		2026-08-07 14:07:53 +03:00
..
backend	Keep cross-encoder rerank scores apart when they saturate	2026-08-07 14:07:53 +03:00
frontend	Fix app Dockerfile	2026-07-24 16:29:28 +03:00
.env.example	Update docs for app	2026-01-13 12:18:13 +02:00
docker-compose.dev.yml	Bind app example backend to loopback and document its lack of auth	2026-07-08 11:37:03 +03:00
docker-compose.yml	Bind app example backend to loopback and document its lack of auth	2026-07-08 11:37:03 +03:00
haiku.rag.yaml.example	replace fixed-radius expansion with section-bounded algorithm	2026-04-16 12:11:53 +03:00
README.md	Bind app example backend to loopback and document its lack of auth	2026-07-08 11:37:03 +03:00

README.md

haiku.rag Chat App

A conversational RAG interface built with CopilotKit and pydantic-ai's AG-UI protocol.

Note: An illustrative example meant as a starting point, with no authentication. The compose files bind the backend to 127.0.0.1; don't expose it to an untrusted network.

Prerequisites

Docker and Docker Compose
A haiku.rag database (created via the haiku-rag CLI)
An LLM API key (Anthropic, OpenAI, or local Ollama)

Quick Start

Set up environment variables:

cp .env.example .env
# Edit .env with your API keys and database path

Configure the LLM and embedding models:

cp haiku.rag.yaml.example haiku.rag.yaml
# Edit haiku.rag.yaml to configure your models

Start the app:
```
docker compose up -d
```
Open the chat interface: http://localhost:3000

Configuration

Environment Variables

Variable	Description	Required
`DB_PATH`	Path to your haiku.rag LanceDB database	Yes
`ANTHROPIC_API_KEY`	Anthropic API key	One LLM key required
`OPENAI_API_KEY`	OpenAI API key	One LLM key required
`OLLAMA_BASE_URL`	Ollama server URL (default: `http://host.docker.internal:11434`)	For local models
`LOGFIRE_TOKEN`	Pydantic Logfire token for debugging	No

haiku.rag.yaml

Configure the LLM, embeddings, and search settings:

qa:
  model:
    provider: anthropic  # or openai, ollama
    name: claude-sonnet-4-20250514

embeddings:
  model:
    provider: ollama
    name: nomic-embed-text

search:
  limit: 10

See haiku.rag.yaml.example for all options.

Development

For local development with hot reloading:

docker compose -f docker-compose.dev.yml up -d --build

Backend code changes reload automatically
Frontend available at http://localhost:3000
Backend API at http://localhost:8001

Architecture

┌─────────────────┐     ┌─────────────────┐     ┌─────────────────┐
│    Frontend     │────▶│     Backend     │────▶│   haiku.rag     │
│  (CopilotKit)   │     │  (pydantic-ai)  │     │   (LanceDB)     │
│  localhost:3000 │     │  localhost:8001 │     │                 │
└─────────────────┘     └─────────────────┘     └─────────────────┘

Backend Endpoints

Endpoint	Method	Description
`/v1/chat/stream`	POST	AG-UI chat streaming
`/api/documents`	GET	List documents in database
`/api/info`	GET	Database statistics
`/api/visualize/{chunk_id}`	GET	Visual grounding for chunks
`/health`	GET	Health check

Chat Capabilities

The chat can:

Search your documents with hybrid vector + full-text search
Answer questions with citations from your knowledge base
Filter by document when you ask about specific files
Show visual grounding for PDF/image sources