danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
haiku.rag/haiku_rag_slim
History
Yiorgis Gozadinos 4aee18dcbe
Operator dashboard at GET /; tighten Logfire span shape 
Self-contained HTML status page served from the ingester's FastAPI app.
Polls /health, /sources, /stats, /jobs?status={claimed,dead,succeeded}
every 3s from the browser and renders queue chips, sources with
last-poll/skip-reason/circuit state, active jobs with cancel, recent
failures with retry, and recently-completed feed with op badges so
DELETE rows are visually distinct from UPSERTs. Zero external deps —
single static HTML, no CDN, no fonts, no images. Works offline.

To support the dashboard:
- New /stats endpoint exposing rolling throughput (5m/30m/1h), worker
  occupancy, oldest-queued age, and per-source DLQ + queue-depth
  breakdowns. Each field is a single SQL aggregation against the queue.
- JobRepo gains count_succeeded_since, oldest_queued_age_seconds,
  counts_by_source.
- SourceSummary gains last_skip_reason. BasePoller now records the
  reason the most recent sweep attempt was skipped ("pending_work" /
  "circuit_open"), cleared on the next successful poll. Closes the
  gap where operators couldn't tell from /sources alone why a source
  wasn't picking up new work.

Auth: dashboard route is unauthenticated (markup only). The JS attaches
the bearer to its own JSON fetches; on 401 it prompts once and stashes
the token in localStorage.

Two Logfire fixes that landed alongside:

- Drop logfire.instrument_fastapi() and the [fastapi] extra. The control
  plane is polled frequently (dashboard + docker healthcheck), so every
  endpoint became a span and drowned the useful traces. logfire itself
  stays — pulled in transitively via pydantic-ai-slim[logfire] — so
  ingester.poller.* / ingester.job / document.* spans keep emitting.

- Wrap FSPoller._handle_watch_change in an ingester.poller.watch_event
  span and pass _enqueue_extra. Without this, the watchfiles callback
  ran with no active context, the _otel carrier in job.extra was empty,
  and the worker's ingester.job span surfaced as an orphan trace root
  instead of nesting under the FS event that caused it.
2026-05-26 11:44:47 +03:00
..
haiku/rag Operator dashboard at GET /; tighten Logfire span shape 2026-05-26 11:44:47 +03:00
LICENSE Restructure into uv workspace to support minimal and full installations 2025-11-04 17:59:12 +02:00
pyproject.toml Operator dashboard at GET /; tighten Logfire span shape 2026-05-26 11:44:47 +03:00
README.md Update project description 2025-11-29 11:18:53 +02:00

README.md

haiku.rag-slim

Opinionated agentic RAG powered by LanceDB, Pydantic AI, and Docling - Core package with minimal dependencies.

haiku.rag-slim is the core package for users who want to install only the dependencies they need. Document processing (docling), and reranker support are all optional extras.

For most users, we recommend installing haiku.rag instead, which includes all features out of the box.

Installation

Python 3.12 or newer required

Minimal Installation

uv pip install haiku.rag-slim

Core functionality with OpenAI/Ollama support, MCP server, and Logfire observability. Document processing (docling) is optional.

With Document Processing

uv pip install haiku.rag-slim[docling]

Adds support for 40+ file formats including PDF, DOCX, HTML, and more.

Available Extras

Document Processing:

  • docling - PDF, DOCX, HTML, and 40+ file formats

Embedding Providers:

  • voyageai - VoyageAI embeddings

Rerankers:

  • mxbai - MixedBread AI
  • cohere - Cohere
  • zeroentropy - Zero Entropy

Model Providers:

  • OpenAI/Ollama - included in core (OpenAI-compatible APIs)
  • anthropic - Anthropic Claude
  • groq - Groq
  • google - Google Gemini
  • mistral - Mistral AI
  • bedrock - AWS Bedrock
  • vertexai - Google Vertex AI
# Common combinations
uv pip install haiku.rag-slim[docling,anthropic,mxbai]
uv pip install haiku.rag-slim[docling,groq,logfire]

Usage

See the main haiku.rag repository for:

  • Quick start guide
  • CLI examples
  • Python API usage
  • MCP server setup

Documentation

Full documentation: https://ggozad.github.io/haiku.rag/