From d3a1031ec443690cbd78400fc8dccb92cc96235b Mon Sep 17 00:00:00 2001 From: Yiorgis Gozadinos Date: Thu, 26 Feb 2026 09:09:42 +0200 Subject: [PATCH] Update docs --- CHANGELOG.md | 7 +++++++ docs/cli.md | 7 +++++++ docs/configuration/processing.md | 31 +++++++++++++++++++++++++++++++ docs/python.md | 22 ++++++++++++++++++++++ 4 files changed, 67 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9eec119d..f9f05db4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,13 @@ # Changelog ## [Unreleased] +### Added + +- **Automatic title generation**: Documents can now have titles auto-generated during ingestion via `processing.auto_title: true`. Uses two-tier extraction: structural metadata from DoclingDocument (HTML ``, h1, section headers) first, with LLM fallback via configurable `processing.title_model` +- **`generate_title()`**: Public method on `HaikuRAG` to generate a title for an existing document on demand +- **`rebuild --title-only`**: New rebuild mode that generates titles only for untitled documents without re-chunking or re-embedding +- **`add --title`**: CLI option to set a title when adding text documents + ## [0.32.0] - 2026-02-24 ### Changed diff --git a/docs/cli.md b/docs/cli.md index 4622be47..38f0f390 100644 --- a/docs/cli.md +++ b/docs/cli.md @@ -50,6 +50,9 @@ From text: ```bash haiku-rag add "Your document content here" +# Set a title +haiku-rag add "Your document content here" --title "My Document" + # Attach metadata (repeat --meta for multiple entries) haiku-rag add "Your document content here" --meta author=alice --meta topic=notes ``` @@ -391,6 +394,9 @@ haiku-rag rebuild --rechunk # Only regenerate embeddings (fastest, keeps existing chunks) haiku-rag rebuild --embed-only + +# Only generate titles for untitled documents +haiku-rag rebuild --title-only ``` **Rebuild modes:** @@ -400,6 +406,7 @@ haiku-rag rebuild --embed-only | Full | (default) | Changed converter, source files updated | | Rechunk | `--rechunk` | Changed chunking strategy or chunk size | | Embed only | `--embed-only` | Changed embedding model or vector dimensions | +| Title only | `--title-only` | Generate titles for documents without one | ### Download Models diff --git a/docs/configuration/processing.md b/docs/configuration/processing.md index 938e1b7e..0dae70ba 100644 --- a/docs/configuration/processing.md +++ b/docs/configuration/processing.md @@ -21,6 +21,13 @@ processing: chunking_merge_peers: true # Merge undersized successive chunks chunking_use_markdown_tables: false # Use markdown tables vs narrative format + # Automatic title generation + auto_title: false # Auto-generate titles on ingestion + title_model: # LLM for title generation (fallback) + provider: ollama + name: gpt-oss + enable_thinking: false + # Conversion options (works with both local and remote converters) conversion_options: # OCR settings @@ -201,6 +208,30 @@ picture_description: See [VLM Picture Description with docling-serve](../remote-processing.md#vlm-picture-description-with-docling-serve) for a complete example. +### Automatic Title Generation + +Enable automatic title generation during document ingestion: + +```yaml +processing: + auto_title: true + title_model: + provider: ollama + name: gpt-oss + enable_thinking: false +``` + +When `auto_title` is enabled, haiku.rag attempts to extract a title for each document during ingestion using a two-tier approach: + +1. **Structural extraction** (free, no model calls): Scans the DoclingDocument for semantic labels — HTML `<title>` tags, `<h1>` headings, PDF title blocks, and section headers +2. **LLM fallback**: When no structural title is found (e.g., plain text), generates a title using the configured `title_model` + +Priority order: HTML `<title>` (furniture layer) → h1/PDF title (body layer) → first section header → LLM generation. + +Explicit titles passed via `title=` parameter always take precedence and are never overridden. When updating documents, existing titles are preserved — auto-generation only applies to untitled documents. + +To generate titles for existing untitled documents, use [`rebuild --title-only`](../cli.md#rebuild-database). + ### Local vs Remote Processing **Local processing** (default): diff --git a/docs/python.md b/docs/python.md index 9a21508e..3035fb73 100644 --- a/docs/python.md +++ b/docs/python.md @@ -229,6 +229,28 @@ async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY): - `RebuildMode.FULL` - Re-convert from source files, re-chunk, re-embed (default) - `RebuildMode.RECHUNK` - Re-chunk from existing document content, re-embed - `RebuildMode.EMBED_ONLY` - Keep existing chunks, only regenerate embeddings +- `RebuildMode.TITLE_ONLY` - Generate titles for untitled documents (no re-chunking or re-embedding) + +### Generating Titles + +Generate a title for an existing document on demand: + +```python +title = await client.generate_title(doc) +if title: + await client.update_document(document_id=doc.id, title=title) +``` + +Uses the same two-tier approach as automatic ingestion: structural extraction from DoclingDocument metadata first, with LLM fallback via `processing.title_model`. Unlike ingestion, this method does not catch exceptions — if the LLM call fails, the error propagates. + +To batch-generate titles for all untitled documents, use `RebuildMode.TITLE_ONLY`: + +```python +async for doc_id in client.rebuild_database(mode=RebuildMode.TITLE_ONLY): + print(f"Generated title for {doc_id}") +``` + +See [Automatic Title Generation](configuration/processing.md#automatic-title-generation) for configuration details. ## Maintenance