Update docs

This commit is contained in:
Yiorgis Gozadinos 2026-02-26 09:09:42 +02:00
parent 41f694d377
commit d3a1031ec4
No known key found for this signature in database
4 changed files with 67 additions and 0 deletions

View file

@ -1,6 +1,13 @@
# Changelog
## [Unreleased]
### Added
- **Automatic title generation**: Documents can now have titles auto-generated during ingestion via `processing.auto_title: true`. Uses two-tier extraction: structural metadata from DoclingDocument (HTML `<title>`, h1, section headers) first, with LLM fallback via configurable `processing.title_model`
- **`generate_title()`**: Public method on `HaikuRAG` to generate a title for an existing document on demand
- **`rebuild --title-only`**: New rebuild mode that generates titles only for untitled documents without re-chunking or re-embedding
- **`add --title`**: CLI option to set a title when adding text documents
## [0.32.0] - 2026-02-24
### Changed

View file

@ -50,6 +50,9 @@ From text:
```bash
haiku-rag add "Your document content here"
# Set a title
haiku-rag add "Your document content here" --title "My Document"
# Attach metadata (repeat --meta for multiple entries)
haiku-rag add "Your document content here" --meta author=alice --meta topic=notes
```
@ -391,6 +394,9 @@ haiku-rag rebuild --rechunk
# Only regenerate embeddings (fastest, keeps existing chunks)
haiku-rag rebuild --embed-only
# Only generate titles for untitled documents
haiku-rag rebuild --title-only
```
**Rebuild modes:**
@ -400,6 +406,7 @@ haiku-rag rebuild --embed-only
| Full | (default) | Changed converter, source files updated |
| Rechunk | `--rechunk` | Changed chunking strategy or chunk size |
| Embed only | `--embed-only` | Changed embedding model or vector dimensions |
| Title only | `--title-only` | Generate titles for documents without one |
### Download Models

View file

@ -21,6 +21,13 @@ processing:
chunking_merge_peers: true # Merge undersized successive chunks
chunking_use_markdown_tables: false # Use markdown tables vs narrative format
# Automatic title generation
auto_title: false # Auto-generate titles on ingestion
title_model: # LLM for title generation (fallback)
provider: ollama
name: gpt-oss
enable_thinking: false
# Conversion options (works with both local and remote converters)
conversion_options:
# OCR settings
@ -201,6 +208,30 @@ picture_description:
See [VLM Picture Description with docling-serve](../remote-processing.md#vlm-picture-description-with-docling-serve) for a complete example.
### Automatic Title Generation
Enable automatic title generation during document ingestion:
```yaml
processing:
auto_title: true
title_model:
provider: ollama
name: gpt-oss
enable_thinking: false
```
When `auto_title` is enabled, haiku.rag attempts to extract a title for each document during ingestion using a two-tier approach:
1. **Structural extraction** (free, no model calls): Scans the DoclingDocument for semantic labels — HTML `<title>` tags, `<h1>` headings, PDF title blocks, and section headers
2. **LLM fallback**: When no structural title is found (e.g., plain text), generates a title using the configured `title_model`
Priority order: HTML `<title>` (furniture layer) → h1/PDF title (body layer) → first section header → LLM generation.
Explicit titles passed via `title=` parameter always take precedence and are never overridden. When updating documents, existing titles are preserved — auto-generation only applies to untitled documents.
To generate titles for existing untitled documents, use [`rebuild --title-only`](../cli.md#rebuild-database).
### Local vs Remote Processing
**Local processing** (default):

View file

@ -229,6 +229,28 @@ async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY):
- `RebuildMode.FULL` - Re-convert from source files, re-chunk, re-embed (default)
- `RebuildMode.RECHUNK` - Re-chunk from existing document content, re-embed
- `RebuildMode.EMBED_ONLY` - Keep existing chunks, only regenerate embeddings
- `RebuildMode.TITLE_ONLY` - Generate titles for untitled documents (no re-chunking or re-embedding)
### Generating Titles
Generate a title for an existing document on demand:
```python
title = await client.generate_title(doc)
if title:
await client.update_document(document_id=doc.id, title=title)
```
Uses the same two-tier approach as automatic ingestion: structural extraction from DoclingDocument metadata first, with LLM fallback via `processing.title_model`. Unlike ingestion, this method does not catch exceptions — if the LLM call fails, the error propagates.
To batch-generate titles for all untitled documents, use `RebuildMode.TITLE_ONLY`:
```python
async for doc_id in client.rebuild_database(mode=RebuildMode.TITLE_ONLY):
print(f"Generated title for {doc_id}")
```
See [Automatic Title Generation](configuration/processing.md#automatic-title-generation) for configuration details.
## Maintenance