Update docs
This commit is contained in:
parent
41f694d377
commit
d3a1031ec4
4 changed files with 67 additions and 0 deletions
|
|
@ -1,6 +1,13 @@
|
|||
# Changelog
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- **Automatic title generation**: Documents can now have titles auto-generated during ingestion via `processing.auto_title: true`. Uses two-tier extraction: structural metadata from DoclingDocument (HTML `<title>`, h1, section headers) first, with LLM fallback via configurable `processing.title_model`
|
||||
- **`generate_title()`**: Public method on `HaikuRAG` to generate a title for an existing document on demand
|
||||
- **`rebuild --title-only`**: New rebuild mode that generates titles only for untitled documents without re-chunking or re-embedding
|
||||
- **`add --title`**: CLI option to set a title when adding text documents
|
||||
|
||||
## [0.32.0] - 2026-02-24
|
||||
|
||||
### Changed
|
||||
|
|
|
|||
|
|
@ -50,6 +50,9 @@ From text:
|
|||
```bash
|
||||
haiku-rag add "Your document content here"
|
||||
|
||||
# Set a title
|
||||
haiku-rag add "Your document content here" --title "My Document"
|
||||
|
||||
# Attach metadata (repeat --meta for multiple entries)
|
||||
haiku-rag add "Your document content here" --meta author=alice --meta topic=notes
|
||||
```
|
||||
|
|
@ -391,6 +394,9 @@ haiku-rag rebuild --rechunk
|
|||
|
||||
# Only regenerate embeddings (fastest, keeps existing chunks)
|
||||
haiku-rag rebuild --embed-only
|
||||
|
||||
# Only generate titles for untitled documents
|
||||
haiku-rag rebuild --title-only
|
||||
```
|
||||
|
||||
**Rebuild modes:**
|
||||
|
|
@ -400,6 +406,7 @@ haiku-rag rebuild --embed-only
|
|||
| Full | (default) | Changed converter, source files updated |
|
||||
| Rechunk | `--rechunk` | Changed chunking strategy or chunk size |
|
||||
| Embed only | `--embed-only` | Changed embedding model or vector dimensions |
|
||||
| Title only | `--title-only` | Generate titles for documents without one |
|
||||
|
||||
### Download Models
|
||||
|
||||
|
|
|
|||
|
|
@ -21,6 +21,13 @@ processing:
|
|||
chunking_merge_peers: true # Merge undersized successive chunks
|
||||
chunking_use_markdown_tables: false # Use markdown tables vs narrative format
|
||||
|
||||
# Automatic title generation
|
||||
auto_title: false # Auto-generate titles on ingestion
|
||||
title_model: # LLM for title generation (fallback)
|
||||
provider: ollama
|
||||
name: gpt-oss
|
||||
enable_thinking: false
|
||||
|
||||
# Conversion options (works with both local and remote converters)
|
||||
conversion_options:
|
||||
# OCR settings
|
||||
|
|
@ -201,6 +208,30 @@ picture_description:
|
|||
|
||||
See [VLM Picture Description with docling-serve](../remote-processing.md#vlm-picture-description-with-docling-serve) for a complete example.
|
||||
|
||||
### Automatic Title Generation
|
||||
|
||||
Enable automatic title generation during document ingestion:
|
||||
|
||||
```yaml
|
||||
processing:
|
||||
auto_title: true
|
||||
title_model:
|
||||
provider: ollama
|
||||
name: gpt-oss
|
||||
enable_thinking: false
|
||||
```
|
||||
|
||||
When `auto_title` is enabled, haiku.rag attempts to extract a title for each document during ingestion using a two-tier approach:
|
||||
|
||||
1. **Structural extraction** (free, no model calls): Scans the DoclingDocument for semantic labels — HTML `<title>` tags, `<h1>` headings, PDF title blocks, and section headers
|
||||
2. **LLM fallback**: When no structural title is found (e.g., plain text), generates a title using the configured `title_model`
|
||||
|
||||
Priority order: HTML `<title>` (furniture layer) → h1/PDF title (body layer) → first section header → LLM generation.
|
||||
|
||||
Explicit titles passed via `title=` parameter always take precedence and are never overridden. When updating documents, existing titles are preserved — auto-generation only applies to untitled documents.
|
||||
|
||||
To generate titles for existing untitled documents, use [`rebuild --title-only`](../cli.md#rebuild-database).
|
||||
|
||||
### Local vs Remote Processing
|
||||
|
||||
**Local processing** (default):
|
||||
|
|
|
|||
|
|
@ -229,6 +229,28 @@ async for doc_id in client.rebuild_database(mode=RebuildMode.EMBED_ONLY):
|
|||
- `RebuildMode.FULL` - Re-convert from source files, re-chunk, re-embed (default)
|
||||
- `RebuildMode.RECHUNK` - Re-chunk from existing document content, re-embed
|
||||
- `RebuildMode.EMBED_ONLY` - Keep existing chunks, only regenerate embeddings
|
||||
- `RebuildMode.TITLE_ONLY` - Generate titles for untitled documents (no re-chunking or re-embedding)
|
||||
|
||||
### Generating Titles
|
||||
|
||||
Generate a title for an existing document on demand:
|
||||
|
||||
```python
|
||||
title = await client.generate_title(doc)
|
||||
if title:
|
||||
await client.update_document(document_id=doc.id, title=title)
|
||||
```
|
||||
|
||||
Uses the same two-tier approach as automatic ingestion: structural extraction from DoclingDocument metadata first, with LLM fallback via `processing.title_model`. Unlike ingestion, this method does not catch exceptions — if the LLM call fails, the error propagates.
|
||||
|
||||
To batch-generate titles for all untitled documents, use `RebuildMode.TITLE_ONLY`:
|
||||
|
||||
```python
|
||||
async for doc_id in client.rebuild_database(mode=RebuildMode.TITLE_ONLY):
|
||||
print(f"Generated title for {doc_id}")
|
||||
```
|
||||
|
||||
See [Automatic Title Generation](configuration/processing.md#automatic-title-generation) for configuration details.
|
||||
|
||||
## Maintenance
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue