Harden history against tag annotation failures; document tag restore
This commit is contained in:
parent
5453c00c95
commit
ce1b9ac88e
5 changed files with 94 additions and 3 deletions
|
|
@ -3,7 +3,7 @@
|
|||
|
||||
### Added
|
||||
|
||||
- Database tags: `haiku-rag tag create/list/delete`, tags shown in `history`. Vacuum retains versions back to the oldest tag.
|
||||
- Database tags: `haiku-rag tag create/list/delete/restore`, tags shown in `history`. `tag restore` creates a `before-restore-*` safety tag before changing live state. Vacuum retains versions back to the oldest tag.
|
||||
|
||||
### Changed
|
||||
|
||||
|
|
|
|||
33
docs/cli.md
33
docs/cli.md
|
|
@ -542,7 +542,7 @@ haiku-skills chat --use-entrypoints --skill medic
|
|||
|
||||
## Tags
|
||||
|
||||
Tags name the current database state so you can return to it. A tag covers every table in the database. Tagged versions survive `vacuum`; everything older than your oldest tag is retained until that tag is deleted, so remove tags you no longer need.
|
||||
A tag names the current database state. It is a logical snapshot composed of one LanceDB tag on each of the five tables, created from a single version snapshot.
|
||||
|
||||
```bash
|
||||
# Tag the current state, e.g. at deploy time or after an ingestion run
|
||||
|
|
@ -555,6 +555,37 @@ haiku-rag tag list
|
|||
haiku-rag tag delete release-1
|
||||
```
|
||||
|
||||
A tag present on every table is complete. A tag missing from some tables (created outside haiku.rag, or left behind by a failure) is partial. `tag list` marks partial tags. Partial tags can be listed and deleted but never restored.
|
||||
|
||||
Tagged versions survive `vacuum`. Vacuum retains the oldest tagged version and every newer version; versions older than the oldest tag remain eligible for cleanup. Delete tags you no longer need so cleanup can advance.
|
||||
|
||||
### Restore
|
||||
|
||||
`tag restore` brings the database back to a tagged state:
|
||||
|
||||
```bash
|
||||
haiku-rag tag restore release-1
|
||||
```
|
||||
|
||||
Restore changes the live state. It is not a read-only view: each table gets a new latest version equal to the tagged one, and reads and writes continue from there. Versions written after the tag remain in history until vacuum removes them.
|
||||
|
||||
Before changing anything, restore creates a complete safety tag (`before-restore-<timestamp>`) for the current state and reports it, so you always have a named path back:
|
||||
|
||||
```bash
|
||||
haiku-rag tag create release-1 --db /path/to/db.lancedb
|
||||
# Stop all writers before either restore.
|
||||
haiku-rag tag restore release-1 --db /path/to/db.lancedb --yes
|
||||
haiku-rag tag list --db /path/to/db.lancedb
|
||||
haiku-rag tag restore before-restore-YYYYMMDDTHHMMSSZ --db /path/to/db.lancedb --yes
|
||||
```
|
||||
|
||||
Restore is a maintenance operation:
|
||||
|
||||
- Stop all ingestion and other writers before restoring and keep them stopped until it finishes.
|
||||
- The operation is coordinated but not transactionally atomic across tables. On failure it attempts to roll back to the pre-restore state and reports whether the rollback succeeded.
|
||||
- `--yes` only skips the confirmation prompt. It provides no locking and no concurrent-writer protection.
|
||||
- Restore never migrates. Restoring a tag from an older haiku.rag version completes normally, and the next open reports the required migration. Run `haiku-rag migrate` explicitly.
|
||||
|
||||
### Version History
|
||||
|
||||
View version history for database tables:
|
||||
|
|
|
|||
|
|
@ -434,6 +434,28 @@ await client.vacuum()
|
|||
|
||||
This compacts tables and removes historical versions to keep disk usage in check. It’s safe to run anytime, for example after bulk imports or periodically in long‑running apps.
|
||||
|
||||
### Tags
|
||||
|
||||
Tag the current database state and restore it later, for example after an ingestion run. A tag covers all five tables and is created from a single version snapshot:
|
||||
|
||||
```python
|
||||
await client.store.create_tag("release-1")
|
||||
|
||||
tags = await client.store.list_tags()
|
||||
for name, info in tags.items():
|
||||
print(name, info.tables, info.complete)
|
||||
|
||||
await client.store.delete_tag("release-1")
|
||||
```
|
||||
|
||||
`restore_tag` brings the live database back to a tagged state. It creates a complete safety tag for the current state before changing any table and returns its name:
|
||||
|
||||
```python
|
||||
safety_tag = await client.store.restore_tag("release-1")
|
||||
```
|
||||
|
||||
Restore is a maintenance operation: stop all other writers first. A tag present on only some tables is partial; `list_tags` reports it via `missing_tables`, and partial tags can be deleted but never restored. Vacuum retains the oldest tagged version and everything newer, so delete tags you no longer need.
|
||||
|
||||
### Rebuilding the Database
|
||||
|
||||
```python
|
||||
|
|
|
|||
|
|
@ -300,7 +300,13 @@ class HaikuRAGApp: # pragma: no cover
|
|||
|
||||
self.console.print("[bold]Version History[/bold]")
|
||||
|
||||
tags = await store.list_tags()
|
||||
try:
|
||||
tags = await store.list_tags()
|
||||
except Exception as exc:
|
||||
tags = {}
|
||||
self.console.print(
|
||||
f"[yellow]Tag annotations unavailable: {escape(str(exc))}[/yellow]"
|
||||
)
|
||||
|
||||
for table_name in tables:
|
||||
versions = await store.list_table_versions(table_name)
|
||||
|
|
|
|||
|
|
@ -397,3 +397,35 @@ async def test_app_tag_rendering_escapes_markup(tmp_path):
|
|||
|
||||
output = app.console.export_text()
|
||||
assert output.count(hostile) == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_app_history_survives_tag_annotation_failure(tmp_path):
|
||||
"""history degrades to version history without annotations, with a
|
||||
warning, when aggregate tag loading fails."""
|
||||
from rich.console import Console
|
||||
|
||||
config = AppConfig(
|
||||
lancedb=LanceDBConfig(
|
||||
uri="s3://bucket/path",
|
||||
storage_options={"endpoint": "http://localhost:9000"},
|
||||
)
|
||||
)
|
||||
app = HaikuRAGApp(db_path=tmp_path / "db.lancedb", config=config)
|
||||
app.console = Console(record=True, width=200)
|
||||
|
||||
with patch("haiku.rag.store.engine.Store") as mock_store_cls:
|
||||
mock_store = AsyncMock()
|
||||
mock_store.list_tags = AsyncMock(side_effect=RuntimeError("tags boom"))
|
||||
mock_store.list_table_versions = AsyncMock(
|
||||
return_value=[{"version": 1, "timestamp": "2026-07-15 10:00:00"}]
|
||||
)
|
||||
mock_store_cls.return_value.__aenter__ = AsyncMock(return_value=mock_store)
|
||||
mock_store_cls.return_value.__aexit__ = AsyncMock(return_value=False)
|
||||
|
||||
await app.history(table="documents")
|
||||
|
||||
output = app.console.export_text()
|
||||
assert "v1" in output
|
||||
assert "2026-07-15 10:00:00" in output
|
||||
assert "tags boom" in output
|
||||
|
|
|
|||
Loading…
Reference in a new issue