CLI script to migrate existing tenants to the data model introduced
by #202-205. Idempotent and resumable — re-running is a no-op.
tools/migrate_06.py
- Step 1: backfill_lifecycle — infers documents.lifecycle_state from
analysis_jobs.status + chunks_json presence
Failed analyses, no completed -> Failed
Completed + chunks_json -> Chunked
Completed, no chunks_json -> Parsed
Otherwise -> Uploaded
- Step 2: materialize_chunks — promotes the latest chunks_json blob
for each doc into rows in the new chunks table. Stable id derivation
via uuid5(NAMESPACE_OID, '<doc>|<seq>|<text>') so re-running lands
on the same ids
- Step 3: backfill_links — for any doc that has chunks materialized,
creates a (doc, default-store) link in Ingested state with a freshly
computed chunkset_hash. Treats 'chunks exist' as proxy for 'doc was
ingested into the legacy single index' — sufficient for the typical
pre-0.6.0 deployment, with OpenSearch reindex documented separately
- Step 4: reaggregate — applies #203's aggregate_lifecycle rule so
doc-chunked transitions to Ingested
Persistence
- migration_progress table for resumability + per-step idempotency
CLI
python -m tools.migrate_06 # full migration
python -m tools.migrate_06 --dry-run # plan only, no writes
python -m tools.migrate_06 --only-step <name> # rerun a phase
Tests
- 9 tests: inference rule per (completed, failed, chunks_json) tuple,
end-to-end migration on a hand-built three-doc snapshot, idempotency
(second run = no writes), --dry-run writes nothing, deterministic
chunk ids across reruns
Refs #206
f85195be0c