Compare commits

...

35 commits

Author SHA1 Message Date
Pier-Jean Malandrino
27e3323bff feat(#257): surface gating via STUDIO_MODE + RAG_PIPELINE master flags
Some checks failed
Auto-close issues on release branch merge / Close referenced issues (push) Has been cancelled
Introduces two master feature flags that select which UI surface is
exposed, replacing the previous "delete legacy pages" approach with a
softer isolation:

- STUDIO_MODE_ENABLED  (default false) — legacy OCR-debug surface
- RAG_PIPELINE_ENABLED (default true)  — new doc-centric ingestion + viz

At least one master must be enabled (validated server-side at startup).
Sub-flags (inspect / linked / ask) are effective only when the RAG
pipeline master is on.

CHUNKS_MODE_ENABLED renamed to LINKED_MODE_ENABLED in anticipation of
T3 (Linked view replaces the Chunks tab). The DocMode union value
'chunks' is preserved for now and will be renamed in T3 alongside the
route segment, to keep this PR scoped.

Router-level guard added: requests to a route whose surface is disabled
are redirected to the other surface's landing page (or /home as a
defensive fallback). Logic extracted into a pure resolveSurface helper
with full test coverage.

i18n strings that pointed users to "Studio" rewritten to be surface-
agnostic ("from the library" / "depuis la bibliothèque") since Studio
is hidden by default in 0.6.1.

Backend:
- infra/settings.py: add studio_mode_enabled + rag_pipeline_enabled;
  rename chunks_mode_enabled → linked_mode_enabled; add at-least-one
  master validation in __post_init__
- api/schemas.py: HealthResponse exposes both master flags + renamed
  sub-flag
- main.py: health endpoint wires the new fields
- tests: surface-flag + renamed sub-flag assertions

Frontend:
- features/feature-flags/store: add studioMode + ragPipeline registry
  entries; rename chunksMode → linkedMode; sub-flags now require
  ragPipeline enabled; modeFlags() maps linkedModeEnabled → key 'chunks'
  (transitional)
- shared/routing/resolveSurface: pure helper + tests
- app/router: beforeEach guard consumes resolveSurface
- shared/i18n: Studio-pointing strings rewritten (en + fr) + test sync
- features/reasoning: stale "from StudioPage" comment generalized
2026-05-11 16:18:34 +02:00
Pier-Jean Malandrino
fa2c7d7525 fix(analyses): filter list endpoint by documentId — prevent cross-doc bbox leak
GET /api/analyses?documentId=X retournait toutes les analyses (le query
param était ignoré). Le frontend prenait alors la première COMPLETED de
la réponse, qui pouvait être celle d'un autre doc → bboxes d'un autre
doc projetées sur l'image en cours.

Backend
- analysis_repo: nouvelle méthode find_by_document(document_id, limit, offset)
- analysis_service: expose find_by_document
- api/analyses: GET /api/analyses accepte ?documentId=... (alias Pydantic
  pour respecter la règle ruff N803). Si présent, filtre via la nouvelle
  méthode; sinon comportement inchangé.
- test: test_list_analyses_filtered_by_document vérifie le routing vers
  find_by_document quand le query param est fourni.

Frontend (defensive)
- DocInspectTab / DocAskTab: filtre client-side
  analyses.find(a => a.documentId === requestedId && a.status === 'COMPLETED')
  pour rester safe même si un backend antérieur ignore le param.
2026-05-11 15:00:57 +02:00
Pier-Jean Malandrino
5b5da2cd4b test(#256): Karate UI regression for Doc tab chunk mode
Setup via API (upload + analyse + wait COMPLETED), then drive the UI
to the chunks tab and assert the chunk list renders. Includes a direct
HTTP probe on /api/documents/{id}/chunks so an API-only regression
fails the test even if the UI somehow swallows it.

Tagged @critical so it runs in the smoke suite.
2026-05-11 15:00:57 +02:00
Pier-Jean Malandrino
b4ad874600 fix(#256): drop pushDocumentToStore duplicate, align rechunk return shape
The frontend had two endpoints pointing at the same operation:
  - features/document/api.ts:pushDocumentToStore  → POST /push
  - features/chunks/api.ts:pushChunksToStore      → POST /chunks/push

Backend now exposes a single /chunks/push route (semantically more
accurate — it's chunks that get pushed, not the doc itself), so the
document-side variant is removed. document/store.ts:pushToStore now
delegates to chunks/api.

rechunkDocument returned {jobId} based on the old async-job model.
The new backend rechunk runs synchronously and returns the chunk
list directly. The DocsLibraryPage caller now reports a count
(via new docs.rechunkDone i18n key) instead of a fake job id.

Adds data-e2e=tab-{mode} on the doc workspace tab strip so the
new Karate scenario can target the chunks tab without text matching.

Extends chunks/api.test.ts with HTTP error-propagation guards on
every call — the original 404 bug would have surfaced in CI.
2026-05-11 15:00:57 +02:00
Pier-Jean Malandrino
ef80e22342 feat(#256): add /api/documents/{id}/chunks routes + ChunkService
The frontend was wired against /api/documents/{id}/chunks/* (canonical
doc-centric chunkset) but the backend never exposed those routes — the
chunk tab in the doc workspace 404'd. The domain entities (Chunk,
ChunkEdit, ChunkPush) and persistence repos already existed since #205;
what was missing was the service + API layer that connects them.

ChunkService owns all canonical chunkset invariants (sequence ordering,
soft-delete + audit log atomicity) and shares the chunker port with
AnalysisService so chunking strategy stays a single implementation.

AnalysisService grew a duck-typed promoter hook that copies the chunks
of the first successful analysis into the canonical chunkset. The hook
is idempotent so subsequent ad-hoc analyses (Studio / OCR Debug) never
overwrite hand-edited state.

Routes added (all additive, /api/documents prefix):
  GET    /{id}/chunks
  POST   /{id}/chunks
  PATCH  /{id}/chunks/{chunkId}
  DELETE /{id}/chunks/{chunkId}
  POST   /{id}/chunks/{chunkId}/split
  POST   /{id}/chunks/merge
  POST   /{id}/rechunk
  GET    /{id}/tree
  GET    /{id}/diff?store=...
  POST   /{id}/chunks/push
2026-05-11 15:00:57 +02:00
Pier-Jean Malandrino
75a2db087b docs(#256): flesh out design doc with chunk service + 9 routes plan 2026-05-11 15:00:57 +02:00
Pier-Jean Malandrino
7fe9a8a823 docs(#256): scaffold design doc for chunk-mode 404 2026-05-07 10:22:45 +02:00
Pier-Jean Malandrino
df1bbef143 feat(#251): expose backend error detail in apiFetch errors
Le wrapper apiFetch jetait `API error: 422` sans le body. Désormais lit
detail (string ou liste Pydantic) et le concatène au message :
`422: Neo4j config requires a non-empty 'index_name'`.

Permet aux pages StoreCreatePage / StoreEditPage de surfacer la cause
réelle du refus serveur dans l'errorMessage du form.
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
151505cdbf feat(#251): add Neo4j as a StoreKind
- domain : StoreKind.NEO4J ajouté à l'enum
- service : validation config par kind étendue (Neo4j requires non-empty index_name)
- frontend : Neo4jConfigForm (index_name + database optionnelle), dispatcher StoreConfigForm, dropdown kind, i18n FR/EN
- tests service : 2 cas Neo4j (create OK + missing index_name 422)

Auth (URI/user/password) reste pilotée par les variables d'environnement
globales (cf. infra/neo4j.py) — la config par store ne porte que les
paramètres routables (index, database).
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
d55066005d feat(#251): nav CRUD sur StoresListPage + StoreDetailPage
- StoresListPage : bouton "Nouveau store" header + ligne action Edit/Delete (delete bloqué sur 'default'), navigation par slug, colonne "Défaut", confirm window.confirm
- StoreDetailPage : actions Edit / Delete dans le header, redirection vers la liste après suppression
- api.test.ts : couvre la nouvelle shape StoreInfo (slug + isDefault + embedder)
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
7ad25aeb2a feat(#251): API client typé create/update/delete + StoreForm + Create/Edit pages
Frontend store API client :
- Types StoreInfo (avec slug + isDefault + embedder), StoreDetail, StoreCreatePayload, StoreUpdatePayload
- Fonctions createStore (POST), updateStore (PATCH), deleteStore (DELETE), fetchStore (GET /:slug)
- Tests Vitest étendus pour les 4 nouveaux endpoints

Composants formulaire (features/store/ui/) :
- StoreForm : champs communs (name/slug/kind/embedder/isDefault) avec validation slug locale, lock du slug pour 'default'
- StoreConfigForm : dispatcher dynamique par kind (extensible)
- OpenSearchConfigForm : champ indexName, validation requise

Pages CRUD :
- StoreCreatePage (/index/new) : formulaire + redirect liste
- StoreEditPage (/index/:store/edit) : préremplit via fetchStore, redirige vers la fiche

Routes + i18n :
- Routes ROUTES.STORE_CREATE et ROUTES.STORE_EDIT ajoutées (names + routes.ts)
- Clés i18n FR/EN sous stores.* et storeForm.*
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
245ef40a24 feat(#251): /api/stores router + wiring + schemas Pydantic + tests API
- Schemas camelCase : StoreInfoResponse, StoreResponse, StoreCreate/UpdateRequest, StoreDocEntryResponse
- Router /api/stores : GET (list), POST (create 201), GET/{slug}, PATCH/{slug}, DELETE/{slug} (204)
- Endpoints documents-in-store : GET /{slug}/documents, DELETE /{slug}/documents/{docId}
- Mapping erreurs StoreServiceError → HTTPException via http_status hint
- StoreService étendu : list_documents/remove_document avec injection optionnelle de document_repo (filename humain)
- Wiring lifespan : SqliteStoreRepository + SqliteDocumentStoreLinkRepository + StoreService sur app.state
- include_router(stores_router) après documents/analyses
- Tests API : 18 cas couvrant 200/201/204/404/409/422 + camelCase + cycle complet
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
9d2e9ab0f4 feat(#251): StoreService — CRUD orchestration + per-kind config validation
Couche service entre l'API et les repos SQLite :
- list_stores avec enrichissement document_count via document_store_links
- get/create/update/delete avec validations
- Slug normalisé lowercase, motif kebab-case strict
- Unicité name + slug (409 sur collision)
- Single-default invariant via clear_default_except
- Validation config par kind (OpenSearch require index_name) — extensible
- Delete refusé sur le store seedé "default" et sur tout store avec liens
- Erreurs typées (Validation/NotFound/Conflict) avec hint http_status
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
ce140bab43 feat(#251): SqliteStoreRepository — update/delete/find_by_name/clear_default_except
Étend le repo avec les méthodes manquantes pour le CRUD :
- find_by_name : valider unicité name au create/update
- update : remplace les champs mutables (id et created_at intacts)
- clear_default_except : promotion mono-default lors d'un changement de default
- delete : retourne True/False selon suppression effective

Tests dédiés sur chacune des nouvelles méthodes.
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
53d28c2282 fix(workspace): reset doc state on navigation (bbox / analysis state leak)
Quand l'utilisateur naviguait d'un doc à un autre via /docs/:id, Vue
réutilisait l'instance du composant — onMounted ne se redéclenchait pas,
laissant l'analyse précédente (et donc les bbox du StructureViewer)
visibles tant que le nouveau fetch n'était pas terminé.

- DocWorkspacePage: watch props.id pour recharger le doc + :key sur les
  onglets (forçe un remount propre, reset l'état interne du
  StructureViewer comme selectedPage/hiddenTypes).
- DocInspectTab / DocAskTab: watch props.docId avec immediate:true,
  reset l'état au début du load + guard de race condition (ignore les
  réponses pour un docId obsolète).
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
ed2f47fdc8 chore: re-track frontend/package-lock.json (CI requires it)
Le commit précédent l'avait untracké. Or .github/workflows/ci.yml et
release-gate.yml référencent explicitement le lockfile pour le cache npm
et la step `npm ci` exige sa présence. Resultat : CI cassé.

On garde docs/git-workflow/autonomy-mode.md ignoré (artefact local).
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
bb24098c21 chore: gitignore local autonomy-mode doc + untrack frontend lockfile
- Ajoute docs/git-workflow/autonomy-mode.md aux artefacts locaux ignorés.
- Retire frontend/package-lock.json du suivi git (reste sur disque).
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
1e9e57ab2d feat(#243,#244,#245): E8 — Stores pages (list, detail, query)
- features/store: api + types (StoreInfo, StoreDocEntry, QueryResult)
- StoresListPage: table with connection status, click → StoreDetail
- StoreDetailPage: doc table with StatusBadge, remove, bulk remove, link to query
- StoreQueryPage: query form (Ctrl+Enter), topK, scored results with doc links
- i18n: stores.* / storeDetail.* / storeQuery.* keys (FR + EN)

Closes #243
Closes #244
Closes #245
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
2718d86262 feat(#242): DocAskTab — Reasoning intégré au workspace (doc structure + question + trace cliquable) 2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
0062890830 feat(#240,#241): DocInspectTab — Markdown/Elements/Images from Docling analysis 2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
e40b5ba0a5 feat(#225,#224): vocabulary rename index→ingest + stale stores strip
- #225: rename all push/index/pousser/indexé labels to ingest/ingérer/ingéré
  across status tooltips, docs/chunks push modals, and legacy ingestion panel
- #224: add DocStoreLink type + storeLinks on Document; StaleStoresStrip
  component shows per-store state badges with one-click re-ingest buttons
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
88f109ed4c feat(#216): E4/E5 — Doc workspace shell, DocTreeRail, DocWorkspaceHeader, editable chunks editor
Closes #216
Closes #217
Closes #218
Closes #219
Closes #220
Closes #221
Closes #222
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
6527307553 feat(#211): E3 — Document library (/docs), filters, bulk actions, multi-file import, StatusBadge
- StatusBadge component with 6 lifecycle states, compact/full variants, tooltip (#215)
- DocsLibraryPage: table (Name/Status/Stores/Updated), empty state (#211)
- Filter bar: status pills, store chips, debounced search, URL sync (#212)
- Multi-select with sticky bulk bar: Re-chunk, Push to store modal, Delete (#213)
- DocsNewPage: multi-file drop zone with per-file progress, sequential upload (#214)
- Document.stores?: string[] field, formatRelativeTime(iso, locale) helper
- rechunkDocument / pushDocumentToStore API + store actions (rechunk / pushToStore)
- i18n keys status.*, docs.*, docsNew.* in FR + EN

Closes #211, Closes #212, Closes #213, Closes #214, Closes #215
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
7b6d64c434 feat(#210): feature-flag mode gating + deep-link redirect
Backend
- HealthResponse exposes inspectModeEnabled / chunksModeEnabled /
  askModeEnabled (additive; defaults true). main.py /api/health
  populates them from settings.
- infra/settings.py: three new env-var-driven booleans (defaults true)
  parsed in from_env() like the existing reasoning_enabled flag.
- tests/test_api_endpoints.py: extra assertion that /api/health
  surfaces the three new fields with their defaults.

Frontend — flag store
- features/feature-flags/store.ts: FeatureFlag union extended with
  inspectMode / chunksMode / askMode. New entries in featureRegistry
  are gated on context fields populated from health. Missing fields
  fall back to true so a frontend pointed at an older backend keeps
  every mode visible.
- store gains a modeFlags() helper returning Record<DocMode, boolean>
  so the routing guard does not need to know the FeatureFlag union.

Frontend — routing
- shared/routing/resolveMode.ts: pure resolver. If the requested mode
  is enabled, return it; else first enabled in priority ask > chunks
  > inspect; else null.
- app/router/index.ts: beforeEach guard scoped to ROUTES.DOC_WORKSPACE.
  Disabled mode → rewrite ?mode= to the first enabled one. All three
  off → redirect to /docs?reason=no-mode-enabled.

Frontend — flash
- pages/DocsLibraryPage.vue: shows a banner when ?reason=no-mode-enabled
  is set. #211 will move this into the proper library page banner.
- i18n flags.allModesDisabled added in fr + en.

Tests
- shared/routing/resolveMode.test.ts (6 cases): every (requested,
  enabled) combination including all-disabled, priority order,
  missing requested.
- features/feature-flags/store.test.ts: three new cases covering the
  new fields in /api/health, fall-back-to-true on missing fields, and
  modeFlags() shape.

Refs #210
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
058d718116 feat(#209): rework sidebar nav (Home / Docs / Stores / Runs / Settings)
The sidebar was the project's actual nav (the design doc called it
'top nav' but the existing app is sidebar-driven). Five entries
reflecting the doc-centric IA replace the seven analysis-centric ones.

Sidebar
- shared/ui/AppSidebar.vue: data-driven over a NavItem[] table
  instead of seven hard-coded RouterLinks. Inline SVG icon
  components keep the visual weight close to the previous version
  with no new dependency.
- Five entries: Home / Docs (★ primary, bolder label) / Stores /
  Runs / Settings.
- Active state via matchesActive(path, prefixes): exact match for
  Home, prefix match (with segment / query boundary) for the others.
  Pure helper in shared/ui/navActive.ts, unit tested.

Removed from the sidebar (legacy pages still reachable by URL):
  Studio, Documents, Search, Reasoning, History.

i18n
- nav.docs / nav.stores / nav.runs added in fr + en, grouped under
  the 0.6.0 nav block.
- Legacy nav labels (nav.studio / nav.documents / nav.history /
  nav.reasoning / nav.search) kept — the legacy pages still render
  headings using them. Duplicate nav.search entries removed (the key
  now lives once per locale, in the nav block).

Tests
- shared/ui/navActive.test.ts (5 cases): exact-match for Home,
  prefix boundary semantics, multi-prefix matching, empty list edge.

Refs #209
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
ab5de2aac3 feat(#208): doc workspace breadcrumb (Studio > <doc> > <mode>)
Adds the breadcrumb anchoring the user across modes on the doc
workspace. Empty / hidden on routes that don't opt in.

Components
- shared/breadcrumb/AppBreadcrumb.vue: data-driven, accessible
  (<nav aria-label>, <ol>, aria-current=page on the leaf).
  Renders nothing when crumbs.length === 0.
- shared/breadcrumb/types.ts: Crumb = LinkCrumb | LeafCrumb
  discriminated union.
- shared/breadcrumb/store.ts: Pinia store + useCrumbs(source)
  composable that auto-clears on unmount. Accepts a static array
  OR a reactive ref/computed so pages with async doc fetches can
  rebuild crumbs as data lands.
- shared/breadcrumb/text.ts: truncate(text, max) helper.

Wiring
- App.vue main outlet now sits below <AppBreadcrumb>; the shell
  reads from useBreadcrumbStore so pages don't need teleports.
- DocWorkspacePage provides Studio > <id-truncated> > <mode-label>;
  once the doc is fetched (#216 / E4), the id placeholder will
  swap for the truncated filename.

i18n
- breadcrumb.aria, breadcrumb.studio, breadcrumb.mode.{ask,inspect,
  chunks} added in fr + en.

Tests
- shared/breadcrumb/text.test.ts: 5 cases on truncate.
- shared/breadcrumb/store.test.ts: store actions + useCrumbs reactive/
  static seeding (the lifecycle-clear path is covered end-to-end by
  the integration tests landing in #211 / #216 — the project doesn't
  use @vue/test-utils so a unit test for onBeforeUnmount is overkill).

Refs #208
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
47e0d197b3 feat(#207): document-centric routing skeleton
Adds the routing scaffold for the doc-centric pivot. Each new route
renders a placeholder page until E3/E4/E5 implement them; legacy
routes (/studio, /documents, /history, /search, /reasoning) keep
working in parallel.

New routes (Vue Router, history mode):
  /docs                   library (placeholder, #211)
  /docs/new               import (placeholder, #214)
  /docs/:id?mode=         workspace (placeholder, #216)
  /index                  stores list (placeholder, 0.7.0)
  /index/:store           store detail (placeholder)
  /index/:store/query     RAG playground (placeholder)
  /runs                   run history (placeholder)
  /runs/:id               run detail (placeholder)

Mode parsing
- shared/routing/modes.ts: DocMode union ('ask'|'inspect'|'chunks'),
  parseMode() returns the default ('ask') for missing or unknown
  values. #210 will layer feature-flag-aware redirection on top.

Route names
- shared/routing/names.ts: ROUTES typed const so callers do
  router.push({ name: ROUTES.DOC_WORKSPACE, ... }) instead of
  stringly-typed names.

Pages
- ComingSoonShell shared component: card + back-home link, themed
  with existing CSS tokens.
- 8 thin placeholder pages (one per new route) that compose the shell
  and forward route params.
- i18n keys under comingSoon.* added in fr + en.

Tests
- shared/routing/modes.test.ts (10 cases): isDocMode + parseMode +
  ALL_MODES invariants.
- app/router/router.test.ts (5 cases): every doc-centric route
  resolves to a component, legacy routes still work, doc workspace
  receives id and parsed mode as props, unknown mode falls back to
  ask, unknown path redirects to home.

Routes table extracted to routes.ts so tests build a router with
createMemoryHistory() (no window required) instead of needing the
production createWebHistory() router.

Refs #207
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
8234a3c23f docs(design): E2 design docs for 0.6.0 navigation refactor
Adds technical design docs for the navigation epic of the doc-centric
pivot:

- 207 — Document-centric routing (/docs, /docs/:id?mode=, /index/:store, /runs)
- 208 — Doc workspace breadcrumb (Studio > <doc> > <mode>)
- 209 — Sidebar nav rework (Home / Docs / Stores / Runs / Settings)
- 210 — Feature flag mode gating (deep-link redirect + flag exposure)

Status: Accepted on all four. Each doc spells out the contract,
alternatives considered, risks per audit dimension, and testing
strategy. Backwards-compatibility is preserved throughout: legacy
routes and pages keep working until E3/E4/E5 explicitly replace them.

Refs #207 #208 #209 #210
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
0c09274836 chore(#206): backfill 0.6.0 doc-centric data model
CLI script to migrate existing tenants to the data model introduced
by #202-205. Idempotent and resumable — re-running is a no-op.

tools/migrate_06.py
- Step 1: backfill_lifecycle — infers documents.lifecycle_state from
  analysis_jobs.status + chunks_json presence
    Failed analyses, no completed       -> Failed
    Completed + chunks_json             -> Chunked
    Completed, no chunks_json           -> Parsed
    Otherwise                           -> Uploaded
- Step 2: materialize_chunks — promotes the latest chunks_json blob
  for each doc into rows in the new chunks table. Stable id derivation
  via uuid5(NAMESPACE_OID, '<doc>|<seq>|<text>') so re-running lands
  on the same ids
- Step 3: backfill_links — for any doc that has chunks materialized,
  creates a (doc, default-store) link in Ingested state with a freshly
  computed chunkset_hash. Treats 'chunks exist' as proxy for 'doc was
  ingested into the legacy single index' — sufficient for the typical
  pre-0.6.0 deployment, with OpenSearch reindex documented separately
- Step 4: reaggregate — applies #203's aggregate_lifecycle rule so
  doc-chunked transitions to Ingested

Persistence
- migration_progress table for resumability + per-step idempotency

CLI
    python -m tools.migrate_06              # full migration
    python -m tools.migrate_06 --dry-run    # plan only, no writes
    python -m tools.migrate_06 --only-step <name>   # rerun a phase

Tests
- 9 tests: inference rule per (completed, failed, chunks_json) tuple,
  end-to-end migration on a hand-built three-doc snapshot, idempotency
  (second run = no writes), --dry-run writes nothing, deterministic
  chunk ids across reruns

Refs #206
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
648f2a5c0d feat(#205): promote chunks to first-class entities + audit trail
The data and domain layers for the chunks editor (#219-224 in 0.6.0).
Chunks were previously stored as a JSON blob in analysis_jobs.chunks_json;
this commit makes them first-class persisted entities with stable IDs,
soft-delete, and an immutable audit log.

Domain
- Chunk: persistent entity with id, document_id, sequence, text,
  headings, source_page, bboxes, doc_items, token_count, timestamps,
  deleted_at (soft delete)
- ChunkEdit: immutable audit row (action, actor, at, before, after,
  parents, children, reason)
- ChunkPush: snapshot of which chunk_ids landed in which store at push
- ChunkEditAction enum: insert/update/delete/merge/split
- domain/chunk_editing.py: pure operations on a chunkset (insert,
  update, delete, merge, split). Each returns a new chunkset and the
  affected chunk(s); errors raise ChunkEditingError.

Persistence
- Three new tables: chunks, chunk_edits, chunk_pushes (FK + indexes)
- SqliteChunkRepository (insert, insert_many, update, soft_delete,
  find_for_document, find_by_id; respects deleted_at)
- SqliteChunkEditRepository (append-only audit log; paginated reads
  ordered newest-first; per-chunk history)
- SqliteChunkPushRepository (per-(doc, store) latest snapshot)

Ports
- ChunkRepository, ChunkEditRepository, ChunkPushRepository protocols
  added to domain/ports.py

Tests
- 17 tests for the pure chunk-editing operations covering insert /
  update / delete / merge / split, sequence shifts, lineage, error
  paths (out-of-range, missing id, deleted target, cross-document)
- 11 tests for the three repositories: round-trips, soft-delete
  filtering, history ordering, lineage round-trip, cascade-delete with
  document, find_latest semantics

Service orchestration (ChunkEditingService — atomic chunk + audit
write) and the API endpoints land in a follow-up commit on the same
feature branch / next release. The data + domain foundation here is
what unblocks #219-224.

Refs #205
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
4b59ef2330 feat(#204): deterministic chunkset hash for auto-stale detection
Adds the pure-domain hash function that summarises a chunkset for
stale-detection purposes. Recorded on each DocumentStoreLink at push
time (#203 ships the column slot); compared against the recomputed
current hash to flip a link to Stale when the source has drifted.

domain/hashing.py
- chunkset_hash(chunks: Iterable[ChunkResult]) -> str
- SHA-256 over (text, source_page, headings) per chunk
- Excludes bboxes / doc_items / token_count by design
- 0x1F separator between chunks defends against the join-attack
  (split A+B vs concat AB)

Tests
- 9 tests: determinism, sensitivity per included field, exclusion of
  rendering-only fields, join-attack resistance, order sensitivity,
  empty-input invariant
- Locked fixture: a hand-built 3-chunk input has a fixed expected hash;
  CI fails loud if anyone changes the canonical inputs without updating
  the fixture deliberately (and the release notes)

Service integration (recompute on chunk write, set on push) lands with
#205 once chunks are first-class — direct integration on the legacy
chunks_json path is deliberately deferred to keep #204 focused.

Refs #204
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
d81f36d899 feat(#203): per (document, store) ingestion state
Introduces the data layer for multi-store ingestion. Documents can now
live in multiple stores, each with its own Ingested/Stale/Failed state.
The doc-level lifecycle (#202) becomes the aggregate over all per-store
links, computed by a pure domain function.

Domain
- Store entity (name, slug, kind, embedder, config, is_default)
- DocumentStoreLink entity with mark_ingested / mark_stale / mark_failed
  helpers
- StoreKind and DocumentStoreLinkState enums
- aggregate_lifecycle(): pure function — Failed > Stale > Ingested
  > fallback (the doc's pre-link Uploaded/Parsed/Chunked state)

Persistence
- New tables 'stores' and 'document_store_links' with the right indexes
  (doc_id, store_id, state) and a UNIQUE (doc, store) on the link
- Default 'opensearch' store seeded idempotently in init_db, embedder
  pulled from DEFAULT_EMBEDDER (fallback bge-m3)
- SqliteStoreRepository (find_by_slug, find_by_id, get_default, …)
- SqliteDocumentStoreLinkRepository with ON CONFLICT … DO UPDATE upsert

Ports
- StoreRepository and DocumentStoreLinkRepository protocols added

Tests
- 14 new tests: seed idempotency, insert/find round-trips, UNIQUE
  constraint, cascade delete with the document, every link state
  round-trips, aggregation rule with all branches

Refs #203
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
ff0b795b6d feat(#202): introduce Document lifecycle state machine
Adds a first-class lifecycle state to every document, distinct from
AnalysisJob.status. The lifecycle describes the document as a whole and
is the foundation for the doc-centric pivot in 0.6.0.

Domain
- DocumentLifecycleState enum (Uploaded/Parsed/Chunked/Ingested/Stale/Failed)
- Document.lifecycle_state and lifecycle_state_at fields
- Document.transition_to() validates against a transition table
  (domain/lifecycle.py) and returns a DocumentLifecycleChanged event
- InvalidLifecycleTransitionError on disallowed transitions

Persistence
- ALTER TABLE documents to add the two columns (default 'Uploaded')
- New index idx_documents_lifecycle_state for filter perf
- _COLUMN_MIGRATIONS refactored to support multiple tables
- _POST_MIGRATION_DDL list for indexes on freshly-added columns
- SqliteDocumentRepository.update_lifecycle()

Services
- AnalysisService drives transitions on parse / chunk / re-chunk / fail
  via _transition_document(); idempotent and resilient (logs WARN and
  continues if a stale state is somehow encountered)

API
- DocumentResponse exposes lifecycleState + lifecycleStateAt
  (additive — existing 'status' field kept for backwards compat)

Frontend
- Document type extended with lifecycleState and lifecycleStateAt
- DocumentLifecycleState union literal mirroring the backend enum

Tests
- 24 new tests in test_lifecycle.py covering transitions, idempotency,
  invariant preservation, and event emission
- test_repos.py: round-trip + every-enum-value check + update_lifecycle
- test_chunking.py: rechunk path now mocks document_repo correctly

Refs #202
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
805d881d65 docs(design): E1 design docs for 0.6.0 doc-centric ingest
Adds technical design docs for the foundation of the doc-centric ingest pivot:

- 202 — Document lifecycle state machine
- 203 — Per (document, store) ingestion state
- 204 — Auto-detect Stale state via chunk content hash
- 205 — Audit trail for chunk edits (chunks → first-class entity)
- 206 — Migration of existing documents to the new model

Status: Accepted on all five. Each doc spells out the domain entities,
persistence schema, services orchestration, API contract, alternatives
considered, risks, and testing strategy. ADR placeholders called out
where load-bearing decisions warrant a follow-up document.

Refs #202 #203 #204 #205 #206
2026-05-05 09:38:39 +02:00
Pier-Jean Malandrino
5444838d6c chore: gitignore local-only artifacts on release/0.6.0
Bootstrap commit for the doc-centric ingest release:
- .github/ISSUE_TEMPLATE/issue.md (local copy)
- docs/claude-code-commands.md (personal notes)
- docs/design/TEMPLATE.md (local design template)
- document-parser/tests/test_local_converter.py (local test scratch)
2026-05-05 09:38:39 +02:00
124 changed files with 18005 additions and 229 deletions

8
.gitignore vendored
View file

@ -49,3 +49,11 @@ profiles/
# E2E tests — Maven build outputs & Chrome user data
e2e/**/target/
# Local-only artifacts (release/0.6.0 bootstrap)
.github/ISSUE_TEMPLATE/issue.md
docs/claude-code-commands.md
docs/design/TEMPLATE.md
document-parser/tests/test_local_converter.py
docs/git-workflow/autonomy-mode.md

View file

@ -0,0 +1,292 @@
# Design: Document lifecycle state machine
- **Issue:** #202
- **Title on issue:** [FEATURE] Introduce Document lifecycle state machine
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** backend: domain · persistence · services · api · frontend: features/document · shared
- **Audit dimensions likely touched:** Hexagonal Architecture · DDD · Tests · Documentation
- **ADR spawned?:** no
---
## 1. Problem
Studio today tracks ingestion as a side-effect of an `AnalysisJob`. The only "state" a document carries is implicit — derived by joining the document with its latest analysis job (`PENDING`, `RUNNING`, `COMPLETED`, `FAILED`) and, separately, by checking whether OpenSearch holds rows for `doc_id`. There is no explicit, persisted lifecycle of the document itself.
This blocks the 0.6.0 doc-centric pivot: we cannot show a document library `/docs` with status badges, we cannot tell the user "this doc is ingested but stale on store X", and we cannot drive a chunks editor that knows whether its draft is committed. Every later feature in E3, E4, and E5 reads or transitions a document state.
This design introduces a first-class **document lifecycle** as a domain concept: a single canonical state on each document, validated transitions, persisted, and surfaced via the API.
## 2. Goals
- [ ] Add a `DocumentLifecycleState` enum to the domain with six states: `Uploaded`, `Parsed`, `Chunked`, `Ingested`, `Stale`, `Failed`.
- [ ] `Document` carries `lifecycle_state` (current state) + `lifecycle_state_at` (last transition timestamp).
- [ ] State transitions are validated in the domain layer; invalid transitions raise `InvalidLifecycleTransition`.
- [ ] State changes emit a domain event `DocumentLifecycleChanged(previous, current, at)`.
- [ ] Persisted via aiosqlite migration; existing rows default to `Uploaded` (refined by #206).
- [ ] Surfaced on the document API DTO (camelCase: `lifecycleState`, `lifecycleStateAt`).
## 3. Non-goals
- Per-`(doc, store)` state — that is **#203** and uses a different table.
- Auto-stale detection via hash — that is **#204**; this issue only declares the `Stale` state value.
- UI rendering of the badge — that is **#215**.
- Migrating production data — that is **#206**; this issue ships the schema + default value only.
- Refactoring `AnalysisJob.status`. The two concepts coexist: `AnalysisJob.status` describes **a single conversion attempt**; `Document.lifecycle_state` describes **the doc as a whole**.
## 4. Context & constraints
### Existing code surface
- `document-parser/domain/models.py``Document` dataclass (line 27), `AnalysisJob` (line 38), `AnalysisStatus` enum.
- `document-parser/domain/value_objects.py` — value objects.
- `document-parser/persistence/database.py``_MIGRATIONS` list and `_run_migrations()` checking PRAGMA `table_info()`.
- `document-parser/persistence/document_repo.py``DocumentRepository` (aiosqlite).
- `document-parser/api/documents.py` — REST endpoints.
- `document-parser/api/schemas.py` — Pydantic DTOs with `alias_generator=_to_camel`.
- `frontend/src/features/document/store.ts` and `api.ts`.
### Hexagonal Architecture constraints (backend)
- `DocumentLifecycleState` is a **value object** — it lives in `domain/value_objects.py` with zero imports from `api`/`persistence`/`infra`.
- The transition rules are domain logic — they live as methods on `Document` (or a small `DocumentLifecycle` policy module in `domain/`). No HTTP, no DB.
- `DocumentRepository` (the port) gains a write path for the new fields. The aiosqlite adapter is the only place that knows about the column.
- API layer translates the enum to camelCase string via Pydantic.
### Deployment modes
Both `latest-local` and `latest-remote` images use the same SQLite schema → migration applies to both. No HF Space-specific concern. No feature flag (the lifecycle is always on; only UI surfaces are flagged).
### Hard constraints
- SQLite schema additive only. No column drops, no rename. New column has a `DEFAULT` so existing rows do not break on read.
- API contract additive only. New fields appear; nothing is removed.
- `pages_json` stays snake_case (existing exception).
## 5. Proposed design
### 5.1 Domain
Add to `document-parser/domain/value_objects.py`:
```python
from enum import StrEnum
class DocumentLifecycleState(StrEnum):
UPLOADED = "Uploaded"
PARSED = "Parsed"
CHUNKED = "Chunked"
INGESTED = "Ingested"
STALE = "Stale"
FAILED = "Failed"
```
Allowed transitions (a directed graph; `* → Failed` always allowed):
```
Uploaded → Parsed | Failed
Parsed → Chunked | Failed
Chunked → Ingested | Chunked | Failed # re-chunking is allowed
Ingested → Stale | Chunked | Failed # re-ingest stays Ingested via 203
Stale → Ingested | Chunked | Failed
Failed → Uploaded | Parsed | Chunked # explicit retry sets the new target
```
`Stale` is set by the auto-detect logic (#204) — never reached as a result of a manual action.
Add to `document-parser/domain/models.py`:
```python
@dataclass
class Document:
...
lifecycle_state: DocumentLifecycleState = DocumentLifecycleState.UPLOADED
lifecycle_state_at: datetime | None = None
def transition_to(self, target: DocumentLifecycleState, *, now: datetime) -> "DocumentLifecycleChanged":
if not _is_allowed(self.lifecycle_state, target):
raise InvalidLifecycleTransition(self.lifecycle_state, target)
previous = self.lifecycle_state
self.lifecycle_state = target
self.lifecycle_state_at = now
return DocumentLifecycleChanged(self.id, previous, target, now)
```
`InvalidLifecycleTransition` lives in `domain/exceptions.py` (create if missing). `_is_allowed` is a pure function over a static transition table.
`DocumentLifecycleChanged` is a frozen dataclass in `domain/events.py`. No event bus is wired in 0.6.0 — the event is *returned* from the transition call so services can log / persist / publish later. This avoids introducing infra in this issue.
### 5.2 Persistence
Schema migration appended to `_MIGRATIONS` in `document-parser/persistence/database.py`:
```sql
ALTER TABLE documents ADD COLUMN lifecycle_state TEXT NOT NULL DEFAULT 'Uploaded';
ALTER TABLE documents ADD COLUMN lifecycle_state_at TEXT;
CREATE INDEX IF NOT EXISTS idx_documents_lifecycle_state ON documents(lifecycle_state);
```
The `_run_migrations()` PRAGMA check ensures idempotency — re-running on an already-migrated DB is a no-op. The index supports the `/docs` library filter by status (#212).
`DocumentRepository.save(doc)` and `update_lifecycle(doc_id, state, at)` are the two write paths. Read returns the columns directly populated into the dataclass.
### 5.3 Infra adapters
None. The lifecycle is database-only.
### 5.4 Services
`DocumentService` and `AnalysisService` are the two callers that drive transitions.
| Trigger | From | To | Caller |
|---|---|---|---|
| Upload completes | — | `Uploaded` | `DocumentService.upload()` |
| Parse succeeds | `Uploaded` (or `Failed` retry) | `Parsed` | `AnalysisService` |
| Chunking succeeds | `Parsed` (or `Chunked` re-chunk) | `Chunked` | `AnalysisService` |
| Ingestion succeeds | `Chunked` (or `Stale`) | `Ingested` | `IngestionService` (touched in #203) |
| Failure on any pipeline step | any | `Failed` | the failing service |
Transition calls are atomic with the underlying write (e.g., chunking writes `chunks_json` *and* transitions to `Chunked` in the same SQL transaction).
### 5.5 API
`DocumentResponse` (in `document-parser/api/schemas.py`) gains:
```python
class DocumentResponse(BaseModel):
...
lifecycle_state: str = Field(serialization_alias="lifecycleState")
lifecycle_state_at: datetime | None = Field(default=None, serialization_alias="lifecycleStateAt")
```
No new endpoint. The existing `GET /api/documents` and `GET /api/documents/{id}` start returning the two fields. No 4xx removal — purely additive.
`POST /api/documents/{id}/lifecycle` is **not** added in this issue; transitions are driven by pipeline events, not by the user.
### 5.6 Frontend — feature module
Touched: `frontend/src/features/document/`.
- `api.ts` — bump the `Document` type with `lifecycleState: DocumentLifecycleState` and `lifecycleStateAt: string | null`.
- `store.ts` — store retains the new fields as-is.
- No UI work — that is #211 / #215. This issue ships the typed surface only so #211 can render against it.
`shared/types.ts` gains a re-exported `DocumentLifecycleState` union literal type matching the backend enum.
### 5.7 Cross-cutting
- No feature flag. The lifecycle is permanent infrastructure.
- i18n: tooltip strings for the six states are added in `shared/i18n.ts`, keyed `lifecycle.<state>`. Used by #215.
## 6. Alternatives considered
### Alternative A — Reuse `AnalysisJob.status`
- **Summary:** Promote `AnalysisJob.status` to the document's state, removing the separate `Document.lifecycle_state`.
- **Why not:** `AnalysisJob` has a 1:N relationship with `Document` (a doc can be re-analyzed). The status of the *latest* job is not the same thing as the lifecycle of the *document*. It also conflates parse/chunk/ingest into one state machine, which #202#205 explicitly want to separate.
### Alternative B — Computed state, no column
- **Summary:** Derive the state on-the-fly from analysis-job rows + OpenSearch presence.
- **Why not:** Joining + remote calls on every document list query is too expensive for `/docs` with 1k+ rows. A persisted column lets us index it for the filter (#212). The cost of denormalisation is one extra write per pipeline step.
## 7. API & data contract
### Endpoints
| Method | Path | Request | Response | Breaking? |
|--------|------|---------|----------|-----------|
| GET | `/api/documents` | — | `DocumentResponse[]` (now with `lifecycleState`, `lifecycleStateAt`) | No (additive) |
| GET | `/api/documents/{id}` | — | `DocumentResponse` (now with `lifecycleState`, `lifecycleStateAt`) | No (additive) |
### Persistence schema
```sql
ALTER TABLE documents ADD COLUMN lifecycle_state TEXT NOT NULL DEFAULT 'Uploaded';
ALTER TABLE documents ADD COLUMN lifecycle_state_at TEXT;
CREATE INDEX IF NOT EXISTS idx_documents_lifecycle_state ON documents(lifecycle_state);
```
Existing rows default to `Uploaded`; #206 refines them.
### Env vars / config
None.
### Breaking changes
Additive only.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Drift between `lifecycle_state` and the actual pipeline state (e.g. crash mid-write leaves `Chunked` recorded but no chunks) | DDD | Medium | Medium | Failed `/docs` rendering; tests catch on integration | Wrap the data write + transition in a single SQL transaction; never write the transition before the data |
| Existing migrations break on environments mid-flight | CI/Build | Low | High | Migration test on a snapshot DB | The migration is purely additive (`ADD COLUMN ... DEFAULT`) — safe to apply on any version |
| Confusion between `AnalysisJob.status` and `Document.lifecycle_state` for newcomers | Documentation | High | Low | Reviews ask | A short paragraph in `docs/architecture.md` describing the two concepts side by side |
| `Failed` state without a known reason makes debugging harder | Tests / Documentation | Medium | Medium | Operator complaints | The transition payload carries an optional `reason: str` persisted in a small JSON column or in the analysis-job's `error_message` |
## 9. Testing strategy
### Backend — pytest (`document-parser/tests/`)
- **Unit (`tests/domain/`):**
- `test_lifecycle_transitions.py` — table-driven test of every (from, to) pair: allowed → updates state, disallowed → raises `InvalidLifecycleTransition`.
- `test_lifecycle_event.py` — successful transition returns the right `DocumentLifecycleChanged` event.
- **Persistence (`tests/persistence/`):**
- `test_document_repo_lifecycle.py` — write a doc, read it back, lifecycle fields round-trip; default value is `Uploaded`; index exists.
- **Services / integration:**
- `test_analysis_service_transitions.py` — parse pipeline drives `Uploaded → Parsed`; chunking drives `Parsed → Chunked`; failure → `Failed`.
### Frontend — Vitest
- `features/document/api.test.ts` — DTO parses `lifecycleState` correctly into the typed enum.
- `features/document/store.test.ts` — store exposes `lifecycleState` on the cached document.
### E2E — Karate UI
Out of scope for this issue. E2E coverage lands with #211 (the library page that renders the badge).
### Manual QA
1. `docker compose -f docker-compose.dev.yml up`.
2. Upload a doc → `GET /api/documents/{id}` returns `"lifecycleState": "Uploaded"`.
3. Run analysis → state becomes `"Parsed"` then `"Chunked"`.
4. Force a failure (corrupt PDF) → state becomes `"Failed"`.
## 10. Rollout & observability
### Release branch
`release/0.6.0` (this work).
### Feature flag / staged rollout
None. Lifecycle persistence is always on; UI surfaces are flagged separately (per #210).
### Observability
- Each transition is logged at `INFO` with structured keys: `event=lifecycle_changed doc_id=<id> from=<state> to=<state>`.
- No new Prometheus counter in this issue (added in #211 if useful for the library page).
### Rollback plan
The migration is additive. Reverting the code (without dropping the column) leaves the column populated but unused — safe. If the column itself must go, write a follow-up migration; SQLite supports column drop since 3.35 and we run a recent enough version.
## 11. Open questions
- Should we record the `reason` for `Failed` transitions on the document directly, or rely on the linked `AnalysisJob.error_message`? **Decision for 0.6.0:** rely on `AnalysisJob.error_message`; revisit if multiple non-analysis sources of failure emerge.
- Do we want a typed `DocumentLifecycleEvent` table for audit, or are logs enough? **Decision:** logs only in 0.6.0; #205's `chunk_edits` table is the audit substrate, doc-level events can be added later if needed.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/202
- **Related issues:** #203 (per-store state), #204 (auto-stale), #205 (audit trail), #206 (migration), #211 (library), #215 (status badges)
- **ADRs:** none planned
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`
- Audit master: `docs/audit/master.md`

View file

@ -0,0 +1,329 @@
# Design: Per (document, store) ingestion state
- **Issue:** #203
- **Title on issue:** [FEATURE] Track ingestion state per (document, store) pair
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** backend: domain · persistence · services · api · infra · frontend: features/ingestion · features/document · shared
- **Audit dimensions likely touched:** Hexagonal Architecture · DDD · Decoupling · Tests · Documentation
- **ADR spawned?:** ADR-NNN — "Multiple vector stores as first-class entities"
---
## 1. Problem
In the current architecture, "the vector store" is a single OpenSearch index hard-coded to `docling-studio-chunks`. Documents either are or are not indexed in that one place. The new doc-centric model assumes documents can live in **multiple stores** simultaneously — `rh-corpus-v3`, `legal-v1`, a snapshot for a customer A/B — and each `(doc, store)` pair has its own state.
This is also where the killer flow lives: a customer reports bad answers from a specific store; the eng opens the doc, sees that it is `Ingested` in `rh-corpus-v3` and `Stale` in `rh-corpus-v2`, fixes the chunks, and re-ingests **the right one**. Without a per-pair state, that flow is impossible.
## 2. Goals
- [ ] Introduce a `Store` entity in the domain (id, name, kind, config, embedder).
- [ ] Introduce a `DocumentStoreLink` entity tying a document to a store with its own state and metadata.
- [ ] States on the link: `Ingested`, `Stale`, `Failed`. (No `Uploaded`/`Parsed`/`Chunked` — those are doc-level only.)
- [ ] The `Document.lifecycle_state` (#202) aggregates the per-store states by a documented rule.
- [ ] API exposes the per-store list on the document DTO.
- [ ] One default `Store` row is seeded on first migration so existing single-index deployments keep working.
## 3. Non-goals
- Auto-stale detection logic — that is **#204**; this issue ships the schema slot for the content hash but not the detection.
- Multi-tenant isolation between stores — the existing tenant model (if any) is not changed; stores are global today.
- A UI to create/manage stores — that is a follow-up. For 0.6.0 we ship the data model and the seed; a single store is enough for the killer flow on a single tenant.
- Cross-doc bulk re-ingest — that is **#213**; this issue exposes the per-link state needed by it.
## 4. Context & constraints
### Existing code surface
- `document-parser/domain/ports.py``VectorStore` protocol (line 124).
- `document-parser/infra/opensearch_store.py``OpenSearchStore` adapter.
- `document-parser/services/ingestion_service.py``IngestionService.ingest()` and `ensure_index()`.
- `document-parser/persistence/database.py` — schema + migrations.
- `document-parser/api/ingestion.py` — endpoints.
- `frontend/src/features/ingestion/` — Pinia store + API client.
### Hexagonal Architecture constraints (backend)
- `Store` and `DocumentStoreLink` are domain entities (`domain/models.py`). Pure data + invariants. No ORM concern.
- `StoreRepository` and `DocumentStoreLinkRepository` are ports (`domain/ports.py`).
- aiosqlite adapters live in `persistence/store_repo.py` and `persistence/document_store_link_repo.py`.
- The existing `VectorStore` port stays — it represents the **technology** (OpenSearch). The new `Store` entity represents the **logical store** (a named, configurable target). One `VectorStore` adapter can serve many `Store` entities by namespacing the index name.
### Deployment modes
- Single `OpenSearchStore` adapter handles all stores via per-store `index_name = f"docling-studio-{store.slug}"`. Existing default index gets migrated to `docling-studio-default` (with a redirect alias to keep backwards-compatible reads).
- HF Space deployment ships with the same one-store seed.
### Hard constraints
- Existing OpenSearch index must remain readable during migration; reads against the legacy name are aliased.
- API is additive: existing `/api/ingestion/{analysis_id}` keeps working with an implicit default-store target until the UI explicitly picks one (#222).
- Tests must not require a real OpenSearch instance for the link layer — repo tests stub the adapter.
## 5. Proposed design
### 5.1 Domain
`document-parser/domain/value_objects.py`:
```python
class StoreKind(StrEnum):
OPENSEARCH = "opensearch"
# future: PINECONE, QDRANT, …
class DocumentStoreLinkState(StrEnum):
INGESTED = "Ingested"
STALE = "Stale"
FAILED = "Failed"
```
`document-parser/domain/models.py`:
```python
@dataclass
class Store:
id: str
name: str # "rh-corpus-v3"
slug: str # "rh-corpus-v3" (URL-safe; usually = name)
kind: StoreKind
embedder: str # e.g. "bge-m3" — record the embedder used for this store
config: dict # adapter-specific config (index_name override, etc.)
created_at: datetime
is_default: bool
@dataclass
class DocumentStoreLink:
id: str
document_id: str
store_id: str
state: DocumentStoreLinkState
chunkset_hash: str | None # set by #204 on push
last_push_at: datetime | None
last_run_id: str | None # FK to a future runs table; nullable in 0.6.0
error_message: str | None # populated on FAILED state
def mark_ingested(self, *, hash_: str, at: datetime, run_id: str | None) -> None: ...
def mark_stale(self, *, at: datetime) -> None: ...
def mark_failed(self, *, at: datetime, error: str) -> None: ...
```
Aggregation rule for `Document.lifecycle_state` (defined as a pure function in `domain/lifecycle_aggregation.py`):
```
if any link state == FAILED → Document is FAILED
elif any link state == STALE → Document is STALE
elif any link state == INGESTED → Document is INGESTED
elif chunks present (state Chunked) → Document keeps its current state
else → keep current state
```
The aggregation runs as a side effect of any link write. The doc's own state column (#202) is the materialized result; it is not recomputed at read time.
### 5.2 Persistence
```sql
CREATE TABLE IF NOT EXISTS stores (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
slug TEXT NOT NULL UNIQUE,
kind TEXT NOT NULL,
embedder TEXT NOT NULL,
config TEXT NOT NULL DEFAULT '{}', -- JSON
is_default INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS document_store_links (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
store_id TEXT NOT NULL REFERENCES stores(id) ON DELETE CASCADE,
state TEXT NOT NULL,
chunkset_hash TEXT,
last_push_at TEXT,
last_run_id TEXT,
error_message TEXT,
UNIQUE (document_id, store_id)
);
CREATE INDEX IF NOT EXISTS idx_dsl_doc ON document_store_links(document_id);
CREATE INDEX IF NOT EXISTS idx_dsl_store ON document_store_links(store_id);
CREATE INDEX IF NOT EXISTS idx_dsl_state ON document_store_links(state);
```
Seed migration inserts one row in `stores`:
```sql
INSERT OR IGNORE INTO stores (id, name, slug, kind, embedder, is_default, created_at)
VALUES ('default', 'default', 'default', 'opensearch',
'<env: DEFAULT_EMBEDDER, fallback bge-m3>',
1, datetime('now'));
```
The OpenSearch index name for `slug=default` is `docling-studio-default`. To keep existing data readable, an alias migration in `OpenSearchStore.ensure_index()` adds `docling-studio-chunks` as a read-only alias to `docling-studio-default` if the legacy index exists with rows. Operators with non-trivial data trigger a one-shot reindex via #206.
### 5.3 Infra adapters
`OpenSearchStore` is parameterised by store slug:
```python
class OpenSearchStore(VectorStore):
def __init__(self, client, index_prefix: str = "docling-studio") -> None: ...
def _index_for(self, store_slug: str) -> str:
return f"{self.index_prefix}-{store_slug}"
```
Calls become `index_chunks(store_slug, doc_id, chunks, embeddings)` etc. The protocol in `domain/ports.py` is updated to take `store_slug` as an explicit argument.
### 5.4 Services
New: `StoreService` (`document-parser/services/store_service.py`) — list, get, create (admin only — locked behind a future flag; for 0.6.0 only seeded rows exist).
`IngestionService.ingest(analysis_id, store_slug = "default")`:
1. Read chunks from `analysis_jobs.chunks_json`.
2. Compute embeddings.
3. Call adapter `index_chunks(store_slug, ...)`.
4. Upsert link via `DocumentStoreLinkRepository.upsert(doc_id, store_id, state=Ingested, chunkset_hash=..., at=now)`.
5. Recompute document aggregate state via the rule in §5.1.
If any step fails: link is marked `Failed` with the error; doc state aggregates to `Failed`.
### 5.5 API
`schemas.py`:
```python
class StoreLinkResponse(BaseModel):
storeId: str
storeName: str
state: str # DocumentStoreLinkState value
chunksetHash: str | None
lastPushAt: datetime | None
lastRunId: str | None
class DocumentResponse(BaseModel):
... # from #202
stores: list[StoreLinkResponse] = Field(default_factory=list)
```
`stores` is a read-side aggregate computed by `DocumentService.find_by_id()` joining the link table.
New endpoint `GET /api/stores` returns the list of stores (id, name, slug, embedder). Used by #222's target picker.
### 5.6 Frontend — feature module
- `frontend/src/features/ingestion/``Ingestion.run()` accepts an optional `storeSlug`; defaults to "default" if not supplied.
- New `frontend/src/features/stores/` — Pinia store, API client, types.
- `frontend/src/features/document/api.ts``Document` type gains `stores: StoreLink[]`.
### 5.7 Cross-cutting
- Feature flag: none.
- i18n: `stores.state.<state>` keys in `shared/i18n.ts`.
- New env var documented in `.env.example`: `DEFAULT_EMBEDDER` (existing implicitly; now formalised in the seed).
## 6. Alternatives considered
### Alternative A — One row per (doc, store) embedded in the analysis job
- **Summary:** Add a JSON column `stores_json` on `analysis_jobs`.
- **Why not:** Conflates an *analysis attempt* with a *long-lived ingestion link*. Re-analysis would erase per-store state. The relational shape is necessary for filters, indexes, and idempotent updates.
### Alternative B — One adapter per store (parallel `VectorStore` instances)
- **Summary:** Hold a registry of `VectorStore` adapters keyed by store id.
- **Why not:** Duplicates connection pools, breaks singleton pattern in FastAPI's dependency wiring, and forces N OpenSearch clients for what is logically one client serving N indexes. Parameterising the existing single adapter is cheaper.
## 7. API & data contract
### Endpoints
| Method | Path | Request | Response | Breaking? |
|--------|------|---------|----------|-----------|
| GET | `/api/documents/{id}` | — | now includes `stores: StoreLinkResponse[]` | No (additive) |
| GET | `/api/stores` | — | `StoreResponse[]` | No (new) |
| POST | `/api/ingestion/{analysis_id}` | `{ "storeSlug": "default" }` (optional) | unchanged | No (additive) |
### Persistence schema
See §5.2.
### Env vars / config
| Name | Default | Allowed | Notes |
|------|---------|---------|-------|
| `DEFAULT_EMBEDDER` | `bge-m3` | any registered embedder slug | recorded on the seeded `default` store |
| `DOCLING_STUDIO_INDEX_PREFIX` | `docling-studio` | URL-safe slug | adapter-level prefix for OpenSearch indexes |
### Breaking changes
Additive.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Existing OpenSearch data invisible after rename | Decoupling | Medium | High | Smoke test on staging | Read alias `docling-studio-chunks → docling-studio-default` set in `ensure_index()`; explicit reindex documented in runbook |
| Multiple stores with conflicting embedders mixed at search time | Security/Performance | Low | Medium | Mismatched dim error from OpenSearch | Embedder is recorded per store; queries route by store; cross-store search is out of scope for 0.6.0 |
| Link table grows large over time on prod corpus | Performance | Medium | Medium | Slow `/api/documents/{id}` join | Index on `document_id` and `store_id`; pagination on the doc list (#211) |
| Aggregation rule drifts from per-store reality on partial writes | DDD | Medium | High | Tests fail | Aggregation runs in the same transaction as the link write |
## 9. Testing strategy
### Backend — pytest
- **Unit (domain):** `test_store_link_transitions.py`, `test_lifecycle_aggregation.py` (table-driven over per-store state combinations).
- **Persistence:** `test_store_repo.py`, `test_document_store_link_repo.py` (round-trip; UNIQUE constraint enforced; cascade delete works).
- **Services / integration:** `test_ingestion_service_with_store.py` (default store path), `test_ingestion_service_two_stores.py` (push to two stores → two links).
- **Architecture:** import-boundary test ensures `domain/` does not import `infra/opensearch_store.py`.
### Frontend — Vitest
- `features/stores/api.test.ts` — list endpoint round-trip.
- `features/document/store.test.ts``Document.stores` populated from API.
### E2E — Karate UI
Out of scope here. Lands with #211 / #218.
### Manual QA
1. Boot. `GET /api/stores` returns the seeded `default`.
2. Upload a doc, run ingestion → `GET /api/documents/{id}` returns `stores[0].state == "Ingested"`.
3. Inspect SQLite: `document_store_links` has the row.
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
None for the data; #222 will gate the multi-store UI.
### Observability
- Log `store_link_changed` with `doc_id`, `store_id`, `from`, `to`, `at`.
- Counter (Prometheus, future): `ingestion_links_total{state}`. Optional in 0.6.0.
### Rollback plan
The migration is additive. To roll back the code: revert; the unused tables stay empty for new docs but contain rows for already-pushed docs — harmless (no read path uses them after revert). For full cleanup, a follow-up migration drops the two tables.
## 11. Open questions
- Should `Store.config` be typed in the domain (via a discriminated union on `kind`)? **Decision for 0.6.0:** keep as opaque `dict`; introduce a typed wrapper when we add a second `kind`.
- Cross-store search (single query → many stores) — **explicitly punted** to a later release.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/203
- **Related issues:** #202 (lifecycle), #204 (hash), #205 (audit), #206 (migration), #211 (library), #213 (bulk push), #222 (push UI), #223 (diff-aware ingest)
- **ADRs:** ADR — "Multiple vector stores as first-class entities" (to be drafted alongside the implementation PR)
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`
- ADR guide: `docs/architecture/adr-guide.md`

View file

@ -0,0 +1,265 @@
# Design: Auto-detect Stale state via chunk content hash
- **Issue:** #204
- **Title on issue:** [FEATURE] Auto-detect Stale state via chunk content hash
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** backend: domain · persistence · services · frontend (read-only)
- **Audit dimensions likely touched:** Hexagonal Architecture · DDD · Performance · Tests · Security
- **ADR spawned?:** no
---
## 1. Problem
Without auto-detection, a user who edits a chunk has to remember which stores hold the doc and click "re-ingest" everywhere. They will forget. The next query against the stale store returns the old embedding, and the customer reports "I fixed it yesterday and it's still wrong". Manual flagging is a bug factory.
The detection contract is: **the system, not the user, knows when a stored chunkset diverges from the current draft chunkset.** Implementation = a deterministic hash recorded at push time and compared on read or on chunk write.
## 2. Goals
- [ ] Define a deterministic `chunkset_hash(chunks: list[ChunkResult])` function.
- [ ] Store the hash on each `DocumentStoreLink` at push time (#203 ships the column slot).
- [ ] On any chunk modification, recompute the current chunkset hash and compare against each link; if mismatch, mark the link `Stale` and re-aggregate the document state.
- [ ] On document read (single-doc API), compare on the fly as a safety net so older drift is caught.
- [ ] Unit tests prove: edit a chunk → link state becomes `Stale`; re-push → link state becomes `Ingested` with the new hash.
## 3. Non-goals
- Per-chunk change tracking — that's the audit trail (#205); this issue cares about the *aggregate* hash only.
- Background sweeper job that scans the whole corpus for drift — for 0.6.0, detection is event-driven (on chunk write) + on-read; a sweeper can come later.
- Diff-aware re-ingest at the chunk granularity — that's #223; this issue tells you *whether* a re-ingest is needed, not which chunks to re-embed.
- A user-facing toggle to "force stale" — out of scope.
## 4. Context & constraints
### Existing code surface
- `document-parser/domain/value_objects.py``ChunkResult` (line 93).
- `document-parser/services/analysis_service.py` — chunking pipeline.
- `document-parser/persistence/database.py``analysis_jobs.chunks_json` (the canonical chunkset today).
- `document-parser/services/ingestion_service.py` — push pipeline.
- `document-parser/api/ingestion.py` — push endpoint.
- `document-parser/persistence/document_store_link_repo.py` (created by #203).
### Hexagonal Architecture constraints
- `chunkset_hash` is a **pure domain function** — lives in `domain/hashing.py`. No I/O. No timestamps. Deterministic over the input.
- Detection (compare + transition) is orchestrated in services. Persistence reads the stored hash via the link repo.
- The hash is opaque to API/frontend in 0.6.0; surfaced only as a debug field on `StoreLinkResponse` (#203 added it).
### Hard constraints
- Hash function must be **stable across processes / machines / Python versions** — so SHA-256 (`hashlib`), not Python `hash()`. No salt.
- Must be cheap on a 500-chunk doc — a single linear pass, no JSON re-parse on hot path.
- Result is a hex string, length 64. Stored as `TEXT` in SQLite.
## 5. Proposed design
### 5.1 Domain
`document-parser/domain/hashing.py`:
```python
import hashlib
import json
from collections.abc import Iterable
from .value_objects import ChunkResult
def chunkset_hash(chunks: Iterable[ChunkResult]) -> str:
"""
Deterministic hash over a chunkset.
Hashed inputs (per chunk, in chunkset order):
- text (str)
- source_page (int | None)
- headings (list[str], preserved order)
Excluded:
- bboxes / doc_items (rendering artefacts; do not affect retrieval semantics)
- token_count (derived; unstable across tokenizers)
"""
h = hashlib.sha256()
for chunk in chunks:
payload = {
"t": chunk.text,
"p": chunk.source_page,
"h": list(chunk.headings or []),
}
h.update(b"\x1f")
h.update(json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode())
return h.hexdigest()
```
Notes:
- The `\x1f` (Unit Separator) byte between chunks defends against the "join attack" where chunk A's tail and chunk B's head produce the same hash as chunk A+B merged.
- `separators=(",", ":")` is the canonical compact JSON form.
- The exclusion list is intentional and documented inline — changing it bumps every doc to `Stale` once and is a deliberate one-time event (covered in §10 Rollback).
`detect_stale_links(current_hash, links) -> list[DocumentStoreLink]` — pure helper returning the subset of links whose `chunkset_hash != current_hash`.
### 5.2 Persistence
No new schema. The `chunkset_hash` column was added by #203 on `document_store_links`.
A new index supports the on-read safety net:
```sql
CREATE INDEX IF NOT EXISTS idx_dsl_doc_state ON document_store_links(document_id, state);
```
### 5.3 Infra adapters
None. Hashing is in-memory.
### 5.4 Services
Two call sites:
**A. On chunk write (event-driven detection)**
`AnalysisService.persist_chunks(doc_id, chunks)`:
1. Persist chunks (existing path: `analysis_jobs.chunks_json`).
2. Compute `current_hash = chunkset_hash(chunks)`.
3. Read all `DocumentStoreLink` rows for `doc_id`.
4. For each link with `chunkset_hash != current_hash` and `state in (Ingested,)`:
- `link.mark_stale(at=now)` and persist.
5. Re-aggregate the document state (#202).
This runs in the same transaction as the chunk write. The full pass is O(N_links) which is small (a doc rarely lives in more than a handful of stores).
**B. On push completion (set the new hash)**
Already covered by #203's `IngestionService.ingest`. Add the `current_hash` argument when calling `link.mark_ingested(hash_=current_hash, at=now, run_id=...)`.
**C. On read (safety net)**
`DocumentService.find_by_id(id)`:
1. Fetch document + links (existing).
2. Compute `current_hash` (cheap; cached per request).
3. Mark any `Ingested` link with mismatched hash as `Stale` and persist (best-effort, swallowed if write fails — log).
This guards against drift caused by direct DB writes / restored backups / older deploys.
### 5.5 API
No new endpoint. `StoreLinkResponse.chunksetHash` (already added by #203) is now actually populated.
### 5.6 Frontend — feature module
No changes in this issue. #224 (stale indicator) reads the existing field.
### 5.7 Cross-cutting
- Feature flag: none.
- Logs: `INFO event=stale_detected doc_id=<id> store_id=<id> previous_hash=<8> current_hash=<8>` (truncated for privacy / log volume).
- ADR: not required. The choice of SHA-256 + canonical JSON is documented inline in `hashing.py`.
## 6. Alternatives considered
### Alternative A — Per-chunk hashes only (no chunkset hash)
- **Summary:** Skip the aggregate hash; track each chunk's hash and detect "any per-chunk hash drift".
- **Why not:** Per-chunk hashing is needed for #223 (diff-aware re-embed) but not for "is this link stale at all?". A single `chunkset_hash` is one-comparison cheap; per-chunk is N-comparisons. Both will exist after #223 lands; this issue ships the cheap top-level signal.
### Alternative B — Store updated_at instead of hash
- **Summary:** Compare `chunks.updated_at > link.last_push_at`.
- **Why not:** Brittle. Re-running a pipeline on identical input bumps `updated_at` without semantic change. Hash is content-addressed and survives idempotent rewrites.
### Alternative C — MD5 / xxHash
- **Summary:** Use a faster non-cryptographic hash.
- **Why not:** SHA-256 is fast enough on the volumes in scope (`<500 chunks`, microseconds in CPython hashlib via OpenSSL bindings). The cryptographic hash also gives us collision resistance for free. xxHash would require an extra dependency.
## 7. API & data contract
### Endpoints
No additions. Existing `StoreLinkResponse.chunksetHash` becomes populated.
### Persistence schema
```sql
CREATE INDEX IF NOT EXISTS idx_dsl_doc_state ON document_store_links(document_id, state);
```
### Env vars / config
None.
### Breaking changes
None.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Hash function changes (e.g. someone adds `bboxes` to the input) silently invalidates every link | DDD | Medium | High | Every doc flips to `Stale` post-deploy | Hash function is in a single file with a docstring listing the canonical inputs; CI fixture asserts a fixed hash for a known chunkset. Bumping requires updating the fixture deliberately. |
| Read-side detection updates state under a GET (write-on-read) | Performance / Decoupling | Low | Medium | Slow / unexpected SQL in read paths | Best-effort, swallowed write; only triggered when the API actually serves the doc detail page (already a write-allowed code path). Disabled if a query param `?refresh=false` is set (future). |
| Unicode normalization issues (NFC vs NFD) produce different hashes for "the same" text | Tests | Low | Medium | Drift after a copy-paste from a Mac | Document the policy: text is stored as-is, no normalization; if drift appears, normalize at write time, not at hash time. |
| JSON ordering instability across Python versions | Tests | Low | High | Hash mismatch on different prod nodes | `separators=(",", ":")` + `ensure_ascii=False` + explicit key order in the dict literal. Reviewer checklist mentions this. |
## 9. Testing strategy
### Backend — pytest
- **Unit (domain):**
- `test_chunkset_hash_determinism.py` — same input → same output across multiple invocations.
- `test_chunkset_hash_sensitivity.py` — every input field in the canonical list changes the hash; excluded fields (`bboxes`, `token_count`) do not.
- `test_chunkset_hash_join_attack.py` — separating into different chunks must produce different hash from concatenation.
- **Locked fixture** `test_chunkset_hash_fixture.py` — a hand-built 3-chunk input whose hash is hard-coded; CI fails if anyone changes the function silently.
- **Services:**
- `test_stale_detection_on_edit.py` — edit a chunk → link state becomes `Stale`.
- `test_stale_clears_on_repush.py` — re-push → link state becomes `Ingested` with the new hash.
- `test_stale_safety_net_on_read.py` — direct DB tampering → next read flips state to `Stale`.
### Frontend — Vitest
None new in this issue.
### E2E — Karate UI
Out of scope here; lands with #224.
### Manual QA
1. Push a doc to the default store → `chunksetHash` populated, `state == "Ingested"`.
2. Edit a chunk via API → `state == "Stale"`.
3. Re-push → `state == "Ingested"`, new hash.
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
None. Detection is always on; cheap; correctness improvement.
### Observability
- Log lines as in §5.7.
- One-time bump scenario: if we ever change the canonical input list, every link will flip to `Stale` once. That is a deliberate decision; the operator must be informed via release notes, and a one-shot reindex job is recommended (out of scope here).
### Rollback plan
The migration is additive (a new index). Reverting the code leaves the existing `chunkset_hash` column populated but unused — harmless. The index can be dropped in a follow-up.
## 11. Open questions
- Should the safety-net read-side check be opt-in via a query param? **Decision:** always-on for 0.6.0; revisit if the cost shows up in profiles.
- Should headings include the *path* (parent → leaf) or just the leaf? **Decision:** the full ordered list as it sits on `ChunkResult.headings`. If the source mutates the list semantics, that is a separate domain concern.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/204
- **Related issues:** #202 (lifecycle), #203 (per-store state), #205 (audit), #206 (migration), #223 (diff-aware re-ingest), #224 (stale indicator)
- **ADRs:** none planned
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`

View file

@ -0,0 +1,372 @@
# Design: Audit trail for chunk edits
- **Issue:** #205
- **Title on issue:** [FEATURE] Audit trail for chunk edits (who, when, before/after)
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** backend: domain · persistence · services · api · frontend: features/chunking · shared
- **Audit dimensions likely touched:** Hexagonal Architecture · DDD · Tests · Security · Documentation
- **ADR spawned?:** ADR-NNN — "Chunks become a first-class persisted entity"
---
## 1. Problem
Once chunks become editable in 0.6.0 (E5), production teams will need to answer "who changed what, when, and why is the answer suddenly different?". Without an audit trail, a regression caused by a chunk edit is impossible to investigate.
Today the architecture makes this hard because **chunks are not persisted as first-class records**. They live as a JSON blob inside `analysis_jobs.chunks_json` and as derived rows in OpenSearch. There is no chunk identity that survives a re-parse, no row to attach an audit record to, and no way to retrieve "the version of these chunks at the last push".
This issue elevates chunks to a first-class persisted entity (with stable IDs and content versioning), introduces an immutable `chunk_edits` table, and exposes a snapshot API consumed by the visual diff (#221).
## 2. Goals
- [ ] Promote chunks to a first-class persisted entity with stable IDs across edits.
- [ ] Every chunk operation (create / update / delete / merge / split) writes an immutable `chunk_edits` record with: actor, timestamp, action, before-state, after-state, optional reason.
- [ ] API: `GET /api/documents/{id}/chunks/history` returns the edit timeline.
- [ ] API: `GET /api/documents/{id}/chunks?at=<push_id>` returns the chunkset snapshot at a given push.
- [ ] Backwards compatible: existing reads of `analysis_jobs.chunks_json` keep working until #206 finishes the migration.
## 3. Non-goals
- Undo / redo UI — out of scope (the audit trail is the substrate; UI comes later).
- Rollback to an arbitrary past state — not in 0.6.0; use the snapshot API for read-only inspection.
- A diff *view* — that is **#221**.
- Per-chunk content hash — that is part of **#223**, not this issue. (#204's `chunkset_hash` is at the link granularity.)
- Authentication / authorization model rework — `actor` defaults to a hard-coded `"system"` until the auth layer is ready.
## 4. Context & constraints
### Existing code surface
- `document-parser/domain/value_objects.py``ChunkResult` (transient).
- `document-parser/domain/models.py``AnalysisJob.chunks_json` (current chunkset storage).
- `document-parser/services/analysis_service.py` — chunking pipeline.
- `document-parser/persistence/database.py` — schema + migrations.
- `document-parser/api/` — no chunks endpoint today; needs creation.
- `frontend/src/features/chunking/` — Pinia store + API client.
### Hexagonal Architecture constraints
- New domain entities `Chunk` and `ChunkEdit` live in `domain/models.py`.
- `ChunkRepository` and `ChunkEditRepository` are ports (`domain/ports.py`).
- aiosqlite adapters in `persistence/chunk_repo.py` and `persistence/chunk_edit_repo.py`.
- A new `ChunkEditingService` in `services/` orchestrates the operations and writes audit records atomically with the chunk write.
- The existing `analysis_jobs.chunks_json` becomes a **legacy fallback** — read by `ChunkRepository.list_for_doc()` if no rows exist in the new `chunks` table. #206 backfills.
### Hard constraints
- Stable `chunk.id` across edits. A chunk that was split into two creates two new ids; merging two chunks produces a third new id. The "lineage" is recorded in `chunk_edits`. (No "ship the same id" hack.)
- Immutable audit table. Once written, never updated.
- No PII leak in audit records — `actor` is whatever the auth layer hands us; `before`/`after` payloads are the chunk text and metadata only.
- Atomicity: an edit + its audit row are written in the same SQL transaction.
## 5. Proposed design
### 5.1 Domain
`document-parser/domain/models.py`:
```python
@dataclass
class Chunk:
id: str # uuid4 hex
document_id: str
sequence: int # ordering within the doc; gaps allowed
text: str
headings: list[str]
source_page: int | None
bboxes: list[Bbox] # carried for rendering; not part of identity
doc_items: list[str]
token_count: int | None
created_at: datetime
updated_at: datetime
deleted_at: datetime | None # soft delete
class ChunkEditAction(StrEnum):
INSERT = "insert"
UPDATE = "update"
DELETE = "delete"
MERGE = "merge"
SPLIT = "split"
@dataclass(frozen=True)
class ChunkEdit:
id: str
document_id: str
chunk_id: str | None # null on MERGE result row (uses children_ids)
action: ChunkEditAction
actor: str # "system" until auth lands
at: datetime
before: dict | None # JSON snapshot — None for INSERT
after: dict | None # JSON snapshot — None for DELETE
parents: list[str] # for SPLIT result rows: source chunk id; for MERGE: source ids
children: list[str] # inverse links
reason: str | None
```
Domain operations on a chunkset (`domain/chunk_editing.py`):
```python
def insert(chunks: list[Chunk], at_position: int, new_chunk: Chunk) -> list[Chunk]: ...
def update(chunks: list[Chunk], chunk_id: str, *, text: str, headings: list[str]) -> list[Chunk]: ...
def delete(chunks: list[Chunk], chunk_id: str) -> list[Chunk]: ...
def merge(chunks: list[Chunk], chunk_ids: list[str]) -> tuple[list[Chunk], Chunk]:
"""Returns the updated list and the new merged chunk."""
def split(chunks: list[Chunk], chunk_id: str, at_offset: int) -> tuple[list[Chunk], Chunk, Chunk]:
"""Returns the updated list and the two new chunks."""
```
These are pure. The service wraps each call with audit-record generation.
### 5.2 Persistence
```sql
CREATE TABLE IF NOT EXISTS chunks (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
sequence INTEGER NOT NULL,
text TEXT NOT NULL,
headings TEXT NOT NULL DEFAULT '[]', -- JSON
source_page INTEGER,
bboxes TEXT NOT NULL DEFAULT '[]', -- JSON
doc_items TEXT NOT NULL DEFAULT '[]', -- JSON
token_count INTEGER,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
deleted_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_chunks_doc ON chunks(document_id);
CREATE INDEX IF NOT EXISTS idx_chunks_doc_seq ON chunks(document_id, sequence);
CREATE TABLE IF NOT EXISTS chunk_edits (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
chunk_id TEXT,
action TEXT NOT NULL,
actor TEXT NOT NULL DEFAULT 'system',
at TEXT NOT NULL,
before_json TEXT,
after_json TEXT,
parents_json TEXT NOT NULL DEFAULT '[]',
children_json TEXT NOT NULL DEFAULT '[]',
reason TEXT
);
CREATE INDEX IF NOT EXISTS idx_chunk_edits_doc_at ON chunk_edits(document_id, at);
CREATE INDEX IF NOT EXISTS idx_chunk_edits_chunk ON chunk_edits(chunk_id);
-- Snapshot table — captures the chunkset hash at every successful push.
CREATE TABLE IF NOT EXISTS chunk_pushes (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
store_id TEXT NOT NULL REFERENCES stores(id) ON DELETE CASCADE,
chunkset_hash TEXT NOT NULL,
chunk_ids TEXT NOT NULL, -- JSON array of chunk ids in order at push time
pushed_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_chunk_pushes_doc_store ON chunk_pushes(document_id, store_id);
```
Note: `chunk_pushes.chunk_ids` materialises the chunkset at push time. Combined with the immutable `chunk_edits` history, we can reconstruct any past chunkset by replay (slower) — but `chunk_ids` is the cheap path.
### 5.3 Infra adapters
None.
### 5.4 Services
`document-parser/services/chunk_editing_service.py`:
```python
class ChunkEditingService:
def __init__(self, chunks: ChunkRepository, edits: ChunkEditRepository, ...): ...
async def insert(self, doc_id, at_position, payload, *, actor, reason=None) -> Chunk: ...
async def update(self, doc_id, chunk_id, payload, *, actor, reason=None) -> Chunk: ...
async def delete(self, doc_id, chunk_id, *, actor, reason=None) -> None: ...
async def merge(self, doc_id, chunk_ids, *, actor, reason=None) -> Chunk: ...
async def split(self, doc_id, chunk_id, at_offset, *, actor, reason=None) -> tuple[Chunk, Chunk]: ...
async def history(self, doc_id) -> list[ChunkEdit]: ...
async def chunkset_at(self, doc_id, push_id) -> list[Chunk]: ...
```
Every mutating method runs in a single SQL transaction:
1. Apply the domain operation.
2. Persist chunk rows (insert / update / soft-delete).
3. Insert `chunk_edits` row with full before/after snapshot.
4. Trigger #204's stale detection on the doc.
`history()` — paginated (default 50, max 500), ordered `at DESC`.
`chunkset_at(push_id)` — read `chunk_pushes.chunk_ids`, return the matching `chunks` rows in that order.
`IngestionService.ingest()` (touched in #203) writes a row to `chunk_pushes` on success.
### 5.5 API
New router `document-parser/api/chunks.py`:
```
GET /api/documents/{id}/chunks List current chunks
GET /api/documents/{id}/chunks?at=<push_id> Snapshot at push
GET /api/documents/{id}/chunks/history Edit timeline (paginated)
POST /api/documents/{id}/chunks Insert chunk
PATCH /api/documents/{id}/chunks/{chunk_id} Update chunk
DELETE /api/documents/{id}/chunks/{chunk_id} Soft-delete chunk
POST /api/documents/{id}/chunks/merge { chunkIds: [...], reason? } → new chunk
POST /api/documents/{id}/chunks/{chunk_id}/split { atOffset, reason? } → two new chunks
```
DTOs (`schemas.py`, camelCase via alias):
```python
class ChunkResponse(BaseModel):
id: str
sequence: int
text: str
headings: list[str]
sourcePage: int | None
tokenCount: int | None
bboxes: list[BboxDto]
docItems: list[str]
updatedAt: datetime
class ChunkEditResponse(BaseModel):
id: str
chunkId: str | None
action: str
actor: str
at: datetime
before: dict | None
after: dict | None
parents: list[str]
children: list[str]
reason: str | None
```
### 5.6 Frontend — feature module
Touched: `frontend/src/features/chunking/`.
- `api.ts` — clients for the seven endpoints above.
- `store.ts` — Pinia store holds `chunks`, `history`, and a draft layer for optimistic updates with rollback on failure.
- `ui/` — primitives only in this issue (`ChunkRow.vue`, `ChunkActionsMenu.vue`); the full editor is built in #219 / #220 / #221 on top.
- `data-e2e` selectors named upfront: `chunk-row`, `chunk-actions-menu`, `chunk-action-merge`, etc.
### 5.7 Cross-cutting
- Feature flag: existing `chunking` flag (in `/api/health`) gates the editing endpoints. When `false`, the endpoints return `403`.
- i18n: `chunks.action.*`, `chunks.reason.placeholder` keys.
- Shared types: `DocumentChunk`, `ChunkEdit` re-exported from `shared/types.ts`.
## 6. Alternatives considered
### Alternative A — Keep chunks in `analysis_jobs.chunks_json`, version the JSON
- **Summary:** Add a `version` column on `analysis_jobs` and a `chunk_edits` table referencing JSON paths inside `chunks_json`.
- **Why not:** No chunk identity → `before`/`after` lookups become regex on JSON. Splits and merges have nowhere to record lineage. Reviewers and customer support cannot navigate the audit. The architectural cost is paid sooner or later — pay it now while the data is small.
### Alternative B — Use OpenSearch as the source of truth for chunks
- **Summary:** Treat the OpenSearch document as the authoritative chunk; record audit metadata there.
- **Why not:** OpenSearch is a derived index. It is the *projection* of the corpus, not its source. Replacing OpenSearch (Pinecone, Qdrant) would lose history. Source of truth belongs in SQLite.
### Alternative C — Event-sourcing only (no current-state table)
- **Summary:** Drop `chunks` table; replay `chunk_edits` to compute current state.
- **Why not:** Replay cost on every read is unacceptable for chunks-editor latency and for #221's diff. Hybrid (current state + immutable history) is the standard pragmatic shape.
## 7. API & data contract
### Endpoints
See §5.5 — eight new endpoints, all behind the existing `chunking` feature flag.
### Persistence schema
See §5.2.
### Env vars / config
| Name | Default | Allowed | Notes |
|------|---------|---------|-------|
| `CHUNK_HISTORY_PAGE_DEFAULT` | `50` | int 1500 | default page size for history |
| `CHUNK_HISTORY_PAGE_MAX` | `500` | int | hard cap |
### Breaking changes
None. Existing chunks in `chunks_json` remain accessible via the legacy fallback in `ChunkRepository.list_for_doc()` until #206 finishes the migration.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Audit table growth on heavy editing customers | Performance | Medium | Medium | Slow `chunk_edits` queries | Indexed by (doc, at). Pagination on the API. Future: archival job. |
| Race between two parallel edits on the same chunk | DDD | Low | Medium | Lost update | All mutations go through `ChunkEditingService` which serialises via the SQL transaction; client sends `If-Match` with `updatedAt` for optimistic concurrency on `PATCH` / `DELETE` |
| Audit before/after reveals sensitive content if shared | Security | Medium | Medium | Audit export shared accidentally | The audit endpoint is gated by the same auth as chunk edits; export does not include audit by default; admin-only flag in a future release |
| Schema drift: `chunks_json` and `chunks` table diverge during migration window | DDD | Medium | High | Diff between the two on the same doc | #206 is the single source of truth; new edits write to `chunks` only; reads prefer `chunks` and fall back to `chunks_json` |
## 9. Testing strategy
### Backend — pytest
- **Unit (domain):**
- `test_chunk_editing_pure.py` — insert / update / delete / merge / split on in-memory lists; properties verified (lengths, sequences, identity rules).
- **Persistence:**
- `test_chunk_repo.py` — round-trip, soft delete, ordering by sequence.
- `test_chunk_edit_repo.py` — write + read history; immutability (UPDATE on `chunk_edits` is rejected by a CHECK constraint or service-level guard).
- **Services / integration:**
- `test_chunk_editing_service_audit_atomicity.py` — failure mid-write rolls back both chunk and audit.
- `test_chunkset_at_snapshot.py``chunkset_at(push_id)` returns the right snapshot after edits.
- `test_legacy_fallback.py` — read chunks for a doc that has only `chunks_json` (pre-migration).
### Frontend — Vitest
- `features/chunking/api.test.ts` — eight endpoints round-trip.
- `features/chunking/store.test.ts` — optimistic update + rollback on API failure; history pagination.
### E2E — Karate UI
Out of scope here; the editor lands in #219 / #220 with E2E tags `@critical @ui`.
### Manual QA
1. Trigger an INSERT / UPDATE / DELETE / MERGE / SPLIT.
2. `GET /api/documents/{id}/chunks/history` shows the action with `before`/`after`.
3. After a successful push, `GET /api/documents/{id}/chunks?at=<push_id>` returns the chunkset captured at push.
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
The existing `chunking` flag (in `/api/health`) controls whether the editing endpoints accept writes. Reads (`GET`) are always allowed (consistent with the chunks tab being read-only when the flag is off).
### Observability
- Each mutation logs: `INFO event=chunk_edit doc_id=<id> chunk_id=<id> action=<action> actor=<actor>`.
- Counter (Prometheus, future): `chunk_edits_total{action}`.
### Rollback plan
The migration is additive. Reverting the code leaves the new tables but no writers. Reads from the chunks tab fall back to the legacy `chunks_json`. Editing endpoints disappear (404). If a clean rollback is needed, drop the three new tables.
## 11. Open questions
- Should `actor` carry a structured object (id, name, role) or stay as a free-form string? **Decision for 0.6.0:** free-form string (`"system"` today; whatever auth provides later). Structured upgrade is non-breaking thanks to JSON-friendly columns.
- Should we generate a *human-readable* diff in the audit row (line-level), or only store before/after JSON? **Decision:** store JSON; render the diff client-side in #221.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/205
- **Related issues:** #202 (lifecycle), #203 (per-store), #204 (auto-stale), #206 (migration), #219 (editor view), #220 (edit actions), #221 (visual diff), #222 (push)
- **ADRs:** ADR — "Chunks become a first-class persisted entity"
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`
- ADR guide: `docs/architecture/adr-guide.md`

View file

@ -0,0 +1,265 @@
# Design: Migrate existing documents to the new lifecycle state model
- **Issue:** #206
- **Title on issue:** [CHORE] Migrate existing documents to the new lifecycle state model
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** backend: persistence · services · infra (CLI) · frontend (none)
- **Audit dimensions likely touched:** CI/Build · Tests · Documentation · Security
- **ADR spawned?:** no
---
## 1. Problem
#202, #203, and #205 introduce four new tables / columns: `documents.lifecycle_state`, `stores`, `document_store_links`, `chunks`, `chunk_edits`, `chunk_pushes`. Existing tenants already have documents, analysis jobs, and chunks living in `analysis_jobs.chunks_json` plus rows in OpenSearch. After deploy, these documents must appear in `/docs` with sensible state — otherwise the page looks broken (every doc shows `Uploaded` or unknown).
This issue ships an idempotent migration that backfills:
- `documents.lifecycle_state` and `lifecycle_state_at`.
- One `stores` row (the `default` seeded by #203, no-op if already present).
- One `document_store_links` row per document already represented in OpenSearch under the legacy index.
- `chunks` rows materialized from existing `analysis_jobs.chunks_json`.
- `chunk_pushes` rows reconstructed for documents currently indexed.
It also reindexes from `docling-studio-chunks` to `docling-studio-default` if needed, so the new store-aware code can read them.
## 2. Goals
- [ ] Idempotent CLI command `python -m document_parser.tools.migrate_06` (re-runs safely; second run is a no-op).
- [ ] `--dry-run` flag prints what would change without writing.
- [ ] Inference rules below produce a sensible state for every existing document.
- [ ] Per-tenant counts logged at the end (parsed=N, chunked=N, ingested=N, failed=N).
- [ ] Documented in `docs/runbooks/release-0.6.0-migration.md`.
- [ ] Tested on a snapshot DB representing the three relevant pre-states (no analysis, completed analysis, indexed in OpenSearch).
## 3. Non-goals
- Re-running ingestion. The migration only touches metadata; it does not re-embed.
- A web UI for migration progress — operator-only, run via CLI.
- Migrating tenants on environments that have already adopted parts of the new schema (e.g. a partial pre-release deploy) — out of scope; an emergency hotfix path is documented separately.
- Moving to a new vector backend — out of scope.
## 4. Context & constraints
### Existing code surface
- `document-parser/persistence/database.py``_run_migrations()` runs schema DDL on app start.
- `document-parser/persistence/document_repo.py`, `analysis_repo.py` — read paths.
- `document-parser/infra/opensearch_store.py` — read access to existing OpenSearch index.
- New repos from #203 / #205: `store_repo.py`, `document_store_link_repo.py`, `chunk_repo.py`, `chunk_edit_repo.py`.
### Hexagonal Architecture constraints
- The migration is a **CLI script** in `document-parser/tools/migrate_06.py`. It uses repositories only — never raw SQL outside of repo modules. This keeps the upgrade path testable like any service.
- The reindex helper (`docling-studio-chunks → docling-studio-default`) lives in `infra/opensearch_store.py` as a method, called by the script.
### Hard constraints
- Idempotent: re-running on an already-migrated DB makes zero writes.
- Resumable: a crash mid-run can be restarted; checkpoints stored in a tiny `migration_progress` table (rows: `(name, completed_at)`).
- No assumption about uptime: the script must work on a quiesced DB OR a live one (transactions short, single-row scope).
- Read-only on OpenSearch by default; write happens only when the operator explicitly passes `--reindex-default-store`.
### Deployment modes
Run once per environment (`local`, `staging`, `prod`). HF Space deployments — same script, run by the deploy step before the new image starts serving traffic.
## 5. Proposed design
### 5.1 Domain
No new domain code. The migration is a coordinator over existing repos.
### 5.2 Persistence
The schema additions ship in #202 / #203 / #205 migrations (already applied at app start). This issue contributes:
```sql
CREATE TABLE IF NOT EXISTS migration_progress (
name TEXT PRIMARY KEY,
completed_at TEXT NOT NULL
);
```
This is the only new table. It records each migration step's completion to enable resumability.
### 5.3 Infra adapters
`OpenSearchStore.copy_legacy_to_default(*, dry_run: bool) -> int`
1. If `docling-studio-chunks` exists and `docling-studio-default` does not, create the new index with the same mapping.
2. Reindex via `_reindex_op` (OpenSearch reindex API) from legacy → default.
3. Add a read-only alias `docling-studio-chunks → docling-studio-default`.
4. Returns the number of documents copied.
### 5.4 Services
`document-parser/services/migration_06_service.py` — orchestrator with one method per step. Each step is bracketed by:
```python
if not progress.is_done(step_name):
do_step(...)
progress.mark_done(step_name)
```
Steps:
1. **`seed_default_store`** — `INSERT OR IGNORE INTO stores (...) VALUES ('default', ...)`. (Already in #203's migration; this step is a guard for older deploys that skipped the seed.)
2. **`backfill_document_lifecycle_state`** — for each document, infer state:
- Has at least one `analysis_jobs` row with `status=COMPLETED` and a non-null `chunks_json``Chunked` (refined below).
- Has at least one row indexed in OpenSearch (legacy or default index) → `Ingested`.
- Has only `analysis_jobs` rows with `status=COMPLETED` and no `chunks_json``Parsed`.
- Has only `analysis_jobs` rows with `status=FAILED``Failed`.
- Else → `Uploaded`.
3. **`materialize_chunks_from_chunks_json`** — for each `analysis_jobs` row with non-null `chunks_json`, parse and insert rows into `chunks` table. Stable id derivation: `f"chunk-{document_id}-{sequence:05d}-{sha256(text)[:8]}"` so re-runs are deterministic.
4. **`backfill_links_from_opensearch`** — for each document, query OpenSearch (default index, then legacy as fallback) for the count of indexed chunks. If non-zero:
- Insert `document_store_links` row (state = `Ingested`).
- Compute the chunkset hash via #204's function over the materialized `chunks`.
- Set `link.chunkset_hash`.
- Insert a `chunk_pushes` row with the materialized chunk ids (best-effort: ordered by sequence).
5. **`reaggregate_document_lifecycle`** — recompute the doc state by combining the link states (#203's aggregation rule) and update `documents.lifecycle_state` accordingly. This may upgrade or downgrade the value set in step 2 (e.g. `Chunked → Ingested`).
6. **`copy_legacy_index`** *(optional, behind `--reindex-default-store`)* — call `OpenSearchStore.copy_legacy_to_default()`.
Each step is wrapped in its own transaction; partial failure leaves earlier steps committed.
### 5.5 API
None. CLI only.
### 5.6 Frontend
None.
### 5.7 Cross-cutting
- Logging: structured at `INFO` per step, at `ERROR` per failed item with `doc_id`. End summary printed to stdout in a fixed table.
- Configuration via CLI flags only (no env vars introduced for this script):
```
python -m document_parser.tools.migrate_06 \
[--dry-run] \
[--reindex-default-store] \
[--limit N] \
[--only-step <step_name>]
```
- Documentation: `docs/runbooks/release-0.6.0-migration.md` describes the operator flow (backup → run dry-run → run real → validate via `/api/documents`).
## 6. Alternatives considered
### Alternative A — Schema migration applies everything at app boot
- **Summary:** Embed inference logic in `_run_migrations()` so the app starts and self-heals.
- **Why not:** Migration is observable and debuggable as a CLI; baking it into boot time risks slow startup and silent failures. Operators want to run it during a maintenance window with a dry-run first.
### Alternative B — Migrate on demand (lazy)
- **Summary:** Add a "needs migration" check at runtime, materialize chunks for a doc only on first read.
- **Why not:** Surfaces the partial state in the API (`stores: []` for unmigrated docs even if they are indexed). The library page (#211) becomes a half-true representation of reality. Eager migration is simpler.
### Alternative C — Run only the schema; let users re-ingest manually
- **Summary:** Skip backfill; users re-trigger ingestion from the UI as needed.
- **Why not:** A tenant with 10k docs cannot click 10k re-ingest buttons. The killer flow promises "your existing corpus already has state".
## 7. API & data contract
No API changes.
### Persistence schema
See §5.2 (one new table: `migration_progress`).
### CLI
```
python -m document_parser.tools.migrate_06 [flags]
```
All flags are documented in `--help`.
### Breaking changes
None.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Migration crashes mid-run | CI/Build | Medium | Medium | Error logs | Resumable via `migration_progress`; re-run picks up where it left off |
| Wrong inference for a doc edge case (e.g. multiple stores already) | DDD | Low | Medium | Operator validation (sample 10 docs by hand) | Dry-run lists changes; operator can `--only-step` to redo a single step |
| OpenSearch reindex copies bad data | Decoupling | Low | High | Diff in document counts | Reindex is opt-in (`--reindex-default-store`); operator runs it deliberately; alias keeps legacy reads working |
| Hash mismatch after migration (chunks materialized differ from what was indexed) | DDD | Medium | Medium | Newly-migrated link shows `Stale` immediately | Acceptable behaviour: it correctly tells the user "your indexed chunks may not match the current source"; operator can choose to re-ingest or leave as-is |
| Missing analysis-job rows for a doc | Tests | Low | Low | Doc shows `Uploaded` despite being indexed | Inference falls back to OpenSearch presence; if both empty, `Uploaded` is correct |
## 9. Testing strategy
### Backend — pytest
- **Unit (services):** `test_migration_inference.py` — table-driven: every `(analysis_state, has_chunks_json, indexed)` tuple → expected `lifecycle_state`.
- **Integration:** `test_migration_idempotency.py` — run twice → second run zero writes; mid-run abort + restart → final state matches one-shot run.
- **Persistence:** `test_chunks_materialization.py``chunks_json` → rows; ids deterministic (re-running produces same ids).
### Snapshot fixture
`document-parser/tests/fixtures/db_pre_06.sqlite` — handcrafted SQLite DB with three documents (uploaded only, completed analysis, indexed in OpenSearch via fake adapter). Migration runs against it and the resulting state is asserted.
### Manual QA
1. Snapshot prod DB.
2. Run `--dry-run` on the snapshot → review the printed plan.
3. Run for real on the snapshot → validate via `/api/documents` that every doc has a sensible `lifecycleState` and (where applicable) `stores`.
4. Run the same against prod during the maintenance window.
### Performance
The script is O(N_docs + N_chunks). For 10k docs / 500k chunks: target < 5 minutes on a single-node SQLite. No parallelism needed in 0.6.0.
## 10. Rollout & observability
### Release branch
`release/0.6.0`. The migration ships in the same release as #202 / #203 / #204 / #205 — operators run it after deploying the new code and before sending traffic.
### Feature flag
None.
### Observability
- Stdout summary table at the end:
```
step wrote skipped
seed_default_store 0 1
backfill_document_lifecycle_state 87 0
materialize_chunks_from_chunks_json 14502 0
backfill_links_from_opensearch 73 14
reaggregate_document_lifecycle 73 0
copy_legacy_index — —
total 14735 15
```
- Logs: per-step start / finish; per-error row.
### Rollback plan
- `migration_progress` rows can be deleted to force re-run a specific step.
- The new tables can be truncated or dropped (data is recoverable from `analysis_jobs.chunks_json` and OpenSearch).
- Reverting application code: the new tables stay populated but unused; safe.
## 11. Open questions
- Should the script open a *long* SQLite transaction or many small ones? **Decision:** many small (per-doc), to keep the DB writeable for the live app if the operator chooses to run the script while traffic is on.
- Hash mismatch on freshly-migrated docs (chunks materialized may differ from what was indexed) — should we *force-mark* them `Stale`, or trust the hash compare to do it implicitly? **Decision:** trust the compare. The result is identical and the path is uniform.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/206
- **Related issues:** #202 (lifecycle), #203 (per-store), #204 (hash), #205 (audit + chunks table)
- **ADRs:** none planned
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`
- Operations playbooks: `docs/operations/`

View file

@ -0,0 +1,216 @@
# Design: Document-centric routing
- **Issue:** #207
- **Title on issue:** [FEATURE] Document-centric routing (/docs, /docs/:id?mode=, /index/:store, /runs)
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** frontend: app/router · pages · shared
- **Audit dimensions likely touched:** Clean Code · Tests · Documentation
- **ADR spawned?:** no
---
## 1. Problem
The current Vue Router (`frontend/src/app/router/index.ts`) is **analysis-centric**: routes are `/studio`, `/documents`, `/search`, `/reasoning`, `/history`. The selected document is held in a Pinia store, not in the URL — so two engineers cannot share a link to "the chunks editor for doc X". This breaks the killer flow ("paste this URL → fix the chunks → re-ingest") that 0.6.0 promises.
The 0.6.0 sitemap puts the document at the centre of every URL: `/docs`, `/docs/:id?mode=ask|inspect|chunks`, plus `/index/:store` for stores and `/runs` for the run history. Mode is a query param so a doc URL stays stable across mode switches.
This issue ships the routing skeleton. The actual page contents come in E3 (`/docs` library — #211), E4 (workspace shell — #216), E5 (chunks editor — #218 onward).
## 2. Goals
- [ ] Add `/docs`, `/docs/new`, `/docs/:id`, `/index`, `/index/:store`, `/index/:store/query`, `/runs`, `/runs/:id` routes.
- [ ] On `/docs/:id`, `?mode=ask|inspect|chunks` is parsed; default = `ask`.
- [ ] Each new route renders a placeholder page (clear "Coming in 0.6.0" message) until E3/E4/E5 implement them.
- [ ] Legacy routes (`/studio`, `/documents`, `/history`, `/search`, `/reasoning`) keep working — no breaking redirect in this issue.
- [ ] Smoke test: each new route renders without error.
## 3. Non-goals
- Building the actual pages — that is E3 / E4 / E5.
- Migrating users from old routes — kept functional in parallel; deprecation comes when the new pages are ready.
- Server-side route enforcement — backend exposes its API on `/api/*`; the routing here is client-side only.
- A site map / generated nav — the sidebar nav rework is **#209**.
- Feature-flag-aware redirection (e.g. mode disabled → redirect to default) — that is **#210**.
## 4. Context & constraints
### Existing code surface
- `frontend/src/app/router/index.ts` — current Vue Router setup (history mode, lazy-loaded pages).
- `frontend/src/pages/` — existing pages (`HomePage.vue`, `StudioPage.vue`, `DocumentsPage.vue`, `HistoryPage.vue`, `SearchPage.vue`, `ReasoningPage.vue`, `SettingsPage.vue`).
- `frontend/src/app/App.vue` — shell with topbar + sidebar + `<RouterView />`.
- `frontend/src/features/feature-flags/` — flag store and `useFeatureFlag` composable.
### Hard constraints
- TypeScript strict — every new route needs a typed `name`.
- No regression on existing routes — old URLs keep returning their current pages until #211 / #216 explicitly replace them.
- Lazy loading is preserved — every new page goes through `() => import(...)`.
### Deployment modes
Same routing for both `latest-local` and `latest-remote`. No HF Space-specific concern.
## 5. Proposed design
### 5.1 Router additions
Append to `frontend/src/app/router/index.ts`:
```ts
{ path: '/docs', name: 'docs-library',
component: () => import('@/pages/DocsLibraryPage.vue') },
{ path: '/docs/new', name: 'docs-new',
component: () => import('@/pages/DocsNewPage.vue') },
{ path: '/docs/:id', name: 'doc-workspace',
component: () => import('@/pages/DocWorkspacePage.vue'),
props: route => ({ id: route.params.id, mode: parseMode(route.query.mode) }) },
{ path: '/index', name: 'stores-list',
component: () => import('@/pages/StoresListPage.vue') },
{ path: '/index/:store', name: 'store-detail',
component: () => import('@/pages/StoreDetailPage.vue'),
props: true },
{ path: '/index/:store/query', name: 'store-query',
component: () => import('@/pages/StoreQueryPage.vue'),
props: true },
{ path: '/runs', name: 'runs',
component: () => import('@/pages/RunsPage.vue') },
{ path: '/runs/:id', name: 'run-detail',
component: () => import('@/pages/RunDetailPage.vue'),
props: true },
```
### 5.2 Mode parser
A pure helper in `frontend/src/shared/routing/modes.ts`:
```ts
export type DocMode = 'ask' | 'inspect' | 'chunks'
const DEFAULT_MODE: DocMode = 'ask'
export function parseMode(raw: unknown): DocMode {
return raw === 'inspect' || raw === 'chunks' ? raw : DEFAULT_MODE
}
```
This is intentionally tiny and testable. #210 will extend it with feature-flag-aware redirection.
### 5.3 Placeholder pages
Each new page is ~30 lines of Vue: a centered card with the page title, a "Coming in 0.6.0" tagline, and a link back to home. They use the existing `useI18n()` strings under a new `comingSoon.*` namespace.
### 5.4 Router types
`frontend/src/shared/routing/names.ts` exports a typed union of route names so callers do `router.push({ name: ROUTES.DOC_WORKSPACE, params: { id } })` instead of stringly-typed names.
```ts
export const ROUTES = {
HOME: 'home',
DOCS_LIBRARY: 'docs-library',
DOCS_NEW: 'docs-new',
DOC_WORKSPACE: 'doc-workspace',
STORES_LIST: 'stores-list',
STORE_DETAIL: 'store-detail',
STORE_QUERY: 'store-query',
RUNS: 'runs',
RUN_DETAIL: 'run-detail',
// ...legacy names kept as-is
} as const
export type RouteName = (typeof ROUTES)[keyof typeof ROUTES]
```
### 5.5 i18n
New keys under `comingSoon.*` in `frontend/src/shared/i18n.ts` (fr + en):
- `comingSoon.title`
- `comingSoon.subtitle.docsLibrary`
- `comingSoon.subtitle.docsNew`
- `comingSoon.subtitle.docWorkspace`
- `comingSoon.subtitle.stores`
- `comingSoon.subtitle.storeDetail`
- `comingSoon.subtitle.storeQuery`
- `comingSoon.subtitle.runs`
- `comingSoon.subtitle.runDetail`
- `comingSoon.backHome`
## 6. Alternatives considered
### Alternative A — Replace existing routes immediately
- **Summary:** Make `/studio` and `/documents` redirect to the new routes in this issue.
- **Why not:** The new pages do not exist yet. Redirecting now means the user lands on a "Coming soon" page where they used to have a working app.
### Alternative B — Hash-mode routing
- **Summary:** Switch to `createWebHashHistory` for the new doc-centric routes.
- **Why not:** History mode is the existing convention and SPA deep-linking still works behind Nginx (already configured). No reason to mix modes.
## 7. API & data contract
No backend changes. The routes are entirely client-side. No env vars.
### Breaking changes
None. Additive.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Placeholder pages confuse users who land on them via shared links | Documentation | Medium | Low | Support tickets | Clear "Coming soon" copy + back-home link |
| Route name collisions with legacy ones | Clean Code | Low | Low | TS error / runtime warning | Use a `ROUTES` constant; legacy names kept |
| Broken nav from sidebar to legacy routes after rename | Decoupling | Low | Medium | Smoke test catches | The sidebar update is explicitly **#209**, not this issue |
## 9. Testing strategy
### Frontend — Vitest
- `app/router/router.test.ts` — every new route resolves to its component (smoke).
- `shared/routing/modes.test.ts``parseMode` returns `ask` for `undefined` / `null` / unknown values; respects `inspect` / `chunks`.
### E2E — Karate UI
Out of scope for this issue (placeholder pages only). E2E coverage lands with #211 (library) and #216 (workspace).
### Manual QA
1. Visit each new URL in the browser → "Coming soon" shell renders without 404.
2. Visit `/docs/abc?mode=chunks` → page receives `mode === 'chunks'`.
3. Visit `/docs/abc?mode=garbage` → page receives `mode === 'ask'` (default).
4. Old routes (`/studio`, `/documents`) still load their existing pages.
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
None. The placeholder pages are visible to anyone; they explain themselves.
### Observability
No new logs. Existing router-error handling unchanged.
### Rollback plan
Revert the router and pages — old setup is untouched.
## 11. Open questions
- Should `/docs/:id` 404 if the doc id is unknown, or render the workspace shell with an error state? **Decision for 0.6.0:** the workspace handles the "doc not found" case in #216; this issue ships the placeholder which always renders.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/207
- **Related issues:** #208 (breadcrumb), #209 (nav), #210 (FF mode gating), #211 (library page), #216 (workspace), #218+ (modes)
- **ADRs:** none planned
- **Project docs:**
- Architecture: `docs/architecture.md`
- Frontend conventions: `frontend/CLAUDE.md`

View file

@ -0,0 +1,180 @@
# Design: Doc workspace breadcrumb
- **Issue:** #208
- **Title on issue:** [ENHANCEMENT] Refactor breadcrumb to Studio <doc> <mode>
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** frontend: shared/ui · app
- **Audit dimensions likely touched:** Clean Code · Tests · Documentation
- **ADR spawned?:** no
---
## 1. Problem
There is no breadcrumb component today. The user lands on `/docs/:id?mode=chunks` and has no orientation: which doc, which mode, how to step back. With the new doc-centric URLs introduced in #207, the user navigates between modes on the same doc — a breadcrumb anchors them in the IA.
The shape `Studio <doc title> <mode>` is the agreed pattern: each segment is a link except the last, the doc title is truncated with ellipsis (full title on hover), and the mode segment updates as the user switches tabs without a page reload.
## 2. Goals
- [ ] New `<AppBreadcrumb>` component renders three segments on doc workspace pages: `Studio <doc title> <mode>`.
- [ ] Each non-last segment is clickable: `Studio``/`, `<doc title>``/docs/:id` (default mode).
- [ ] Doc title truncates beyond ~40 chars with ellipsis; full title shown via `title` attribute on hover.
- [ ] Mode segment reflects current `?mode=` and updates without remount.
- [ ] On non-doc routes (`/`, `/runs`, `/index`), the breadcrumb is empty / hidden — no fake breadcrumbs invented.
- [ ] Component is data-driven: takes a `Crumb[]` prop so future routes can supply their own.
## 3. Non-goals
- Auto-deriving the breadcrumb from the route — for 0.6.0 the doc workspace page passes its crumbs explicitly. Auto-derivation is a follow-up if more pages need it.
- Mobile-specific breadcrumb collapsing — a single-line ellipsis is enough for the layout.
- Breadcrumbs on store / run pages — those land in 0.7.0 with their own page work.
## 4. Context & constraints
### Existing code surface
- `frontend/src/app/App.vue` — the shell. The breadcrumb slot will live in the topbar (between the logo and the right-side actions).
- `frontend/src/pages/DocWorkspacePage.vue` (created by #207, placeholder). Will be the first consumer.
- `frontend/src/shared/ui/` — home for shared UI primitives. The new component lives here.
- `frontend/src/shared/routing/names.ts` (created by #207) — typed route names.
- `frontend/src/shared/i18n.ts` — strings.
### Hard constraints
- TypeScript strict — `Crumb` is a discriminated union (link vs leaf).
- Accessibility — `<nav aria-label="breadcrumb">` + `<ol>` + `aria-current="page"` on the leaf.
- No CSS framework lock-in — uses existing token classes from `App.vue` styles, no new lib.
## 5. Proposed design
### 5.1 Component contract
`frontend/src/shared/ui/AppBreadcrumb.vue`:
```ts
type LinkCrumb = { kind: 'link'; label: string; to: RouteLocationRaw }
type LeafCrumb = { kind: 'leaf'; label: string }
type Crumb = LinkCrumb | LeafCrumb
defineProps<{ crumbs: Crumb[] }>()
```
The component renders nothing when `crumbs.length === 0`.
### 5.2 Doc workspace integration
The doc workspace page (placeholder from #207) builds its crumbs:
```ts
const crumbs = computed<Crumb[]>(() => [
{ kind: 'link', label: t('breadcrumb.studio'), to: { name: ROUTES.HOME } },
{
kind: 'link',
label: truncate(doc.value?.filename ?? '...', 40),
to: { name: ROUTES.DOC_WORKSPACE, params: { id: docId.value } },
},
{ kind: 'leaf', label: t(`breadcrumb.mode.${mode.value}`) },
])
```
A small helper `truncate(text, max)` in `shared/ui/text.ts` adds the ellipsis.
### 5.3 Slot wiring in the shell
`App.vue` reserves a slot just under the topbar:
```vue
<slot name="breadcrumb">
<AppBreadcrumb :crumbs="[]" />
</slot>
```
For pages that do not opt in (which is most of them in 0.6.0), nothing renders.
The doc workspace page provides its slot via a `<teleport to="#breadcrumb-slot">` or — simpler — the `App.vue` consults a tiny Pinia store `useBreadcrumbStore()` that pages set via `setCrumbs([...])` on mount and clear on unmount. Goes with the "data-driven" goal.
We pick the **store approach**: zero teleport, plays well with route-level transitions, and the consumer lives in the page (no shell coupling).
### 5.4 i18n
New keys under `breadcrumb.*`:
- `breadcrumb.studio`
- `breadcrumb.mode.ask`
- `breadcrumb.mode.inspect`
- `breadcrumb.mode.chunks`
## 6. Alternatives considered
### Alternative A — Auto-derive crumbs from the route
- **Summary:** A central function that maps each `RouteLocationNormalized` to a `Crumb[]`.
- **Why not:** Doc workspace needs the doc title which is async — the function would need to fetch it. Page-driven is cleaner for now; auto-derivation can layer on top later for static pages.
### Alternative B — Use the page meta object (`route.meta.breadcrumb`)
- **Summary:** Each route declares a static `meta.breadcrumb` array.
- **Why not:** Same async-doc-title problem. Plus it scatters crumb config across the router.
## 7. API & data contract
No backend change. No env vars.
### Breaking changes
None.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Pages forget to clear crumbs on unmount → stale breadcrumb on next route | Clean Code | Medium | Low | Visual regression | The `useBreadcrumbStore` exposes `useCrumbs(crumbs)` composable that auto-clears on unmount via `onBeforeUnmount` |
| Long doc titles break the topbar layout | Clean Code | Low | Low | Visual regression | Truncate at 40 chars + CSS `text-overflow: ellipsis` as a belt-and-braces |
| Accessibility regression | Tests / Documentation | Low | Medium | Audit | `<nav aria-label="breadcrumb">` + `aria-current="page"` on leaf; tested |
## 9. Testing strategy
### Frontend — Vitest
- `shared/ui/AppBreadcrumb.test.ts` — renders nothing when empty; renders 3 crumbs with correct roles; `aria-current="page"` on leaf.
- `shared/ui/text.test.ts``truncate` boundary cases.
- `shared/state/breadcrumbStore.test.ts``setCrumbs` / `clear`; `useCrumbs` composable auto-clears on unmount.
### Manual QA
1. Visit `/docs/abc?mode=ask``Studio abc.pdf Ask`.
2. Switch to chunks mode (when E4 lands) → leaf updates without page flicker.
3. Visit `/runs` → no breadcrumb visible.
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
None.
### Observability
None.
### Rollback plan
Revert; pages stop providing crumbs and the slot stays empty.
## 11. Open questions
- Should we render the breadcrumb on `/docs/new` as `Studio Import`? **Decision:** yes, the page provides 2 crumbs (`Studio Import`). Implemented as a tiny tweak in #214.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/208
- **Related issues:** #207 (routing), #211 (library), #216 (workspace), #218+ (modes)
- **ADRs:** none planned
- **Project docs:** Architecture (`docs/architecture.md`), Frontend conventions (`frontend/CLAUDE.md`)

View file

@ -0,0 +1,175 @@
# Design: Navigation rework — Home / Docs / Stores / Runs / Settings
- **Issue:** #209
- **Title on issue:** [ENHANCEMENT] Rework top navigation (Home / Docs / Stores / Runs / Settings)
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** frontend: shared/ui · app
- **Audit dimensions likely touched:** Clean Code · Tests · Documentation
- **ADR spawned?:** no
---
## 1. Problem
The existing nav is a **left sidebar** (`AppSidebar.vue`) with seven entries: `Home`, `Studio`, `Documents`, `Search`, `Reasoning`, `History`, `Settings`. It mirrors the current execution-centric IA: `Studio` (an analysis), `Documents` (a list), `History` (past runs).
The doc-centric pivot wants a tighter five-entry nav reflecting the new IA: **Home, Docs, Stores, Runs, Settings**. `Docs` is the new primary action (replaces `Documents` + `Studio`). `Stores` is new. `Runs` replaces `History` (same data, new framing). `Search` and `Reasoning` collapse into the doc workspace modes (#216 + #224).
The original PM sitemap called this a "top nav" — the project's actual nav is a **sidebar**. We keep the sidebar (existing convention, established interaction) and rework its entries.
## 2. Goals
- [ ] `AppSidebar.vue` shows five entries in this order: Home, Docs (★ primary), Stores, Runs, Settings.
- [ ] `Docs` visually emphasised — bold label or accent colour token.
- [ ] Active state correctly highlights `Docs` on `/docs`, `/docs/new`, `/docs/:id`.
- [ ] Active state for `Stores` on `/index`, `/index/:store`, `/index/:store/query`.
- [ ] Active state for `Runs` on `/runs`, `/runs/:id`.
- [ ] Legacy entries (`Studio`, `Documents`, `Search`, `Reasoning`, `History`) removed from the sidebar.
- [ ] Legacy URLs still functional (we did not redirect in #207).
## 3. Non-goals
- Removing the legacy *pages* — they keep working; this issue removes them only from the sidebar.
- Mobile hamburger redesign — the existing burger toggle stays.
- Sidebar collapse animation — unchanged.
- Top breadcrumb — that is **#208**.
- A "what's new" tooltip explaining the change — out of scope; release notes cover it.
## 4. Context & constraints
### Existing code surface
- `frontend/src/shared/ui/AppSidebar.vue` — the file to edit. Currently 145 lines.
- `frontend/src/app/App.vue` — embeds `<AppSidebar>`.
- `frontend/src/shared/i18n.ts` — flat keys under `nav.*`.
- `frontend/src/features/feature-flags/store.ts` — flags currently gating `Search` and `Reasoning`.
### Hard constraints
- No new dependencies.
- TypeScript strict — every nav item is typed.
- Existing burger / collapse interaction stays untouched.
## 5. Proposed design
### 5.1 Nav model
A typed array driving the render:
```ts
type NavItem = {
key: string // for v-for and active match
to: RouteLocationRaw // typed via ROUTES
labelKey: string // i18n key
iconKey: string // icon name token
primary?: boolean // visual emphasis (Docs)
matchPrefixes: string[] // prefixes for active-state match
}
const items: NavItem[] = [
{ key: 'home', to: { name: ROUTES.HOME }, labelKey: 'nav.home', iconKey: 'home', matchPrefixes: ['/'] },
{ key: 'docs', to: { name: ROUTES.DOCS_LIBRARY }, labelKey: 'nav.docs', iconKey: 'docs', primary: true, matchPrefixes: ['/docs'] },
{ key: 'stores', to: { name: ROUTES.STORES_LIST }, labelKey: 'nav.stores', iconKey: 'stores', matchPrefixes: ['/index'] },
{ key: 'runs', to: { name: ROUTES.RUNS }, labelKey: 'nav.runs', iconKey: 'runs', matchPrefixes: ['/runs'] },
{ key: 'settings', to: { name: ROUTES.SETTINGS }, labelKey: 'nav.settings', iconKey: 'settings', matchPrefixes: ['/settings'] },
]
```
### 5.2 Active match
`isActive(item, route)`:
- For `/`, the home entry is active only on exact match.
- For all other items, active when the route path **starts with** any of `item.matchPrefixes`.
- Implemented in a tiny pure helper, tested.
### 5.3 Primary emphasis
The `primary: true` item gets an extra CSS class `nav-item--primary` adding a subtle accent (left bar in the sidebar's accent colour, bolder label). No new colour tokens.
### 5.4 i18n
New keys (fr + en):
- `nav.docs`
- `nav.stores`
- `nav.runs`
Removed (or kept around for the legacy pages that still exist): `nav.studio`, `nav.documents`, `nav.history`, `nav.search`, `nav.reasoning`. We **keep** them in `i18n.ts` since the legacy pages still render headings using them. They are no longer surfaced in the sidebar.
### 5.5 Footer
The sidebar footer (OpenSearch dot, GitHub stars, version) stays. The OpenSearch dot moves to be visible regardless of `ingestion` flag — it now reflects the default store's reachability (#203's seed). For 0.6.0 we keep the existing wiring (gated by `ingestion` flag) and revisit when stores get their own page (#231 in 0.7.0).
## 6. Alternatives considered
### Alternative A — Keep all old entries, add the new ones
- **Summary:** Show 10 entries; let the user discover.
- **Why not:** Defeats the point. The pivot is about reducing cognitive load, not adding entries.
### Alternative B — Move the nav to the top bar
- **Summary:** Implement the design doc's literal "top nav".
- **Why not:** The shell is sidebar-driven; switching layout is a much bigger lift and out of scope for E2.
## 7. API & data contract
No backend change. No env vars.
### Breaking changes
None at the API level. UX-level: legacy entries disappear from the sidebar. Their URLs remain valid.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Users of legacy pages think the app removed features | Documentation | Medium | Medium | Support tickets | Release notes; legacy pages still work via direct URL; eventually #211/#216 replace them entirely |
| Active state misfires because two entries match the same prefix | Clean Code | Low | Low | Visual regression | `matchPrefixes` is explicit, longest-prefix wins via simple precedence in the helper |
| Missing icons for Docs / Stores / Runs in the existing icon set | Clean Code | Medium | Low | Visible during review | Audit `iconKey` strings against the icon component before merge; fall back to a sane default if missing |
## 9. Testing strategy
### Frontend — Vitest
- `shared/ui/AppSidebar.test.ts` — renders 5 entries in order; `Docs` carries the primary class; active state correct on each `matchPrefix`.
- `shared/ui/navActive.test.ts` — pure helper unit tests over `isActive(item, path)`.
### Manual QA
1. Visit `/`, `/docs`, `/index/foo`, `/runs`, `/settings` → exactly one entry highlighted.
2. Visit `/docs/abc?mode=chunks``Docs` highlighted.
3. Visit a legacy route `/studio` → no nav entry highlighted (page still loads).
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
None.
### Observability
None.
### Rollback plan
Revert. The legacy nav returns; legacy pages were never broken.
## 11. Open questions
- Do we add a `Help` entry now or wait? **Decision:** wait. The five-entry nav is part of the value proposition.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/209
- **Related issues:** #207 (routing), #208 (breadcrumb), #210 (FF gating), #211 (library), #216 (workspace)
- **ADRs:** none planned
- **Project docs:** Architecture (`docs/architecture.md`), Frontend conventions (`frontend/CLAUDE.md`)

View file

@ -0,0 +1,235 @@
# Design: Feature flags hide entire mode tabs (and redirect deep links)
- **Issue:** #210
- **Title on issue:** [ENHANCEMENT] Feature flags hide entire mode tabs instead of routing to 404
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-04-29
- **Status:** Accepted
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** frontend: features/feature-flags · app/router · pages · shared · backend: api/health
- **Audit dimensions likely touched:** Clean Code · Tests · Security · Documentation
- **ADR spawned?:** no
---
## 1. Problem
Studio gates `Inspect` / `Chunks` / `Ask` per tenant via feature flags. Without explicit handling, a disabled mode is still reachable by URL — a deep link like `/docs/abc?mode=chunks` lands on a half-broken page or 404 when chunks are disabled. The user experience is "the link my colleague sent me is broken".
The fix has three pieces:
1. **Routing-level redirect**: `?mode=<disabled>` rewrites to the first enabled mode, preserving the doc id.
2. **Tab-level visibility**: when E4 builds the workspace tabs (#216), disabled modes are hidden from the tab strip.
3. **Server-side guard**: even if a client sends a request for a disabled mode, the API rejects writes (read-only is OK so the user can still view).
This issue ships **piece 1** (routing redirect) plus the **flag declarations** on `/api/health` and **frontend exposure** that pieces 2 + 3 will consume. Tab visibility (piece 2) is wired in #216. Server-side write guard (piece 3) is wired by the chunks-edit endpoints from #205's follow-up.
## 2. Goals
- [ ] `/api/health` exposes three new booleans: `inspectModeEnabled`, `chunksModeEnabled`, `askModeEnabled`.
- [ ] `useFeatureFlag('inspectMode' | 'chunksMode' | 'askMode')` returns the live value.
- [ ] Routing redirects `/docs/:id?mode=<disabled>` to the first enabled mode, in this priority: `ask``chunks``inspect`. If none are enabled, redirect to `/docs` with a flash message.
- [ ] No 404 on a disabled mode — only redirects.
- [ ] Server-side: nothing yet (the actual write endpoints land elsewhere).
- [ ] E2E (Karate UI): toggle a flag (via env var) → deep link redirects correctly.
## 3. Non-goals
- Tab strip visibility — that is **#216** (workspace).
- Server-side write rejection — that lands on the chunks-edit endpoints (#205 follow-up).
- Per-user flags (vs per-tenant) — out of scope; flags are global to the deployment.
- A flag UI to toggle modes at runtime — operator-only via env vars.
## 4. Context & constraints
### Existing code surface
- `document-parser/api/schemas.py``HealthResponse` already exposes `chunking`, `disclaimer`, `ingestion_available`, `reasoning_available`.
- `document-parser/api/health.py` — endpoint that builds `HealthResponse`.
- `document-parser/infra/settings.py` — env-var driven settings.
- `frontend/src/features/feature-flags/store.ts` — flag map + `isEnabled(name)`.
- `frontend/src/features/feature-flags/useFeatureFlag.ts` — composable.
- `frontend/src/app/router/index.ts` — Vue Router (extended in #207).
### Hexagonal Architecture constraints (backend)
The flag values are read from env vars in `infra/settings.py` (existing pattern). The API layer translates them into the `HealthResponse` DTO. No domain change.
### Hard constraints
- `/api/health` stays additive — three new fields, no breaking renames.
- Frontend behaviour falls back gracefully when fields are absent (older backend version).
- Default for all three flags = `true` (enabled). That preserves current behaviour for existing deployments.
## 5. Proposed design
### 5.1 Backend
`document-parser/infra/settings.py` adds three booleans:
```python
INSPECT_MODE_ENABLED = os.getenv("INSPECT_MODE_ENABLED", "true").lower() == "true"
CHUNKS_MODE_ENABLED = os.getenv("CHUNKS_MODE_ENABLED", "true").lower() == "true"
ASK_MODE_ENABLED = os.getenv("ASK_MODE_ENABLED", "true").lower() == "true"
```
`document-parser/api/schemas.py``HealthResponse` gains:
```python
inspect_mode_enabled: bool = True
chunks_mode_enabled: bool = True
ask_mode_enabled: bool = True
```
`document-parser/api/health.py` populates them from settings.
### 5.2 Frontend — flag store
`frontend/src/features/feature-flags/store.ts` adds three keys to the flag map:
```ts
inspectMode: this.health.inspectModeEnabled ?? true,
chunksMode: this.health.chunksModeEnabled ?? true,
askMode: this.health.askModeEnabled ?? true,
```
`useFeatureFlag('inspectMode' | 'chunksMode' | 'askMode')` returns a `ComputedRef<boolean>`. No new composable; the existing one works on these new keys.
### 5.3 Frontend — routing redirect
A pure helper `frontend/src/shared/routing/resolveMode.ts`:
```ts
export const MODE_PRIORITY: DocMode[] = ['ask', 'chunks', 'inspect']
export function resolveMode(
requested: DocMode | undefined,
enabled: Record<DocMode, boolean>,
): DocMode | null {
if (requested && enabled[requested]) return requested
return MODE_PRIORITY.find(m => enabled[m]) ?? null
}
```
Wired in the router's `beforeEach` guard for `doc-workspace`:
```ts
router.beforeEach((to) => {
if (to.name !== ROUTES.DOC_WORKSPACE) return true
const requested = parseMode(to.query.mode)
const enabled = featureStore.modeFlags() // returns Record<DocMode, boolean>
const resolved = resolveMode(requested, enabled)
if (resolved === null) {
return { name: ROUTES.DOCS_LIBRARY, query: { reason: 'no-mode-enabled' } }
}
if (resolved !== requested) {
return { ...to, query: { ...to.query, mode: resolved } }
}
return true
})
```
Pure helper is unit-testable; the `beforeEach` hook is integration-tested via `router.test.ts`.
### 5.4 i18n
A flash message displayed on `/docs` if `?reason=no-mode-enabled` is present:
- `flags.allModesDisabled` (fr + en).
The flash is rendered by `DocsLibraryPage.vue` (placeholder from #207) on mount; #211 will polish the rendering.
## 6. Alternatives considered
### Alternative A — 404 on disabled mode
- **Summary:** Show a "this mode is not available" 404.
- **Why not:** A deep link sent by a colleague becomes a dead end. Redirect is more graceful.
### Alternative B — Render the workspace and disable the tab
- **Summary:** No redirect; the workspace renders with the disabled mode replaced by a "disabled" placeholder.
- **Why not:** The URL still shows the disabled mode. A user copying it shares the same broken link.
## 7. API & data contract
### Endpoints
| Method | Path | Request | Response | Breaking? |
|--------|------|---------|----------|-----------|
| GET | `/api/health` | — | now includes `inspectModeEnabled`, `chunksModeEnabled`, `askModeEnabled` | No (additive) |
### Env vars
| Name | Default | Allowed | Notes |
|------|---------|---------|-------|
| `INSPECT_MODE_ENABLED` | `true` | `true` / `false` | gates the Inspect mode end-to-end |
| `CHUNKS_MODE_ENABLED` | `true` | `true` / `false` | gates the Chunks mode |
| `ASK_MODE_ENABLED` | `true` | `true` / `false` | gates the Ask mode |
### Breaking changes
None.
## 8. Risks & mitigations
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| All three flags off → user lands on `/docs` with a confusing flash | Clean Code | Low | Low | Visual smoke | The flash explicitly names the cause; admins should never set all three off |
| Frontend caches old flag values across navigation | Decoupling | Low | Medium | Stale UI | The flag store loads on app boot and exposes a `reload()` for support flows; revisit if needed |
| Backend default change breaks existing deployments | Security / Tests | Low | High | E2E catches | Defaults = `true` (enabled). Existing behaviour preserved unless operator opts out |
| Server-side write protection missing in 0.6.0 | Security | Medium | Medium | Manual / next audit | Documented as a follow-up; client-side gating is enough until chunks-edit endpoints land |
## 9. Testing strategy
### Backend — pytest
- `tests/test_schemas.py``HealthResponse` round-trip with the three new fields, defaults preserved.
- `tests/test_settings.py` (or equivalent) — env-var parsing for the three flags.
### Frontend — Vitest
- `shared/routing/resolveMode.test.ts` — full table over `(requested, enabled)` combinations including all-disabled, all-enabled, requested disabled, requested enabled.
- `app/router/router.test.ts``beforeEach` redirect scenarios via a mocked store.
- `features/feature-flags/store.test.ts` — three new flags exposed; falls back to `true` when health response is missing them.
### E2E — Karate UI
A focused smoke under `e2e/api/`: set `CHUNKS_MODE_ENABLED=false` in the test env, hit `/docs/abc?mode=chunks`, expect a redirect to `/docs/abc?mode=ask` (default).
### Manual QA
1. Default deploy → all three modes work.
2. Set `CHUNKS_MODE_ENABLED=false`, restart, visit `/docs/abc?mode=chunks` → redirected to `/docs/abc?mode=ask`.
3. Set all three to `false``/docs/abc?mode=ask` redirects to `/docs?reason=no-mode-enabled`.
## 10. Rollout & observability
### Release branch
`release/0.6.0`.
### Feature flag
This issue **is** the feature flag work. Defaults preserve current behaviour.
### Observability
- A single `INFO` log line on the server at boot: `feature_flags inspect=<true/false> chunks=<true/false> ask=<true/false>`.
- A `WARN` log when all three are off (operator misconfiguration smell).
### Rollback plan
Set all three env vars to `true` (or unset them) → defaults restore the old behaviour.
## 11. Open questions
- Per-tenant flags (multi-tenant deployments) — explicitly punted to post-0.6.0.
- Should the flash message be a banner or a toast? **Decision:** banner, dismissible, persistent until the user navigates away.
## 12. References
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/210
- **Related issues:** #207 (routing), #216 (workspace tabs), #205 follow-up (chunks-edit endpoints + server-side guard)
- **ADRs:** none planned
- **Project docs:** Architecture (`docs/architecture.md`), Frontend conventions (`frontend/CLAUDE.md`), Backend conventions (`document-parser/CLAUDE.md`)

View file

@ -0,0 +1,623 @@
# Design: Doc tab: chunk mode returns 404
<!--
Design doc template for Docling Studio.
One design doc per tracked issue. File path convention:
docs/design/<issue-number>-<kebab-slug>.md
Status lifecycle: Draft → In review → Accepted → Implemented (or Superseded).
Bump the Status line as the doc progresses; do not delete sections on the way.
This template is tailored to the project's architecture and conventions:
- Backend Hexagonal Architecture / ports & adapters
(domain → api/services/persistence/infra)
see docs/architecture.md
- Backend coding standards (FastAPI + Pydantic camelCase, aiosqlite,
Python snake_case internal, max 300 lines/file, 30 lines/function)
see docs/architecture/coding-standards.md
- Frontend feature-based organization (Vue 3 + Pinia, one store per
feature, Composition API, TypeScript strict, data-e2e selectors)
- E2E with Karate UI (NOT Playwright) — see e2e/CONVENTIONS.md
- Audit dimensions used at release gate — see docs/audit/master.md
- ADR process for load-bearing decisions — see docs/architecture/adr-guide.md
The `/conception` command pre-fills the header block and §1 / §2 / §12 from
the linked issue. Everything else is on the author.
-->
- **Issue:** #256
- **Title on issue:** [BUG] Doc tab: chunk mode returns 404
- **Author:** Pier-Jean Malandrino
- **Date:** 2026-05-07
- **Status:** Draft
- **Target milestone:** 0.6.0 — Doc-centric ingest
- **Impacted layers:** backend: api · services · (persistence read-only — déjà en place) · frontend: features/chunks (zéro changement attendu, déjà aligné sur le contrat) · e2e: nouveau scénario Karate
- **Audit dimensions likely touched:** Hexagonal Architecture · DDD · Decoupling · Tests · Documentation
- **ADR spawned?:** no — le modèle canonical `Chunk` (port + repository) existe déjà depuis #205, on ne fait que combler une couche service + API absente.
---
## 1. Problem
<!--
What hurts today, and for whom. Pull from the issue's Context + Current
behavior sections — keep the user's voice, do not paraphrase aggressively.
Two or three short paragraphs is usually enough. If you can't state the
problem in plain language, you are not ready to design a solution.
-->
Dans l'onglet **Doc** (`DocWorkspacePage` → `DocChunksTab`), l'activation du mode **chunk** déclenche un **HTTP 404**.
L'investigation a montré que ce n'est pas un endpoint isolé qui manque, mais **9 routes complètes côté `/api/documents/{id}/...`** qui sont appelées depuis le front mais n'existent pas côté backend. La couche frontend a été développée sur la cible 0.6.0 (doc-centric) avant que la couche API correspondante ne soit posée.
Le domain et la persistence du modèle canonical `Chunk` (entité de première classe, audit log via `ChunkEdit`, snapshots `ChunkPush`) existent **déjà** depuis l'issue #205 (`document-parser/domain/models.py`, `domain/ports.py`, `persistence/chunk_repo.py`, `persistence/chunk_edit_repo.py`). Ce qui manque, c'est la couche **service** et la couche **API** qui les exposent au front.
Note importante: le mode **OCR Debug** de `StudioPage` (`/api/analyses/*`) doit rester intact. Il opère sur des `AnalysisJob` éphémères avec pipeline options variables, ce qui est conceptuellement différent des chunks canonical d'un document. Les deux flux partagent le service de chunking sous-jacent (port `DocumentChunker`) mais exposent des ressources distinctes — pas de duplication d'endpoints.
## 2. Goals
<!--
Concrete, verifiable outcomes. Convert the issue's acceptance criteria into
checkboxes here; the design is "done" when all are satisfied. Keep the list
small — five or fewer goals is a good smell.
-->
- [ ] Mode chunk de l'onglet Doc renvoie la liste des chunks canonical sans 404 (endpoint `GET /api/documents/{id}/chunks` opérationnel).
- [ ] Toutes les actions d'édition fonctionnent: add, edit, delete, split, merge, rechunk (endpoints `POST/PATCH/DELETE` correspondants).
- [ ] Le push vers un store et le diff par store fonctionnent (`POST /push`, `GET /diff?store=…`).
- [ ] L'arbre structurel s'affiche sur l'onglet Inspect (`GET /tree`).
- [ ] Le mode OCR Debug (`StudioPage` → `/api/analyses/*`) reste totalement fonctionnel — non-régression.
- [ ] Promotion automatique: à la première analyse réussie d'un document, ses chunks deviennent canonical (peuplent `chunks` table via `ChunkRepository`).
- [ ] Tests: unit Vitest sur les nouveaux appels, pytest sur le service + router, e2e Karate sur le flow `Doc tab → mode chunk`.
## 3. Non-goals
<!--
What this design explicitly does NOT try to solve — and, for each, where it
*should* be solved (follow-up issue, next milestone, different audit area).
This is the section that saves the review: naming the off-ramps up front
prevents scope creep. If you leave this empty, reviewers will fill it in
for you, badly.
-->
- **Pas de refonte du modèle de données.** Le domain `Chunk` / `ChunkEdit` / `ChunkPush` est déjà posé (#205). On ne touche ni aux dataclasses ni au schéma SQLite.
- **Pas de refonte de StudioPage / OCR Debug.** Le flux `/api/analyses/*` reste tel quel; aucun port ni service de l'AnalysisService n'est modifié au-delà du hook de promotion canonical.
- **Pas de fusion `features/chunking/` ↔ `features/chunks/` côté front.** Ce sont deux features sémantiquement distinctes (acte de chunking debug vs état canonical persistant).
- **Pas de refactor des fichiers > 300 lignes** identifiés dans l'audit (StudioPage, GraphView, ChunksEditor) — sera traité dans des issues séparées du milestone 0.7.0.
- **Pas d'ajout de nouvelle table** ni de nouvelles colonnes — tout est déjà présent.
- **Pas de versioning d'API** — première exposition de ces routes, pas de breaking change.
## 4. Context & constraints
### Cartographie des 9 routes manquantes
Côté front (déjà écrit, fonctionne dès que le back répond):
| Méthode | URL | Front caller |
|---|---|---|
| `GET` | `/api/documents/{id}/chunks` | `features/chunks/api.ts:5` |
| `POST` | `/api/documents/{id}/chunks` | `features/chunks/api.ts:42` |
| `PATCH` | `/api/documents/{id}/chunks/{chunkId}` | `features/chunks/api.ts:13` |
| `DELETE` | `/api/documents/{id}/chunks/{chunkId}` | `features/chunks/api.ts:38` |
| `POST` | `/api/documents/{id}/chunks/{chunkId}/split` | `features/chunks/api.ts:31` |
| `POST` | `/api/documents/{id}/chunks/merge` | `features/chunks/api.ts:20` |
| `POST` | `/api/documents/{id}/rechunk` | `features/document/api.ts:31` |
| `GET` | `/api/documents/{id}/diff?store={name}` | `features/chunks/api.ts:49` |
| `POST` | `/api/documents/{id}/chunks/push` | `features/chunks/api.ts:56` |
| `POST` | `/api/documents/{id}/push` | `features/document/api.ts:35` (alias par doc — voir §5) |
| `GET` | `/api/documents/{id}/tree` | `features/document/api.ts:42` |
### Surface de code touchée
- **Backend (à créer)**:
- `document-parser/services/chunk_service.py` (nouveau)
- `document-parser/api/document_chunks.py` (nouveau router)
- `document-parser/api/schemas.py` (DTOs camelCase)
- `document-parser/main.py` (registration du router + injection du service)
- `document-parser/services/analysis_service.py` (hook de promotion canonical)
- `document-parser/tests/services/test_chunk_service.py` (nouveau)
- `document-parser/tests/api/test_document_chunks.py` (nouveau)
- **Backend (existant, lecture seule)**:
- `document-parser/domain/models.py``Chunk`, `ChunkEdit`, `ChunkPush`
- `document-parser/domain/ports.py``ChunkRepository`, `ChunkEditRepository`, `ChunkPushRepository`, `DocumentChunker`
- `document-parser/persistence/chunk_repo.py`, `chunk_edit_repo.py`
- `document-parser/services/ingestion_service.py` (réutilisé pour push)
- `document-parser/services/store_service.py` (réutilisé pour diff par store)
- **Frontend (zéro changement attendu)**:
- `frontend/src/features/chunks/api.ts` — déjà aligné sur le contrat
- `frontend/src/features/document/api.ts` — déjà aligné
- **E2E (à créer)**:
- `e2e/ui/src/test/resources/documents/doc-tab-chunk-mode.feature`
### Hexagonal Architecture
- Le domain ne change pas. Aucun nouveau port n'est ajouté: tous les besoins sont déjà couverts (`ChunkRepository`, `ChunkEditRepository`, `DocumentChunker`).
- Le nouveau `ChunkService` orchestre des ports existants et délègue à `AnalysisService.rechunk` pour le rechunk du doc canonical (DRY: la mécanique chunk-via-Docling est une seule implémentation).
- L'API ne franchit pas de couche: le router `document_chunks.py` parle uniquement aux services, jamais à `chunk_repo` directement.
- Conséquence: zéro nouvelle dépendance entre couches.
### Deployment modes
- Supporté: `latest-local` et `latest-remote` à l'identique. Aucun comportement spécifique au `CONVERSION_ENGINE``ChunkService` ne touche pas au converter, seulement au chunker (port `DocumentChunker` qui a une seule implémentation).
- HF Space: identique à `latest-remote`. Pas de feature flag à ajouter dans `/api/health` — les chunks canonical font partie du flow doc-centric standard.
### Hard constraints
- Pas de migration SQLite (tables existent déjà).
- Pas de breaking change API (premières routes exposées).
- Performance: les opérations chunk lisent/écrivent ≤ N chunks pour un doc (typiquement < 200). Aucune requête non-bornée. Le rechunk lance Docling chunker en thread (`asyncio.to_thread`), comme `analysis_service.rechunk` actuel.
## 5. Proposed design
<!--
The recommended approach, in enough detail that a competent engineer
outside the immediate context can implement it. Describe contracts, not
code — the PR is where code lives.
Structure this section by layer. Skip a layer if it is genuinely untouched;
do not pad.
### 5.1 Domain
New or changed dataclasses / value objects / ports in `document-parser/domain/`.
No HTTP or DB concerns here. If you are adding a port (`Protocol`), give its
full signature.
### 5.2 Persistence
Schema changes (table, columns, indexes), migration plan, aiosqlite query
shape. Note whether existing rows need a backfill.
### 5.3 Infra adapters
New or changed adapters in `document-parser/infra/` (converter, chunker,
rate limiter, settings). For new env vars, give name / default / allowed
values.
### 5.4 Services
Use-case orchestration in `document-parser/services/`. Services do NOT
implement — they delegate. Describe the call sequence, error handling,
and concurrency (how does this interact with `MAX_CONCURRENT_ANALYSES`?).
### 5.5 API
Endpoint additions / changes in `document-parser/api/`. For each:
- Method + path
- Request DTO (Pydantic, camelCase via alias_generator)
- Response DTO (camelCase; remember `pages_json` stays snake_case)
- Error responses (status codes, shape)
- Whether it is excluded from the rate limiter (like `/api/health`)
### 5.6 Frontend — feature module
Which `frontend/src/features/<name>/` folder, which Pinia store actions,
which API client calls in `api.ts`, which Vue components in `ui/`. Name
new `data-e2e` attributes here (Karate needs them).
### 5.7 Cross-cutting
Feature flags (how the backend advertises capability via `/api/health` and
how the frontend reacts), i18n strings (`shared/i18n.ts`), shared types
(`shared/types.ts`).
Prefer mermaid / ASCII for sequence and data flow. Interfaces are more
valuable than pseudocode.
-->
### 5.1 Domain
**Aucun changement.** Tout est en place depuis #205:
- `Chunk(id, document_id, sequence, text, headings, source_page, bboxes, doc_items, token_count, created_at, updated_at, deleted_at)`
- `ChunkEdit(id, document_id, chunk_id, action ∈ INSERT|UPDATE|DELETE|MERGE|SPLIT, actor, at, before, after, parents, children, reason)`
- `ChunkPush(id, document_id, store_id, chunkset_hash, chunk_ids, …)`
- Ports: `ChunkRepository`, `ChunkEditRepository`, `ChunkPushRepository`, `DocumentChunker`.
### 5.2 Persistence
**Aucun changement.** `chunk_repo.py` et `chunk_edit_repo.py` couvrent CRUD + audit. Le schéma SQLite existe déjà (vérifier `database.py`). Si une colonne ou un index manque pour les nouvelles requêtes (par ex. lookup chunks d'un doc ordonnés par `sequence`), ajouter dans la PR via un script idempotent.
### 5.3 Infra adapters
**Aucun changement.** `DocumentChunker` est déjà implémenté (le même qui sert `analysis_service.rechunk`).
### 5.4 Services
#### Nouveau: `ChunkService``document-parser/services/chunk_service.py`
Orchestre toutes les opérations sur le chunkset canonical d'un document. Délègue aux ports existants. Écrit les `ChunkEdit` en transaction avec chaque mutation.
```python
class ChunkService:
def __init__(
self,
chunk_repo: ChunkRepository,
chunk_edit_repo: ChunkEditRepository,
chunker: DocumentChunker,
document_repo: DocumentRepository,
analysis_service: AnalysisService, # pour le rechunk
): ...
async def list_chunks(self, doc_id: str) -> list[Chunk]: ...
async def add_chunk(self, doc_id: str, text: str, after_id: str | None) -> Chunk: ...
async def update_chunk(self, doc_id: str, chunk_id: str, *, text: str | None, headings: list[str] | None) -> Chunk: ...
async def delete_chunk(self, doc_id: str, chunk_id: str) -> None: ...
async def split_chunk(self, doc_id: str, chunk_id: str, cursor_offset: int) -> list[Chunk]: ...
async def merge_chunks(self, doc_id: str, ids: list[str]) -> Chunk: ...
async def rechunk_document(self, doc_id: str, opts: ChunkingOptions | None) -> list[Chunk]: ...
async def get_tree(self, doc_id: str) -> list[DocTreeNode]: ...
async def diff_against_store(self, doc_id: str, store: str) -> list[ChunkDiff]: ...
async def push_to_store(self, doc_id: str, store: str) -> ChunkPush: ...
```
Règles d'invariants:
- `sequence` des chunks reste dense ascendant. `add_chunk` insère et incrémente les suivants. `delete_chunk` est soft (marque `deleted_at`) pour préserver l'audit; les listings filtrent.
- Toute mutation produit **une ligne `ChunkEdit`** atomiquement.
- `rechunk_document`: appelle `analysis_service.rechunk_for_document(doc_id, opts)` (helper public à exposer dans `AnalysisService` si pas déjà), récupère la liste de chunks, **remplace** le chunkset canonical (delete soft de l'ancien + insert nouveau), enregistre un `ChunkEdit` `RESET`.
- `get_tree`: dérive de la dernière `AnalysisJob` réussie (lecture de `pages_json` / `document_json`). Pas de calcul nouveau.
- `push_to_store` et `diff_against_store`: délèguent à `ingestion_service` / `store_service` existants. Aucune logique dupliquée; le service expose juste l'interface "doc-centric".
#### Hook de promotion canonical — `AnalysisService.create`
À la fin d'une analyse réussie, si le document n'a pas encore de chunkset canonical (`chunk_repo.count_for_document(doc_id) == 0`), peupler depuis `job.chunks_json`:
```python
async def create(self, ...) -> AnalysisJob:
job = await self._run_analysis(...)
if job.status == AnalysisStatus.COMPLETED and job.chunks_json:
if await self._chunk_repo.count_for_document(job.document_id) == 0:
await self._promote_to_canonical(job) # nouveau private method
return job
```
Promotion = parse `chunks_json` → instancie `Chunk(document_id=…)``chunk_repo.bulk_insert(chunks)``chunk_edit_repo.insert(ChunkEdit(action=INSERT, actor='system:initial-analysis'))`.
C'est le **seul** point qui croise les deux contextes (analyse → doc canonical). Reste local à `AnalysisService`.
### 5.5 API
#### Nouveau router: `document-parser/api/document_chunks.py`
Préfixe `/api/documents/{doc_id}`, tag `documents`. Toutes les routes injectent `ChunkService` via `Depends`.
```python
@router.get("/{doc_id}/chunks", response_model=list[ChunkResponse])
@router.post("/{doc_id}/chunks", response_model=ChunkResponse) # body: AddChunkRequest
@router.patch("/{doc_id}/chunks/{chunk_id}", response_model=ChunkResponse) # body: UpdateChunkRequest
@router.delete("/{doc_id}/chunks/{chunk_id}", status_code=204)
@router.post("/{doc_id}/chunks/{chunk_id}/split", response_model=list[ChunkResponse]) # body: SplitChunkRequest { cursorOffset }
@router.post("/{doc_id}/chunks/merge", response_model=ChunkResponse) # body: MergeChunksRequest { ids }
@router.post("/{doc_id}/rechunk", response_model=list[ChunkResponse]) # body: RechunkRequest (optionnel)
@router.get("/{doc_id}/tree", response_model=list[DocTreeNodeResponse])
@router.get("/{doc_id}/diff", response_model=list[ChunkDiffResponse]) # query: store
@router.post("/{doc_id}/chunks/push", response_model=PushSummaryResponse) # body: PushRequest { store }
```
Note: deux URLs présentes côté front pour push (`/{id}/push` dans `document/api.ts` et `/{id}/chunks/push` dans `chunks/api.ts`). Une seule route doit exister côté backend pour respecter la directive "no duplicate". **Choix**: garder `/{id}/chunks/push` (sémantiquement plus juste — c'est bien des chunks qu'on pousse). Mettre à jour `features/document/api.ts:34-39` pour cibler la même URL et virer la duplication.
DTOs Pydantic: tous camelCase (alias_generator), `chunkId` au lieu de `chunk_id` côté wire, `sourcePage` etc.
Erreurs:
- `404` si doc inexistant ou chunk inexistant
- `409` si `merge_chunks` reçoit des ids non contigus (invariant `sequence`)
- `400` si `split` reçoit un `cursorOffset` hors range
### 5.6 Frontend — feature module
**Zéro changement attendu.** `features/chunks/api.ts` et `features/document/api.ts` sont déjà alignés.
Une seule correction à apporter pour respecter "no duplicate endpoints": modifier `features/document/api.ts:35` pour pointer sur `/api/documents/${id}/chunks/push` au lieu de `/api/documents/${id}/push`. Ensuite cette fonction `pushDocumentToStore` peut être supprimée et le caller (`document/store.ts`) bascule sur `pushChunksToStore` de `features/chunks/api.ts`.
Bug latents identifiés par l'audit, à corriger dans la même PR (scope #256):
- `ChunksEditor.vue:235``saveTimer` non clearé sur `onUnmounted`. Ajouter `onBeforeUnmount(() => clearTimeout(saveTimer))`.
- `ChunksEditor.vue:215` — remplacer `alert(...)` natif par le pattern toast partagé.
### 5.7 Cross-cutting (feature flags, i18n, shared types)
- **Feature flags**: aucun. Le mode chunk est un flow standard du doc-centric, pas optionnel.
- **i18n**: aucune nouvelle clé requise; les strings existent déjà côté `chunks.*`.
- **Shared types**: `DocChunk`, `ChunkDiff`, `PushSummary`, `DocTreeNode` existent déjà dans `frontend/src/shared/types.ts`. Vérifier la correspondance camelCase avec les nouveaux DTOs Pydantic; aucun champ nouveau attendu.
## 6. Alternatives considered
<!--
At least two genuine alternatives, each with a one-paragraph description
and the reason it was rejected. "Do nothing" is often a legitimate
alternative — name it if it is. Reviewers use this section to sanity-check
that the recommended design was a choice and not the first thing that
came to mind.
If one of the alternatives represents a significant architectural fork
(e.g. introducing a new service, replacing a library), spawn an ADR under
`docs/architecture/adrs/` and link it in §12 — the design doc captures the
local decision, the ADR captures the cross-cutting one.
-->
### Alternative A — Recâbler le front sur `/api/analyses/*`
- **Summary:** Côté front, résoudre `docId → latest analysis jobId` dans `DocWorkspacePage`, puis appeler les endpoints `/api/analyses/{jobId}/...` existants (rechunk, chunks/{idx}, etc.). Aucun nouveau endpoint backend.
- **Why not:** Conceptuellement faux. Une analyse est une run éphémère avec ses propres pipeline options; un document a un état chunk canonical persistant édité dans le temps. Identifier les deux casserait le modèle 0.6.0 doc-centric et empêcherait d'avoir plusieurs analyses parallèles d'un doc (ce que StudioPage permet) sans muter l'état canonical. Casse aussi l'audit log par chunk (`ChunkEdit`) qui est par doc, pas par job.
### Alternative B — Fusion `Analysis``Document chunks`
- **Summary:** Supprimer `analysis_jobs.chunks_json`, faire que toute analyse écrive directement dans la table `chunks` du doc. StudioPage devient une UI de re-chunking d'un doc.
- **Why not:** Casse l'usage debug de StudioPage (plusieurs runs avec pipeline options ≠ ne peuvent pas coexister). Refactor majeur, hors scope d'un bug fix. Pourrait être envisagé en 0.7.0+ si l'usage de StudioPage évolue.
## 7. API & data contract
### Endpoints (tous nouveaux, additifs)
| Method | Path | Request | Response | Breaking? |
|--------|------|---------|----------|-----------|
| GET | `/api/documents/{id}/chunks` | — | `ChunkResponse[]` | additive |
| POST | `/api/documents/{id}/chunks` | `AddChunkRequest` `{text, afterId?}` | `ChunkResponse` | additive |
| PATCH | `/api/documents/{id}/chunks/{chunkId}` | `UpdateChunkRequest` `{text?, headings?}` | `ChunkResponse` | additive |
| DELETE | `/api/documents/{id}/chunks/{chunkId}` | — | 204 | additive |
| POST | `/api/documents/{id}/chunks/{chunkId}/split` | `SplitChunkRequest` `{cursorOffset}` | `ChunkResponse[]` | additive |
| POST | `/api/documents/{id}/chunks/merge` | `MergeChunksRequest` `{ids}` | `ChunkResponse` | additive |
| POST | `/api/documents/{id}/rechunk` | `RechunkRequest` `{chunkingOptions?}` | `ChunkResponse[]` | additive |
| GET | `/api/documents/{id}/tree` | — | `DocTreeNodeResponse[]` | additive |
| GET | `/api/documents/{id}/diff?store={name}` | — | `ChunkDiffResponse[]` | additive |
| POST | `/api/documents/{id}/chunks/push` | `PushRequest` `{store}` | `PushSummaryResponse` | additive |
Sérialisation **camelCase** côté wire (Pydantic `alias_generator`), snake_case interne. `pages_json` reste l'exception documentée si réutilisé.
### Persistence schema
```sql
-- Aucune migration. Tables existantes (issue #205):
-- chunks(id, document_id, sequence, text, headings_json, source_page, bboxes_json,
-- doc_items_json, token_count, created_at, updated_at, deleted_at)
-- chunk_edits(id, document_id, chunk_id, action, actor, at, before_json, after_json,
-- parents_json, children_json, reason)
-- chunk_pushes(id, document_id, store_id, chunkset_hash, chunk_ids_json, ...)
```
### Env vars / config
| Name | Default | Allowed | Notes |
|------|---------|---------|-------|
| _(aucune)_ | | | Le `ChunkService` ne lit aucune nouvelle config. |
### Breaking changes
**Aucun.** Toutes les routes sont additives, premier passage live. Côté front, suppression de l'alias `pushDocumentToStore` (URL `/push`) au profit de `pushChunksToStore` (URL `/chunks/push`) — pas exposé externement.
<!--
Make the wire contract explicit — this is what the frontend, e2e tests,
and any external consumer will code against.
### Endpoints
| Method | Path | Request | Response | Breaking? |
|--------|------|---------|----------|-----------|
| | | | | |
Remember:
- API serialization is camelCase (Pydantic `alias_generator`).
- Backend internals stay snake_case.
- `pages_json` is the documented exception — it carries raw
`dataclasses.asdict()` output (snake_case).
- Health endpoint (`/api/health`) may need new fields if this design adds
a feature flag.
### Persistence schema
```sql
-- ALTER TABLE / CREATE TABLE statements, with reasoning
```
### Env vars / config
| Name | Default | Allowed | Notes |
|------|---------|---------|-------|
| | | | |
### Breaking changes
Enumerate anything a consumer must change. If there are none, say so
explicitly — "additive only" is a useful commitment.
-->
## 8. Risks & mitigations
<!--
One row per non-trivial risk. Map each to an audit dimension so the
release-gate audit has a clear hook:
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|-----------|--------|---------------|------------------------|
| | Security | | | | |
| | Performance | | | | |
| | Decoupling | | | | |
Common families to scan for:
- **Hexagonal Architecture:** cross-layer imports, leaking HTTP into domain, adapter bypassing its port
- **Security:** rate limiter bypass, path traversal on uploads, SSRF via
the remote converter, unauthenticated data exposure
- **Performance:** synchronous work on the FastAPI event loop,
unbounded queries, new work inside `MAX_CONCURRENT_ANALYSES` budget
- **Tests:** coverage gap on a critical path
- **Documentation:** missing README / env var / i18n entry
A design with "no risks identified" is a design that has not been read
carefully.
-->
| Risk | Audit dimension | Likelihood | Impact | How we notice | Mitigation / rollback |
|------|-----------------|------------|--------|---------------|------------------------|
| Promotion canonical s'exécute deux fois (race) sur deux analyses simultanées d'un doc neuf, dupliquant le chunkset | DDD / Decoupling | basse | élevée (corruption canonical) | tests d'intégration concurrent + count_for_document inattendu en prod | Garde idempotente dans `_promote_to_canonical` (vérif count + insert sous transaction unique) |
| Régression OCR Debug si `analysis_service.create` modifié maladroitement | Tests / Decoupling | moyenne | élevée (StudioPage cassé) | non-régression e2e Karate `full-happy-path.feature` | Hook de promotion isolé en méthode privée, tests pytest qui couvrent les deux cas (analyse 1ère, analyse N) |
| Endpoint `/diff` lent si beaucoup de chunks × store comparé | Performance | basse | moyenne | logs de durée > 1s | Borne dure (max 500 chunks par doc), pagination si dépassé en 0.7 |
| `ChunkEdit` audit log non écrit en cas d'erreur partielle (ex: split crash après insert chunk 1/2) | Hexagonal Architecture / DDD | moyenne | moyenne (audit incohérent) | tests intégration + assert chunk_edit count après chaque mutation | Toutes les mutations en transaction aiosqlite unique chunk + edit |
| `pushDocumentToStore` (URL legacy `/push`) encore appelé quelque part en prod | Decoupling | basse | basse (404 silencieux) | grep + e2e | Suppression complète de la fonction TS, pas d'alias |
| Test e2e Karate flaky si l'analyse initiale n'a pas eu le temps de promouvoir | Tests | moyenne | basse | flaky en CI | E2E poll sur `lifecycle_state == 'parsed'` avant de cliquer sur l'onglet chunk |
## 9. Testing strategy
### Backend — pytest (`document-parser/tests/`)
- **Unit `tests/services/test_chunk_service.py`**:
- `list_chunks` filtre les `deleted_at` non null
- `add_chunk` insert + audit `INSERT` + recale les `sequence` suivants
- `update_chunk` modifie + audit `UPDATE` avec `before/after` correctement remplis
- `delete_chunk` soft-delete + audit `DELETE`
- `split_chunk` produit 2 chunks + 2 audit `SPLIT` avec `parents = [source_id]`
- `merge_chunks` produit 1 chunk + 1 audit `MERGE` avec `parents = [ids]`
- `merge_chunks` rejette ids non contigus (409)
- `rechunk_document` remplace canonical et écrit un audit `RESET`
- `push_to_store` délègue à `ingestion_service` (mock)
- `diff_against_store` délègue à `store_service` (mock)
- **Unit `tests/services/test_analysis_service.py` (existant, à étendre)**:
- `create()` peuple chunks canonical à la 1ère analyse réussie
- `create()` ne touche PAS au canonical aux analyses suivantes (idempotence)
- `create()` ne crée rien si l'analyse échoue
- **Integration `tests/api/test_document_chunks.py`**:
- Chaque route → 200 happy path
- Chaque route → 404 si doc inexistant
- Chaque route mutante → 404 si chunk inexistant
- `merge` → 409 si non contigus
- `split` → 400 si offset hors range
- DTOs camelCase respectés sur input ET output
- **Architecture tests**: vérifier que `api/document_chunks.py` n'importe pas `persistence/*` directement.
### Frontend — Vitest
- **`features/chunks/api.test.ts` (existant, à étendre)**:
- Tests d'erreur HTTP (404, 500) sur `fetchChunks`, `updateChunk`, `mergeChunks`, `splitChunk`, `dropChunk`, `addChunk`, `fetchChunkDiff`, `pushChunksToStore`
- **`features/chunks/store.test.ts` (existant, à étendre)**: `$reset` à chaque changement de `docId` (corrige le bug latent identifié dans l'audit).
- **`features/document/api.test.ts`**: ajouter tests pour `fetchDocumentTree`, `rechunkDocument`. Supprimer `pushDocumentToStore` (réécriture vers `pushChunksToStore`).
- **`features/document/store.test.ts`**: idem.
### E2E — Karate UI (`e2e/`)
- **Nouveau `e2e/ui/src/test/resources/documents/doc-tab-chunk-mode.feature`**:
- Setup via API: upload doc → wait `lifecycle_state == 'parsed'`
- UI: ouvrir `/docs/{id}` → cliquer onglet **Doc** → activer mode `chunk`
- Assert: liste de chunks visible, **pas** de bannière 404, count > 0
- Edit un chunk → assert persistence après reload
- Cleanup via API: delete doc
- Tags: `@critical @ui`
- **Non-régression sur OCR Debug**: vérifier que `e2e/ui/src/test/resources/full-happy-path.feature` (StudioPage) reste vert après les modifications.
### Manual QA
1. `docker-compose -f docker-compose.dev.yml up`
2. Upload un PDF via `/docs/new`, attendre status `parsed`
3. Ouvrir `/docs/{id}` → onglet **Doc** → bouton **chunk** → vérifier liste affichée
4. Ouvrir `/studio` → upload + run analyse OCR → vérifier que c'est inchangé
5. Push doc vers un store → vérifier `/api/documents/{id}/diff?store=…`
### Performance / load
Pas d'objectif chiffré particulier. Surveiller: `rechunk_document` doit rester sous le budget `MAX_CONCURRENT_ANALYSES` (réutilise le même mécanisme que `analysis_service.rechunk`).
<!--
How this design will be verified. Be specific — name files / suites.
### Backend — pytest (`document-parser/tests/`)
- Unit: per-layer (`tests/domain/`, `tests/persistence/`, `tests/services/`)
- Integration: services wired with real aiosqlite + real adapters
- Architecture tests (if applicable): enforce import boundaries
### Frontend — Vitest (`frontend/src/**/*.test.ts`)
- Stores: actions / getters / mocked API
- Pure helpers (e.g. `bboxScaling.ts`-style modules): deterministic
- Components only when behavior is non-trivial; do not test markup
### E2E — Karate UI (`e2e/`)
- Use `data-e2e` selectors — never CSS classes (see e2e/CONVENTIONS.md)
- `retry()` / `waitFor()` — never `Thread.sleep()` / `delay()`
- Setup via API, verify via UI, cleanup via API
- Tag appropriately: `@critical` / `@ui` / `@smoke` / `@regression` / `@e2e`
- **Never Playwright** — Karate is the tool here.
### Manual QA
Steps the reviewer can run locally (`docker-compose.dev.yml` up, scenario
to reproduce). Keep it short — if the manual list is long, automate more.
### Performance / load
Required when the design claims a latency / throughput / memory property,
or touches the conversion hot path.
-->
## 10. Rollout & observability
### Release branch
`release/0.6.0` — c'est l'objet du milestone "Doc-centric ingest". Branche de travail: `fix/256-doc-tab-chunk-mode-404`.
### Feature flag / staged rollout
**Aucun flag.** Le mode chunk fait partie du flow standard 0.6.0; pas de fallback derrière un toggle. La promotion canonical des chunks à la 1ère analyse est inconditionnelle.
### Observability
- Logs structurés à ajouter dans `ChunkService`:
- `chunk.add docId=… chunkId=… sequence=…`
- `chunk.update docId=… chunkId=…`
- `chunk.delete docId=… chunkId=…`
- `chunk.split docId=… sourceId=… newIds=…`
- `chunk.merge docId=… sourceIds=… newId=…`
- `chunk.rechunk docId=… count=… durationMs=…`
- `chunk.push docId=… store=… count=…`
- `chunk.promote docId=… count=… (initial-analysis)`
- Pas de nouvelle métrique Prometheus.
- Erreurs: les `HTTPException` 4xx ne loggent pas (normal); les 5xx loggent avec stack via `logger.exception` (pattern existant dans `documents.py:131`).
### Rollback plan
- Le déploiement est entièrement additif (nouvelles routes + nouveau service). Rollback = revert du commit / image précédente. Aucun changement de schéma à défaire.
- Si un bug critique apparaît côté promotion canonical: désactiver le hook dans `analysis_service.create` via revert ciblé; les analyses continuent de fonctionner sans canonical (l'onglet chunk redeviendrait 404, état antérieur). Pas de perte de données.
- Si un bug critique apparaît côté API chunks: revert du router suffit; l'onglet chunk redevient 404, le reste de l'app reste fonctionnel.
Liens:
- Deployment: `docs/release/*` (`/release:deploy`)
- Rollback: `/release:rollback`
- Incident: `docs/operations/*` (`/ops:incident`)
<!--
How this change gets to production safely.
### Release branch
Which `release/X.Y.Z` is the target? Any coordination with a parallel
release (e.g. R&D branch)?
### Feature flag / staged rollout
Does the change hide behind a flag surfaced via `/api/health`? If so, what
flips the flag, and what is the default? HF Space deployments often need
`deploymentMode === 'huggingface'` gating.
### Observability
- Logs to add / extend (structured, low-cardinality keys)
- Metrics / counters (if added — call out any new Prometheus names)
- New error modes to watch for in `analysis_jobs.status = FAILED`
### Rollback plan
The revert that is safe to apply at any time:
- Which migration is reversible? Which is not?
- Which env var flip disables the feature without a redeploy?
- Any data cleanup needed after rollback?
Link to the existing release / ops playbooks:
- Deployment: `docs/release/*` (also surfaced via `/release:deploy`)
- Rollback: also surfaced via `/release:rollback`
- Incident: `docs/operations/*` (also surfaced via `/ops:incident`)
-->
## 11. Open questions
- Le port `DocumentChunker` accepte-t-il `cursor_offset` pour `split`, ou faut-il l'implémenter manuellement dans `ChunkService.split_chunk` (split textuel basique sans repasser dans Docling) ? À vérifier au moment de l'implémentation.
- Faut-il enregistrer un `ChunkEdit` `RESET` lors du `rechunk_document`, ou un `DELETE` par chunk + `INSERT` par nouveau chunk ? Décision: une seule entrée `RESET` (lisibilité audit), avec `parents = [old_ids]`, `children = [new_ids]`.
- `get_tree`: lit-on `pages_json` ou `document_json` ? À confirmer en lisant `analysis_service.find_by_id` — probablement `document_json` qui contient la structure docling complète.
## 12. References
<!--
Links to everything a future reader would want.
-->
- **Issue:** https://github.com/scub-france/Docling-Studio/issues/256
- **Related PRs / commits:**
- **ADRs:** <ADR-NNN or "none planned">
- **Project docs:**
- Architecture: `docs/architecture.md`
- Coding standards: `docs/architecture/coding-standards.md`
- ADR guide / template: `docs/architecture/adr-guide.md`, `docs/architecture/adr-template.md`
- Audit master: `docs/audit/master.md`
- E2E conventions: `e2e/CONVENTIONS.md`
- **External:** <specs, upstream issues, dashboards, third-party docs>

View file

@ -5,7 +5,7 @@ from __future__ import annotations
import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Request
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from api.schemas import (
AnalysisResponse,
@ -75,9 +75,15 @@ async def create_analysis(body: CreateAnalysisRequest, service: ServiceDep) -> A
@router.get("", response_model=list[AnalysisResponse])
async def list_analyses(service: ServiceDep) -> list[AnalysisResponse]:
"""List all analysis jobs."""
jobs = await service.find_all()
async def list_analyses(
service: ServiceDep,
document_id: str | None = Query(default=None, alias="documentId"),
) -> list[AnalysisResponse]:
"""List analysis jobs, optionally filtered by documentId."""
if document_id:
jobs = await service.find_by_document(document_id)
else:
jobs = await service.find_all()
return [_to_response(j) for j in jobs]

View file

@ -0,0 +1,212 @@
"""Document chunks API router (#256).
Exposes the canonical doc-centric chunkset (CRUD + tree + rechunk + diff
+ push). Distinct from `/api/analyses/{id}/...` which operates on
ephemeral analysis runs (StudioPage / OCR Debug).
All routes mount under `/api/documents/{doc_id}/...`. The router
delegates to `ChunkService` no direct repo / persistence access.
"""
from __future__ import annotations
import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from api.schemas import (
AddChunkRequest,
ChunkDiffResponse,
DocChunkResponse,
DocRechunkRequest,
DocTreeNodeResponse,
MergeChunksRequest,
PushChunksRequest,
PushChunksResponse,
SplitChunkRequest,
UpdateChunkRequest,
)
from services.chunk_service import (
ChunkService,
ChunkServiceError,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/documents", tags=["documents"])
def _get_service(request: Request) -> ChunkService:
svc = getattr(request.app.state, "chunk_service", None)
if svc is None:
raise HTTPException(status_code=503, detail="Chunk service not available")
return svc
ServiceDep = Annotated[ChunkService, Depends(_get_service)]
def _to_response(chunk) -> DocChunkResponse:
return DocChunkResponse(
id=chunk.id,
doc_id=chunk.document_id,
sequence=chunk.sequence,
text=chunk.text,
headings=list(chunk.headings),
source_page=chunk.source_page,
token_count=chunk.token_count,
created_at=str(chunk.created_at),
updated_at=str(chunk.updated_at),
)
def _raise_for(error: ChunkServiceError) -> None:
raise HTTPException(status_code=error.http_status, detail=str(error))
# ---------------------------------------------------------------------------
# CRUD
# ---------------------------------------------------------------------------
@router.get("/{doc_id}/chunks", response_model=list[DocChunkResponse])
async def list_chunks(doc_id: str, service: ServiceDep) -> list[DocChunkResponse]:
"""List the canonical chunkset for a document, ordered by sequence."""
try:
chunks = await service.list_chunks(doc_id)
except ChunkServiceError as e:
_raise_for(e)
return [_to_response(c) for c in chunks]
@router.post("/{doc_id}/chunks", response_model=DocChunkResponse, status_code=201)
async def add_chunk(doc_id: str, body: AddChunkRequest, service: ServiceDep) -> DocChunkResponse:
"""Insert a new chunk (optionally after `afterId`)."""
if not body.text:
raise HTTPException(status_code=400, detail="text is required")
try:
chunk = await service.add_chunk(doc_id, text=body.text, after_id=body.after_id)
except ChunkServiceError as e:
_raise_for(e)
return _to_response(chunk)
@router.patch("/{doc_id}/chunks/{chunk_id}", response_model=DocChunkResponse)
async def update_chunk(
doc_id: str,
chunk_id: str,
body: UpdateChunkRequest,
service: ServiceDep,
) -> DocChunkResponse:
"""Update a chunk's text or title (mapped to first heading)."""
if body.text is None and body.title is None:
raise HTTPException(status_code=400, detail="text or title is required")
headings: list[str] | None = None
if body.title is not None:
headings = [body.title] if body.title else []
try:
chunk = await service.update_chunk(doc_id, chunk_id, text=body.text, headings=headings)
except ChunkServiceError as e:
_raise_for(e)
return _to_response(chunk)
@router.delete("/{doc_id}/chunks/{chunk_id}", status_code=204, response_model=None)
async def delete_chunk(doc_id: str, chunk_id: str, service: ServiceDep) -> None:
"""Soft-delete a chunk."""
try:
await service.delete_chunk(doc_id, chunk_id)
except ChunkServiceError as e:
_raise_for(e)
@router.post("/{doc_id}/chunks/{chunk_id}/split", response_model=list[DocChunkResponse])
async def split_chunk(
doc_id: str,
chunk_id: str,
body: SplitChunkRequest,
service: ServiceDep,
) -> list[DocChunkResponse]:
"""Split a chunk in two at `cursorOffset` characters."""
try:
chunks = await service.split_chunk(doc_id, chunk_id, body.cursor_offset)
except ChunkServiceError as e:
_raise_for(e)
return [_to_response(c) for c in chunks]
@router.post("/{doc_id}/chunks/merge", response_model=DocChunkResponse)
async def merge_chunks(
doc_id: str, body: MergeChunksRequest, service: ServiceDep
) -> DocChunkResponse:
"""Merge contiguous chunks into one. Order in `ids` is irrelevant — the
service sorts by sequence."""
try:
merged = await service.merge_chunks(doc_id, body.ids)
except ChunkServiceError as e:
_raise_for(e)
return _to_response(merged)
# ---------------------------------------------------------------------------
# Rechunk + tree + diff + push
# ---------------------------------------------------------------------------
@router.post("/{doc_id}/rechunk", response_model=list[DocChunkResponse])
async def rechunk_document(
doc_id: str,
body: DocRechunkRequest | None,
service: ServiceDep,
) -> list[DocChunkResponse]:
"""Re-run the chunker on the latest analysis JSON; replace canonical."""
options = body.chunking_options.model_dump() if body and body.chunking_options else None
try:
chunks = await service.rechunk_document(doc_id, options)
except ChunkServiceError as e:
_raise_for(e)
return [_to_response(c) for c in chunks]
@router.get("/{doc_id}/tree", response_model=list[DocTreeNodeResponse])
async def get_tree(doc_id: str, service: ServiceDep) -> list[DocTreeNodeResponse]:
"""Outline of the document built from the latest completed analysis.
Returns `[]` if no analysis is available yet.
"""
try:
tree = await service.get_tree(doc_id)
except ChunkServiceError as e:
_raise_for(e)
return [DocTreeNodeResponse(**node) for node in tree]
@router.get("/{doc_id}/diff", response_model=list[ChunkDiffResponse])
async def diff_against_store(
doc_id: str,
service: ServiceDep,
store: str = Query(..., min_length=1),
) -> list[ChunkDiffResponse]:
"""Diff the canonical chunkset against the last push to `store`."""
try:
diffs = await service.diff_against_store(doc_id, store)
except ChunkServiceError as e:
_raise_for(e)
return [ChunkDiffResponse(**d) for d in diffs]
@router.post("/{doc_id}/chunks/push", response_model=PushChunksResponse)
async def push_chunks(
doc_id: str,
body: PushChunksRequest,
service: ServiceDep,
) -> PushChunksResponse:
"""Push the canonical chunkset to a store and record a `ChunkPush`."""
try:
result = await service.push_to_store(doc_id, body.store)
except ChunkServiceError as e:
_raise_for(e)
return PushChunksResponse(
job_id=result["jobId"],
summary=result["summary"],
)

View file

@ -34,6 +34,8 @@ def _to_response(doc) -> DocumentResponse:
file_size=doc.file_size,
page_count=doc.page_count,
created_at=str(doc.created_at),
lifecycle_state=doc.lifecycle_state.value,
lifecycle_state_at=(str(doc.lifecycle_state_at) if doc.lifecycle_state_at else None),
)

View file

@ -46,6 +46,15 @@ class HealthResponse(_CamelModel):
# available: REASONING_ENABLED=true AND deps importable. Doesn't imply
# Ollama itself is reachable — that's checked per-call.
reasoning_available: bool = False
# 0.6.1 — Surface flags (#257). Master flags select which surface(s)
# the frontend exposes. Defaults match the production target (RAG only).
studio_mode_enabled: bool = False
rag_pipeline_enabled: bool = True
# 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257). Default true
# so frontends pointed at an older backend keep every mode visible.
inspect_mode_enabled: bool = True
linked_mode_enabled: bool = True
ask_mode_enabled: bool = True
class DocumentResponse(_CamelModel):
@ -56,6 +65,11 @@ class DocumentResponse(_CamelModel):
file_size: int | None = None
page_count: int | None = None
created_at: str | datetime
# 0.6.0 — Document lifecycle state machine (#202). The lifecycle
# describes the document as a whole; `status` above is kept for
# backwards compat and currently still maps to `DOCUMENT_STATUS_UPLOADED`.
lifecycle_state: str = "Uploaded"
lifecycle_state_at: str | datetime | None = None
class AnalysisResponse(_CamelModel):
@ -222,3 +236,139 @@ class SearchResponse(_CamelModel):
results: list[SearchResultItem]
total: int
query: str
# ---------------------------------------------------------------------------
# Stores (#251)
# ---------------------------------------------------------------------------
class StoreInfoResponse(_CamelModel):
"""Read model for `GET /api/stores`."""
name: str
slug: str
type: str
embedder: str
is_default: bool
document_count: int
chunk_count: int
connected: bool
error_message: str | None = None
class StoreResponse(_CamelModel):
"""Detailed read model for `GET /api/stores/{slug}`."""
id: str
name: str
slug: str
kind: str
embedder: str
is_default: bool
config: dict
created_at: str | datetime
class StoreCreateRequest(_CamelModel):
name: str
slug: str
kind: str
embedder: str
config: dict = Field(default_factory=dict)
is_default: bool = False
class StoreUpdateRequest(_CamelModel):
"""Partial update — every field is optional. Use `slug` to rename."""
name: str | None = None
slug: str | None = None
kind: str | None = None
embedder: str | None = None
config: dict | None = None
is_default: bool | None = None
class StoreDocEntryResponse(_CamelModel):
doc_id: str
filename: str
state: str
chunk_count: int
pushed_at: str | None = None
# ---------------------------------------------------------------------------
# Doc-centric chunks (#256) — canonical chunkset, distinct from analysis chunks
# ---------------------------------------------------------------------------
class DocChunkResponse(_CamelModel):
"""Canonical doc chunk — wire shape consumed by `features/chunks` on the front."""
id: str
doc_id: str
sequence: int
text: str
headings: list[str] = []
source_page: int | None = None
token_count: int | None = None
created_at: str | datetime
updated_at: str | datetime
class AddChunkRequest(_CamelModel):
text: str
after_id: str | None = None
class UpdateChunkRequest(_CamelModel):
"""Either or both fields. Empty body is a 400 — handled in the router."""
text: str | None = None
title: str | None = None # surfaced as first heading; future: dedicated field
class SplitChunkRequest(_CamelModel):
cursor_offset: int
class MergeChunksRequest(_CamelModel):
ids: list[str]
class DocRechunkRequest(_CamelModel):
"""Optional chunking options. Empty body uses defaults."""
chunking_options: ChunkingOptionsRequest | None = None
class DocTreeNodeResponse(_CamelModel):
ref: str
type: str
label: str
children: list[DocTreeNodeResponse] = []
# Forward-ref resolution (children references the same class).
DocTreeNodeResponse.model_rebuild()
class ChunkDiffResponse(_CamelModel):
chunk_id: str
status: str # 'added' | 'modified' | 'removed' | 'unchanged'
text_diff: str | None = None
class PushSummaryResponse(_CamelModel):
embeds: int
tokens: int
class PushChunksResponse(_CamelModel):
job_id: str
summary: PushSummaryResponse
class PushChunksRequest(_CamelModel):
store: str

View file

@ -0,0 +1,170 @@
"""Stores API router — CRUD on ingestion targets (#251)."""
from __future__ import annotations
import logging
from typing import Annotated
from fastapi import APIRouter, Depends, HTTPException, Request, Response
from api.schemas import (
StoreCreateRequest,
StoreDocEntryResponse,
StoreInfoResponse,
StoreResponse,
StoreUpdateRequest,
)
from domain.value_objects import StoreKind
from services.store_service import (
StoreService,
StoreServiceError,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/stores", tags=["stores"])
def _get_service(request: Request) -> StoreService:
return request.app.state.store_service
ServiceDep = Annotated[StoreService, Depends(_get_service)]
def _parse_kind(value: str) -> StoreKind:
try:
return StoreKind(value)
except ValueError as exc:
valid = ", ".join(k.value for k in StoreKind)
raise HTTPException(
status_code=422,
detail=f"Unknown store kind '{value}'. Valid kinds: {valid}.",
) from exc
def _store_to_response(store) -> StoreResponse:
return StoreResponse(
id=store.id,
name=store.name,
slug=store.slug,
kind=store.kind.value,
embedder=store.embedder,
is_default=store.is_default,
config=store.config,
created_at=str(store.created_at),
)
def _info_to_response(view) -> StoreInfoResponse:
return StoreInfoResponse(
name=view.name,
slug=view.slug,
type=view.kind,
embedder=view.embedder,
is_default=view.is_default,
document_count=view.document_count,
chunk_count=view.chunk_count,
connected=view.connected,
error_message=view.error_message,
)
def _doc_entry_to_response(entry) -> StoreDocEntryResponse:
return StoreDocEntryResponse(
doc_id=entry.doc_id,
filename=entry.filename,
state=entry.state,
chunk_count=entry.chunk_count,
pushed_at=entry.pushed_at,
)
@router.get("", response_model=list[StoreInfoResponse])
async def list_stores(service: ServiceDep) -> list[StoreInfoResponse]:
views = await service.list_stores()
return [_info_to_response(v) for v in views]
@router.post("", response_model=StoreResponse, status_code=201)
async def create_store(
payload: StoreCreateRequest,
service: ServiceDep,
) -> StoreResponse:
kind = _parse_kind(payload.kind)
try:
store = await service.create_store(
name=payload.name,
slug=payload.slug,
kind=kind,
embedder=payload.embedder,
config=payload.config,
is_default=payload.is_default,
)
except StoreServiceError as exc:
raise HTTPException(status_code=exc.http_status, detail=str(exc)) from exc
return _store_to_response(store)
@router.get("/{slug}", response_model=StoreResponse)
async def get_store(slug: str, service: ServiceDep) -> StoreResponse:
try:
store = await service.get_by_slug(slug)
except StoreServiceError as exc:
raise HTTPException(status_code=exc.http_status, detail=str(exc)) from exc
return _store_to_response(store)
@router.patch("/{slug}", response_model=StoreResponse)
async def update_store(
slug: str,
payload: StoreUpdateRequest,
service: ServiceDep,
) -> StoreResponse:
kind = _parse_kind(payload.kind) if payload.kind is not None else None
try:
store = await service.update_store(
slug,
name=payload.name,
new_slug=payload.slug,
kind=kind,
embedder=payload.embedder,
config=payload.config,
is_default=payload.is_default,
)
except StoreServiceError as exc:
raise HTTPException(status_code=exc.http_status, detail=str(exc)) from exc
return _store_to_response(store)
@router.delete("/{slug}", status_code=204)
async def delete_store(slug: str, service: ServiceDep) -> Response:
try:
await service.delete_store(slug)
except StoreServiceError as exc:
raise HTTPException(status_code=exc.http_status, detail=str(exc)) from exc
return Response(status_code=204)
@router.get("/{slug}/documents", response_model=list[StoreDocEntryResponse])
async def list_store_documents(
slug: str,
service: ServiceDep,
) -> list[StoreDocEntryResponse]:
try:
entries = await service.list_documents(slug)
except StoreServiceError as exc:
raise HTTPException(status_code=exc.http_status, detail=str(exc)) from exc
return [_doc_entry_to_response(e) for e in entries]
@router.delete("/{slug}/documents/{doc_id}", status_code=204)
async def remove_store_document(
slug: str,
doc_id: str,
service: ServiceDep,
) -> Response:
try:
await service.remove_document(slug, doc_id)
except StoreServiceError as exc:
raise HTTPException(status_code=exc.http_status, detail=str(exc)) from exc
return Response(status_code=204)

View file

@ -0,0 +1,216 @@
"""Pure-domain operations on a chunkset.
These functions take a chunkset as input and return a new chunkset
(plus, where appropriate, the chunks that were created). They do not
touch the database, do not record audit rows, and do not raise
infrastructure errors. The `ChunkEditingService` (in `services/`)
wraps each call with audit-record generation and atomic persistence.
All operations preserve `sequence` ordering: insertions and splits
shift subsequent sequences upward; deletions / merges leave gaps.
Sequences are 0-based and only required to be strictly increasing
within a document; gaps are explicitly allowed so we never rewrite
many rows for a single edit.
"""
from __future__ import annotations
import uuid
from datetime import UTC, datetime
from domain.models import Chunk
class ChunkEditingError(Exception):
"""Raised by chunk-editing operations on invalid input (missing id,
out-of-range offset, etc.). Subclasses `Exception` rather than
`DomainError` so the API layer can map all of them to 4xx without
a wider catch."""
def _utcnow() -> datetime:
return datetime.now(UTC)
def _new_id() -> str:
return uuid.uuid4().hex
def _index_of(chunks: list[Chunk], chunk_id: str) -> int:
for idx, c in enumerate(chunks):
if c.id == chunk_id:
return idx
raise ChunkEditingError(f"chunk not found: {chunk_id}")
def insert(
chunks: list[Chunk],
*,
at_position: int,
text: str,
document_id: str,
headings: list[str] | None = None,
source_page: int | None = None,
) -> tuple[list[Chunk], Chunk]:
"""Insert a fresh chunk at position `at_position`.
Returns the updated chunkset and the new chunk. Subsequent chunks'
sequences are shifted by +1.
"""
if at_position < 0 or at_position > len(chunks):
raise ChunkEditingError(f"insert position out of range: {at_position}")
now = _utcnow()
new_chunk = Chunk(
id=_new_id(),
document_id=document_id,
sequence=at_position,
text=text,
headings=list(headings or []),
source_page=source_page,
created_at=now,
updated_at=now,
)
out = list(chunks)
for c in out[at_position:]:
c.sequence += 1
out.insert(at_position, new_chunk)
return out, new_chunk
def update(
chunks: list[Chunk],
chunk_id: str,
*,
text: str | None = None,
headings: list[str] | None = None,
) -> tuple[list[Chunk], Chunk]:
"""Update text and/or headings of a chunk in-place.
Returns the updated chunkset and the modified chunk. The chunk's
id and sequence are preserved.
"""
idx = _index_of(chunks, chunk_id)
target = chunks[idx]
if target.deleted_at is not None:
raise ChunkEditingError(f"chunk is deleted: {chunk_id}")
if text is not None:
target.text = text
if headings is not None:
target.headings = list(headings)
target.updated_at = _utcnow()
return list(chunks), target
def delete(chunks: list[Chunk], chunk_id: str) -> tuple[list[Chunk], Chunk]:
"""Soft-delete a chunk. Returns the updated chunkset and the
deleted chunk (still present in the list, with `deleted_at` set)."""
idx = _index_of(chunks, chunk_id)
target = chunks[idx]
if target.deleted_at is not None:
return list(chunks), target # idempotent
target.deleted_at = _utcnow()
target.updated_at = target.deleted_at
return list(chunks), target
def merge(
chunks: list[Chunk],
chunk_ids: list[str],
*,
separator: str = "\n",
) -> tuple[list[Chunk], Chunk]:
"""Merge `chunk_ids` (in order) into a single new chunk.
The new chunk takes the headings of the first source and the
smallest source page. The `id`s of the sources are returned in
the new chunk's lineage via the audit row written by the service
layer.
Returns the updated chunkset and the new merged chunk. The sources
are removed from the chunkset (hard-removed from the list the
service is responsible for soft-deleting their persisted rows so
history queries still resolve them).
"""
if len(chunk_ids) < 2:
raise ChunkEditingError("merge requires at least two chunks")
indices = [_index_of(chunks, cid) for cid in chunk_ids]
sources = [chunks[i] for i in indices]
if any(c.deleted_at for c in sources):
raise ChunkEditingError("cannot merge a deleted chunk")
document_id = sources[0].document_id
if any(c.document_id != document_id for c in sources):
raise ChunkEditingError("merge across documents is not allowed")
merged_text = separator.join(c.text for c in sources)
merged_headings = list(sources[0].headings)
merged_page = min((c.source_page for c in sources if c.source_page is not None), default=None)
merged_sequence = min(c.sequence for c in sources)
now = _utcnow()
new_chunk = Chunk(
id=_new_id(),
document_id=document_id,
sequence=merged_sequence,
text=merged_text,
headings=merged_headings,
source_page=merged_page,
created_at=now,
updated_at=now,
)
source_ids = {c.id for c in sources}
out = [c for c in chunks if c.id not in source_ids]
out.append(new_chunk)
out.sort(key=lambda c: c.sequence)
return out, new_chunk
def split(
chunks: list[Chunk], chunk_id: str, *, at_offset: int
) -> tuple[list[Chunk], Chunk, Chunk]:
"""Split a chunk's text at `at_offset` into two new chunks.
The source chunk is removed from the chunkset (the service-layer
persistence path soft-deletes its row so history queries can
resolve it). The two new chunks inherit headings and source_page
from the source.
Returns the updated chunkset and the two new chunks `(left, right)`.
"""
idx = _index_of(chunks, chunk_id)
target = chunks[idx]
if target.deleted_at is not None:
raise ChunkEditingError(f"chunk is deleted: {chunk_id}")
if at_offset <= 0 or at_offset >= len(target.text):
raise ChunkEditingError(f"split offset out of range for chunk of length {len(target.text)}")
now = _utcnow()
left = Chunk(
id=_new_id(),
document_id=target.document_id,
sequence=target.sequence,
text=target.text[:at_offset],
headings=list(target.headings),
source_page=target.source_page,
created_at=now,
updated_at=now,
)
right = Chunk(
id=_new_id(),
document_id=target.document_id,
sequence=target.sequence + 1,
text=target.text[at_offset:],
headings=list(target.headings),
source_page=target.source_page,
created_at=now,
updated_at=now,
)
out = [c for c in chunks if c.id != target.id]
for c in out:
if c.sequence > target.sequence:
c.sequence += 1
out.extend((left, right))
out.sort(key=lambda c: c.sequence)
return out, left, right

View file

@ -0,0 +1,35 @@
"""Domain events — frozen records that document state transitions.
Events are produced by domain operations (typically returned from a
mutation method on an aggregate). They are pure data no event bus is
wired in 0.6.0; services can choose to log, persist, or publish them
later. Keeping them here keeps the domain layer free of any event-bus
infrastructure.
"""
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from datetime import datetime
from domain.value_objects import DocumentLifecycleState
@dataclass(frozen=True)
class DocumentLifecycleChanged:
"""A Document lifecycle transition occurred.
Attributes:
document_id: id of the document that transitioned.
previous: state the document was in before the transition.
current: state the document is in after the transition.
at: timestamp of the transition (UTC).
"""
document_id: str
previous: DocumentLifecycleState
current: DocumentLifecycleState
at: datetime

View file

@ -0,0 +1,37 @@
"""Domain-level exceptions.
Exceptions defined in this module are raised by domain operations and value
objects when an invariant is violated. They have no infrastructure
dependencies and are safe to import from any layer.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from domain.value_objects import DocumentLifecycleState
class DomainError(Exception):
"""Base class for domain-level errors. Catch this when wiring API
layers if you want a single hook for any invariant violation."""
class InvalidLifecycleTransitionError(DomainError):
"""Raised when a Document.transition_to() call asks for a (source,
target) pair that is not in the allowed transition table.
Carries `source` and `target` so callers can produce a useful error
message without re-discovering them.
"""
def __init__(
self,
*,
source: DocumentLifecycleState,
target: DocumentLifecycleState,
) -> None:
super().__init__(f"Invalid document lifecycle transition: {source.value} -> {target.value}")
self.source = source
self.target = target

View file

@ -0,0 +1,67 @@
"""Deterministic hashing for chunksets — substrate for auto-stale detection (#204).
A `chunkset_hash` summarises the content of a list of chunks for a
document. The hash is recorded on each `DocumentStoreLink` at push time
(#203 ships the column slot). When chunks change, recomputing the hash
and comparing against the stored value tells us whether the link has
gone stale.
Why a hash and not, say, an updated_at?
- Idempotent re-pipelines on identical input bump `updated_at` without
semantic change. A content hash is the only signal that survives
that.
- It is also content-addressed: two different docs that happen to have
the same chunkset get the same hash. Useful for de-duplication
further down the road.
Inputs and exclusions are pinned. Any change to the canonical inputs
re-flips every existing link to Stale once that is a deliberate
release-note event, not a silent migration.
This module is pure: in / out. No I/O. No randomness. No dates.
"""
from __future__ import annotations
import hashlib
import json
from typing import TYPE_CHECKING
if TYPE_CHECKING:
from collections.abc import Iterable
from domain.value_objects import ChunkResult
# Byte separator inserted between chunks so concatenating two chunks does
# not yield the same hash as a single chunk with the joined text. \x1f is
# the Unicode "Information Separator One" — semantically appropriate and
# safe inside arbitrary text.
_CHUNK_SEPARATOR = b"\x1f"
def chunkset_hash(chunks: Iterable[ChunkResult]) -> str:
"""Return a deterministic SHA-256 hex digest over a chunkset.
Hashed inputs (per chunk, in order):
- text (str)
- source_page (int | None)
- headings (list[str], order preserved)
Excluded:
- bboxes / doc_items (rendering artefacts; do not affect retrieval)
- token_count (derived; unstable across tokenizers)
The exclusion list is intentional. Bumping it changes every link's
hash and triggers a one-time corpus-wide flip to `Stale`.
"""
h = hashlib.sha256()
for chunk in chunks:
payload = {
"t": chunk.text,
"p": chunk.source_page,
"h": list(chunk.headings or []),
}
h.update(_CHUNK_SEPARATOR)
h.update(json.dumps(payload, ensure_ascii=False, separators=(",", ":")).encode())
return h.hexdigest()

View file

@ -0,0 +1,83 @@
"""Document lifecycle state machine — pure domain logic.
Defines the canonical state model for a Document in Docling Studio:
Uploaded Parsed Chunked Ingested (Stale|Chunked) Ingested
Stale is set by the auto-detect logic (#204) — never reached by a manual
action. Failed is reachable from any state and represents a pipeline error.
This module contains:
- The transition table (which (from to) pairs are allowed).
- `is_allowed_transition`: the pure check.
- `assert_transition`: raises `InvalidLifecycleTransitionError` when not allowed.
The dataclass that carries the state lives in `domain.models.Document`. The
domain event emitted on transition lives in `domain.events`.
"""
from __future__ import annotations
from domain.exceptions import InvalidLifecycleTransitionError
from domain.value_objects import DocumentLifecycleState
# Allowed transitions: from → set of allowed targets.
# Failed is always reachable as a terminal-ish state (see notes below).
# Self-loops are allowed only where they make pipeline sense (e.g. re-chunk).
_TRANSITIONS: dict[DocumentLifecycleState, frozenset[DocumentLifecycleState]] = {
DocumentLifecycleState.UPLOADED: frozenset(
{
DocumentLifecycleState.PARSED,
DocumentLifecycleState.CHUNKED, # parse + chunk in one pipeline call
DocumentLifecycleState.FAILED,
}
),
DocumentLifecycleState.PARSED: frozenset(
{
DocumentLifecycleState.PARSED, # idempotent re-parse
DocumentLifecycleState.CHUNKED,
DocumentLifecycleState.FAILED,
}
),
DocumentLifecycleState.CHUNKED: frozenset(
{
DocumentLifecycleState.CHUNKED, # re-chunk
DocumentLifecycleState.INGESTED,
DocumentLifecycleState.FAILED,
}
),
DocumentLifecycleState.INGESTED: frozenset(
{
DocumentLifecycleState.STALE,
DocumentLifecycleState.CHUNKED, # re-chunk after ingest
DocumentLifecycleState.INGESTED, # re-push (idempotent)
DocumentLifecycleState.FAILED,
}
),
DocumentLifecycleState.STALE: frozenset(
{
DocumentLifecycleState.INGESTED,
DocumentLifecycleState.CHUNKED,
DocumentLifecycleState.FAILED,
}
),
# Failed is a recoverable state — the operator (or a retry) can move
# back to a non-terminal state by re-running the relevant pipeline step.
DocumentLifecycleState.FAILED: frozenset(
{
DocumentLifecycleState.UPLOADED,
DocumentLifecycleState.PARSED,
DocumentLifecycleState.CHUNKED,
}
),
}
def is_allowed_transition(source: DocumentLifecycleState, target: DocumentLifecycleState) -> bool:
"""Return True iff transitioning from `source` to `target` is allowed."""
return target in _TRANSITIONS.get(source, frozenset())
def assert_transition(source: DocumentLifecycleState, target: DocumentLifecycleState) -> None:
"""Raise `InvalidLifecycleTransitionError` if the transition is not allowed."""
if not is_allowed_transition(source, target):
raise InvalidLifecycleTransitionError(source=source, target=target)

View file

@ -0,0 +1,58 @@
"""Aggregate the per-(document, store) link states into a single
document-level lifecycle state.
The doc lifecycle column is the materialized result of this rule. It is
recomputed any time a link write happens. Read paths use the stored
column directly (cheap) they do not call this rule on every GET.
The rule prefers "more concerning" states first:
any link FAILED -> Document FAILED
any link STALE -> Document STALE
any link INGESTED -> Document INGESTED
no links -> keep the document's current pre-link state
(Uploaded / Parsed / Chunked) the caller is
responsible for not overwriting that.
If you find yourself wanting a fourth case, you probably want a new
link state, not a new aggregation branch.
This module is pure: no I/O, no datetime just data in / data out.
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from domain.value_objects import DocumentLifecycleState, DocumentStoreLinkState
if TYPE_CHECKING:
from collections.abc import Iterable
from domain.models import DocumentStoreLink
def aggregate_lifecycle(
links: Iterable[DocumentStoreLink],
*,
fallback: DocumentLifecycleState,
) -> DocumentLifecycleState:
"""Compute the aggregate document lifecycle state.
Args:
links: every `DocumentStoreLink` for the document. May be empty.
fallback: the lifecycle state to return when there are no links
typically the document's current pre-link state (`Uploaded`,
`Parsed`, or `Chunked`).
Returns:
The aggregate `DocumentLifecycleState`.
"""
states = {link.state for link in links}
if DocumentStoreLinkState.FAILED in states:
return DocumentLifecycleState.FAILED
if DocumentStoreLinkState.STALE in states:
return DocumentLifecycleState.STALE
if DocumentStoreLinkState.INGESTED in states:
return DocumentLifecycleState.INGESTED
return fallback

View file

@ -7,6 +7,17 @@ import uuid
from dataclasses import dataclass, field
from datetime import UTC, datetime
from domain.events import DocumentLifecycleChanged
from domain.lifecycle import assert_transition
from domain.value_objects import (
ChunkBbox,
ChunkDocItem,
ChunkEditAction,
DocumentLifecycleState,
DocumentStoreLinkState,
StoreKind,
)
class AnalysisStatus(enum.StrEnum):
PENDING = "PENDING"
@ -32,6 +43,36 @@ class Document:
page_count: int | None = None
storage_path: str = ""
created_at: datetime = field(default_factory=_utcnow)
lifecycle_state: DocumentLifecycleState = DocumentLifecycleState.UPLOADED
lifecycle_state_at: datetime | None = None
def transition_to(
self,
target: DocumentLifecycleState,
*,
now: datetime | None = None,
) -> DocumentLifecycleChanged:
"""Move the document to `target`, validating the transition.
Returns the corresponding `DocumentLifecycleChanged` event so the
caller (typically a service) can log / persist / publish it. The
event is pure data no event bus is wired in 0.6.0.
Raises:
InvalidLifecycleTransitionError: if (current target) is not in
the allowed transition table.
"""
previous = self.lifecycle_state
assert_transition(previous, target)
at = now or _utcnow()
self.lifecycle_state = target
self.lifecycle_state_at = at
return DocumentLifecycleChanged(
document_id=self.id,
previous=previous,
current=target,
at=at,
)
@dataclass
@ -96,3 +137,144 @@ class AnalysisJob:
self.status = AnalysisStatus.FAILED
self.error_message = error
self.completed_at = _utcnow()
@dataclass
class Store:
"""A logical destination for ingested chunks.
A `Store` represents a named, configurable target for ingestion (e.g.
`rh-corpus-v3`, `legal-v1`). It is decoupled from the underlying
`VectorStore` adapter (which represents the *technology*, like
OpenSearch). One adapter can serve many stores by namespacing the
physical index name on `slug`.
See design doc `docs/design/203-per-store-ingestion-state.md`.
"""
id: str = field(default_factory=_new_id)
name: str = ""
slug: str = ""
kind: StoreKind = StoreKind.OPENSEARCH
embedder: str = ""
config: dict = field(default_factory=dict)
is_default: bool = False
created_at: datetime = field(default_factory=_utcnow)
@dataclass
class DocumentStoreLink:
"""A live record of a document's presence in a single store.
The link carries the per-pair state (Ingested / Stale / Failed) plus
metadata used by the auto-stale detection (#204) and the chunks
editor (#205). One row per (document, store) pair — enforced by a
UNIQUE constraint at the persistence layer.
"""
id: str = field(default_factory=_new_id)
document_id: str = ""
store_id: str = ""
state: DocumentStoreLinkState = DocumentStoreLinkState.INGESTED
chunkset_hash: str | None = None
last_push_at: datetime | None = None
last_run_id: str | None = None
error_message: str | None = None
def mark_ingested(
self,
*,
hash_: str,
at: datetime,
run_id: str | None = None,
) -> None:
"""Record a successful push: chunkset hash + timestamp + run id."""
self.state = DocumentStoreLinkState.INGESTED
self.chunkset_hash = hash_
self.last_push_at = at
self.last_run_id = run_id
self.error_message = None
def mark_stale(self) -> None:
"""Mark the link as stale (source chunkset drifted from pushed)."""
self.state = DocumentStoreLinkState.STALE
self.error_message = None
def mark_failed(self, *, error: str) -> None:
"""Record a failed push attempt."""
self.state = DocumentStoreLinkState.FAILED
self.error_message = error
@dataclass
class Chunk:
"""A persisted chunk — first-class entity introduced by #205.
Replaces the legacy `analysis_jobs.chunks_json` blob. The `id` is
stable across edits except for split/merge which produce new chunks
with new ids; the lineage is recorded in `chunk_edits` rows.
`sequence` controls ordering within a document. Gaps are allowed
splits push subsequent chunks' sequences without rewriting them.
`deleted_at` is non-null when the chunk has been soft-deleted; the
row stays in the table so the audit trail keeps its before/after
pointers valid.
"""
id: str = field(default_factory=_new_id)
document_id: str = ""
sequence: int = 0
text: str = ""
headings: list[str] = field(default_factory=list)
source_page: int | None = None
bboxes: list[ChunkBbox] = field(default_factory=list)
doc_items: list[ChunkDocItem] = field(default_factory=list)
token_count: int | None = None
created_at: datetime = field(default_factory=_utcnow)
updated_at: datetime = field(default_factory=_utcnow)
deleted_at: datetime | None = None
@dataclass(frozen=True)
class ChunkEdit:
"""An immutable audit record describing one mutating operation on
the chunkset of a document. Written atomically with the chunk
write it describes.
`before` and `after` are JSON-serializable snapshots of the chunk
state. They are `None` for the start/end of the chunk's life:
- `before is None` for INSERT
- `after is None` for DELETE
- For MERGE: a single result row carries `parents = [chunk_ids]`
- For SPLIT: two result rows each carry `parents = [source_id]`
and the source's edit row carries `children = [new_id, new_id]`
"""
id: str
document_id: str
chunk_id: str | None
action: ChunkEditAction
actor: str
at: datetime
before: dict | None = None
after: dict | None = None
parents: list[str] = field(default_factory=list)
children: list[str] = field(default_factory=list)
reason: str | None = None
@dataclass(frozen=True)
class ChunkPush:
"""Snapshot of which chunks were pushed to which store, when.
Lets the API answer 'show me the chunkset that was in store X at
push N' without replaying the full audit log.
"""
id: str
document_id: str
store_id: str
chunkset_hash: str
chunk_ids: list[str]
pushed_at: datetime

View file

@ -9,12 +9,23 @@ from __future__ import annotations
from typing import TYPE_CHECKING, Protocol, runtime_checkable
if TYPE_CHECKING:
from domain.models import AnalysisJob, Document
from datetime import datetime
from domain.models import (
AnalysisJob,
Chunk,
ChunkEdit,
ChunkPush,
Document,
DocumentStoreLink,
Store,
)
from domain.value_objects import (
ChunkingOptions,
ChunkResult,
ConversionOptions,
ConversionResult,
DocumentLifecycleState,
LLMProviderType,
ReasoningResult,
)
@ -85,9 +96,95 @@ class DocumentRepository(Protocol):
async def update_page_count(self, doc_id: str, page_count: int) -> None: ...
async def update_lifecycle(
self,
doc_id: str,
state: DocumentLifecycleState,
at: datetime,
) -> None: ...
async def delete(self, doc_id: str) -> bool: ...
class StoreRepository(Protocol):
"""Port for `Store` persistence (introduced by #203)."""
async def insert(self, store: Store) -> None: ...
async def find_all(self) -> list[Store]: ...
async def find_by_slug(self, slug: str) -> Store | None: ...
async def find_by_id(self, store_id: str) -> Store | None: ...
async def get_default(self) -> Store | None: ...
class DocumentStoreLinkRepository(Protocol):
"""Port for `DocumentStoreLink` persistence (introduced by #203)."""
async def upsert(self, link: DocumentStoreLink) -> None:
"""Insert or update by (document_id, store_id)."""
...
async def find_for_document(self, document_id: str) -> list[DocumentStoreLink]: ...
async def find_for_store(self, store_id: str) -> list[DocumentStoreLink]: ...
async def find_one(self, document_id: str, store_id: str) -> DocumentStoreLink | None: ...
async def delete(self, document_id: str, store_id: str) -> bool: ...
class ChunkRepository(Protocol):
"""Port for first-class chunk persistence (introduced by #205)."""
async def insert(self, chunk: Chunk) -> None: ...
async def insert_many(self, chunks: list[Chunk]) -> None: ...
async def update(self, chunk: Chunk) -> None: ...
async def soft_delete(self, chunk_id: str, *, at: datetime) -> bool: ...
async def find_for_document(
self,
document_id: str,
*,
include_deleted: bool = False,
) -> list[Chunk]: ...
async def find_by_id(self, chunk_id: str) -> Chunk | None: ...
async def count_for_document(self, document_id: str) -> int: ...
class ChunkEditRepository(Protocol):
"""Port for the immutable chunk_edits audit log (introduced by #205)."""
async def insert(self, edit: ChunkEdit) -> None: ...
async def find_for_document(
self,
document_id: str,
*,
limit: int = 50,
offset: int = 0,
) -> list[ChunkEdit]: ...
async def find_for_chunk(self, chunk_id: str) -> list[ChunkEdit]: ...
class ChunkPushRepository(Protocol):
"""Port for chunk_pushes snapshots (introduced by #205)."""
async def insert(self, push: ChunkPush) -> None: ...
async def find_by_id(self, push_id: str) -> ChunkPush | None: ...
async def find_latest(self, document_id: str, store_id: str) -> ChunkPush | None: ...
class AnalysisRepository(Protocol):
"""Port for analysis job persistence."""
@ -97,6 +194,8 @@ class AnalysisRepository(Protocol):
async def find_by_id(self, job_id: str) -> AnalysisJob | None: ...
async def find_latest_completed_by_document(self, document_id: str) -> AnalysisJob | None: ...
async def update_status(self, job: AnalysisJob) -> None: ...
async def update_progress(self, job_id: str, current: int, total: int) -> None: ...

View file

@ -14,6 +14,67 @@ DEFAULT_PAGE_WIDTH: float = 612.0
DEFAULT_PAGE_HEIGHT: float = 792.0
class DocumentLifecycleState(StrEnum):
"""Canonical lifecycle of a Document in Docling Studio.
Distinct from `AnalysisStatus` (which describes a single conversion
attempt). The lifecycle describes the document as a whole:
Uploaded raw file persisted, no parse yet
Parsed conversion produced a document tree
Chunked chunker produced a draft chunkset (pre-store)
Ingested chunkset has been embedded into at least one store
Stale a chunkset was edited after a successful push and the
corresponding store no longer matches (#204)
Failed a pipeline step failed; recoverable by retry
Allowed transitions live in `domain.lifecycle._TRANSITIONS`.
"""
UPLOADED = "Uploaded"
PARSED = "Parsed"
CHUNKED = "Chunked"
INGESTED = "Ingested"
STALE = "Stale"
FAILED = "Failed"
class StoreKind(StrEnum):
"""Backing technology of a Store. New backends plug in here without
touching the persistence schema."""
OPENSEARCH = "opensearch"
NEO4J = "neo4j"
class DocumentStoreLinkState(StrEnum):
"""State of a (document, store) ingestion link.
Distinct from `DocumentLifecycleState` the document lifecycle is the
aggregate over all per-store links. A link is `Ingested` when its
chunkset hash matches the source; `Stale` when the source has drifted
after the last push; `Failed` when the last push attempt errored.
"""
INGESTED = "Ingested"
STALE = "Stale"
FAILED = "Failed"
class ChunkEditAction(StrEnum):
"""The five mutating operations the chunks editor supports.
Recorded on every `ChunkEdit` row so the audit trail can answer "who
did what, when, and why" without resorting to JSON-path matching.
"""
INSERT = "insert"
UPDATE = "update"
DELETE = "delete"
MERGE = "merge"
SPLIT = "split"
@dataclass(frozen=True)
class PageElement:
type: str

View file

@ -55,6 +55,18 @@ class Settings:
cors_origins: list[str] = field(
default_factory=lambda: ["http://localhost:3000", "http://localhost:5173"]
)
# 0.6.1 — Surface flags (#257). Two master flags select which UI surface
# is exposed: STUDIO_MODE_ENABLED (legacy OCR-debug) and
# RAG_PIPELINE_ENABLED (new doc-centric ingestion + visualization).
# At least one must be enabled. Sub-flags below gate individual modes
# inside the RAG pipeline surface.
studio_mode_enabled: bool = False
rag_pipeline_enabled: bool = True
# 0.6.0 — Doc workspace mode flags (#210, renamed in #257).
# Sub-flags effective only when rag_pipeline_enabled is true.
inspect_mode_enabled: bool = True
linked_mode_enabled: bool = True
ask_mode_enabled: bool = True
def __post_init__(self) -> None:
errors: list[str] = []
@ -90,6 +102,8 @@ class Settings:
)
if self.embedding_dimension < 1:
errors.append(f"embedding_dimension must be >= 1 (got {self.embedding_dimension})")
if not (self.studio_mode_enabled or self.rag_pipeline_enabled):
errors.append("at least one of STUDIO_MODE_ENABLED / RAG_PIPELINE_ENABLED must be true")
if self.default_table_mode not in ("accurate", "fast"):
errors.append(
f"default_table_mode must be 'accurate' or 'fast' (got '{self.default_table_mode}')"
@ -153,6 +167,18 @@ class Settings:
max_paste_image_size_mb=int(os.environ.get("MAX_PASTE_IMAGE_SIZE_MB", "10")),
paste_allowed_image_types=[t.strip() for t in paste_types_raw.split(",") if t.strip()],
cors_origins=[o.strip() for o in cors_raw.split(",")],
# 0.6.1 — Surface flags (#257).
studio_mode_enabled=os.environ.get("STUDIO_MODE_ENABLED", "false").lower()
in ("1", "true", "yes", "on"),
rag_pipeline_enabled=os.environ.get("RAG_PIPELINE_ENABLED", "true").lower()
in ("1", "true", "yes", "on"),
# 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257).
inspect_mode_enabled=os.environ.get("INSPECT_MODE_ENABLED", "true").lower()
in ("1", "true", "yes", "on"),
linked_mode_enabled=os.environ.get("LINKED_MODE_ENABLED", "true").lower()
in ("1", "true", "yes", "on"),
ask_mode_enabled=os.environ.get("ASK_MODE_ENABLED", "true").lower()
in ("1", "true", "yes", "on"),
)

View file

@ -19,17 +19,25 @@ from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from api.analyses import router as analyses_router
from api.document_chunks import router as document_chunks_router
from api.documents import router as documents_router
from api.ingestion import router as ingestion_router
from api.schemas import HealthResponse
from api.stores import router as stores_router
from infra.rate_limiter import RateLimiterMiddleware
from infra.settings import settings
from persistence.analysis_repo import SqliteAnalysisRepository
from persistence.chunk_edit_repo import SqliteChunkEditRepository, SqliteChunkPushRepository
from persistence.chunk_repo import SqliteChunkRepository
from persistence.database import get_connection, init_db
from persistence.document_repo import SqliteDocumentRepository
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from persistence.store_repo import SqliteStoreRepository
from services.analysis_service import AnalysisConfig, AnalysisService
from services.chunk_service import ChunkService
from services.document_service import DocumentConfig, DocumentService
from services.ingestion_service import IngestionConfig, IngestionService
from services.store_service import StoreService
logging.basicConfig(
level=logging.INFO,
@ -187,11 +195,40 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
document_repo, analysis_repo, neo4j_driver=app.state.neo4j
)
app.state.document_service = _build_document_service(document_repo, analysis_repo)
store_repo = SqliteStoreRepository()
link_repo = SqliteDocumentStoreLinkRepository()
app.state.store_repo = store_repo
app.state.document_store_link_repo = link_repo
app.state.store_service = StoreService(
store_repo=store_repo,
link_repo=link_repo,
document_repo=document_repo,
)
ingestion_service = _build_ingestion_service(neo4j_driver=app.state.neo4j)
app.state.ingestion_service = ingestion_service
if ingestion_service is not None:
app.include_router(ingestion_router)
logger.info("Ingestion router mounted")
# Doc-centric chunks (#256). Wires the canonical chunkset CRUD on top
# of the chunk / chunk_edit / chunk_push repos introduced by #205.
chunk_repo = SqliteChunkRepository()
chunk_edit_repo = SqliteChunkEditRepository()
chunk_push_repo = SqliteChunkPushRepository()
app.state.chunk_repo = chunk_repo
app.state.chunk_service = ChunkService(
chunk_repo=chunk_repo,
chunk_edit_repo=chunk_edit_repo,
chunk_push_repo=chunk_push_repo,
document_repo=document_repo,
analysis_repo=analysis_repo,
chunker=_build_chunker(),
ingestion_service=ingestion_service,
)
# The analysis service promotes the first analysis's chunks into the
# canonical chunkset (idempotent), so the doc workspace lights up the
# moment a doc is parsed for the first time.
app.state.analysis_service.set_chunk_promoter(app.state.chunk_service)
logger.info("Docling Studio backend ready (engine=%s)", settings.conversion_engine)
try:
yield
@ -223,7 +260,9 @@ if settings.rate_limit_rpm > 0:
)
app.include_router(documents_router)
app.include_router(document_chunks_router)
app.include_router(analyses_router)
app.include_router(stores_router)
# Graph view — mounted regardless; individual requests 503 if Neo4j is absent.
from api.graph import router as graph_router # noqa: E402
@ -304,4 +343,11 @@ async def health() -> HealthResponse:
# actual Ollama reachability is checked lazily at call-time to avoid
# blocking health checks on the LLM host.
reasoning_available=runner is not None and runner.is_available,
# 0.6.1 — Surface flags (#257).
studio_mode_enabled=settings.studio_mode_enabled,
rag_pipeline_enabled=settings.rag_pipeline_enabled,
# 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257).
inspect_mode_enabled=settings.inspect_mode_enabled,
linked_mode_enabled=settings.linked_mode_enabled,
ask_mode_enabled=settings.ask_mode_enabled,
)

View file

@ -66,6 +66,19 @@ class SqliteAnalysisRepository:
rows = await cursor.fetchall()
return [_row_to_job(r) for r in rows]
async def find_by_document(
self, document_id: str, *, limit: int = 200, offset: int = 0
) -> list[AnalysisJob]:
"""Return analysis jobs for a given document, newest first."""
async with get_connection() as db:
cursor = await db.execute(
f"{_SELECT_WITH_DOC} WHERE aj.document_id = ? "
"ORDER BY aj.created_at DESC LIMIT ? OFFSET ?",
(document_id, limit, offset),
)
rows = await cursor.fetchall()
return [_row_to_job(r) for r in rows]
async def find_by_id(self, job_id: str) -> AnalysisJob | None:
"""Find an analysis job by ID (with document filename), or return None."""
async with get_connection() as db:

View file

@ -0,0 +1,145 @@
"""Chunk edit / push repositories — immutable audit log + push snapshots (#205)."""
from __future__ import annotations
import json
from datetime import UTC, datetime
from domain.models import ChunkEdit, ChunkPush
from domain.value_objects import ChunkEditAction
from persistence.database import get_connection
def _parse_iso(value: str | None) -> datetime | None:
if value is None or value == "":
return None
parsed = datetime.fromisoformat(value)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed
def _row_to_edit(row) -> ChunkEdit:
return ChunkEdit(
id=row["id"],
document_id=row["document_id"],
chunk_id=row["chunk_id"],
action=ChunkEditAction(row["action"]),
actor=row["actor"],
at=_parse_iso(row["at"]) or datetime.now(UTC),
before=json.loads(row["before_json"]) if row["before_json"] else None,
after=json.loads(row["after_json"]) if row["after_json"] else None,
parents=json.loads(row["parents_json"]) if row["parents_json"] else [],
children=json.loads(row["children_json"]) if row["children_json"] else [],
reason=row["reason"],
)
def _row_to_push(row) -> ChunkPush:
return ChunkPush(
id=row["id"],
document_id=row["document_id"],
store_id=row["store_id"],
chunkset_hash=row["chunkset_hash"],
chunk_ids=json.loads(row["chunk_ids"]) if row["chunk_ids"] else [],
pushed_at=_parse_iso(row["pushed_at"]) or datetime.now(UTC),
)
class SqliteChunkEditRepository:
"""SQLite implementation of the ChunkEditRepository port.
The audit log is append-only: this repo offers `insert` and reads,
but no update or delete. Aborted edits should not produce an audit
row in the first place that contract is enforced in the service
layer (audit + chunk write happen in the same SQL transaction).
"""
async def insert(self, edit: ChunkEdit) -> None:
async with get_connection() as db:
await db.execute(
"""INSERT INTO chunk_edits
(id, document_id, chunk_id, action, actor, at,
before_json, after_json, parents_json, children_json, reason)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(
edit.id,
edit.document_id,
edit.chunk_id,
edit.action.value,
edit.actor,
str(edit.at),
json.dumps(edit.before) if edit.before is not None else None,
json.dumps(edit.after) if edit.after is not None else None,
json.dumps(edit.parents),
json.dumps(edit.children),
edit.reason,
),
)
await db.commit()
async def find_for_document(
self,
document_id: str,
*,
limit: int = 50,
offset: int = 0,
) -> list[ChunkEdit]:
async with get_connection() as db:
cursor = await db.execute(
"""SELECT * FROM chunk_edits
WHERE document_id = ?
ORDER BY at DESC
LIMIT ? OFFSET ?""",
(document_id, limit, offset),
)
rows = await cursor.fetchall()
return [_row_to_edit(r) for r in rows]
async def find_for_chunk(self, chunk_id: str) -> list[ChunkEdit]:
async with get_connection() as db:
cursor = await db.execute(
"SELECT * FROM chunk_edits WHERE chunk_id = ? ORDER BY at ASC",
(chunk_id,),
)
rows = await cursor.fetchall()
return [_row_to_edit(r) for r in rows]
class SqliteChunkPushRepository:
"""SQLite implementation of the ChunkPushRepository port."""
async def insert(self, push: ChunkPush) -> None:
async with get_connection() as db:
await db.execute(
"""INSERT INTO chunk_pushes
(id, document_id, store_id, chunkset_hash, chunk_ids, pushed_at)
VALUES (?, ?, ?, ?, ?, ?)""",
(
push.id,
push.document_id,
push.store_id,
push.chunkset_hash,
json.dumps(push.chunk_ids),
str(push.pushed_at),
),
)
await db.commit()
async def find_by_id(self, push_id: str) -> ChunkPush | None:
async with get_connection() as db:
cursor = await db.execute("SELECT * FROM chunk_pushes WHERE id = ?", (push_id,))
row = await cursor.fetchone()
return _row_to_push(row) if row else None
async def find_latest(self, document_id: str, store_id: str) -> ChunkPush | None:
async with get_connection() as db:
cursor = await db.execute(
"""SELECT * FROM chunk_pushes
WHERE document_id = ? AND store_id = ?
ORDER BY pushed_at DESC
LIMIT 1""",
(document_id, store_id),
)
row = await cursor.fetchone()
return _row_to_push(row) if row else None

View file

@ -0,0 +1,143 @@
"""Chunk repository — SQLite CRUD for first-class chunks (#205)."""
from __future__ import annotations
import json
from dataclasses import asdict
from datetime import UTC, datetime
from domain.models import Chunk
from domain.value_objects import ChunkBbox, ChunkDocItem
from persistence.database import get_connection
def _parse_iso(value: str | None) -> datetime | None:
if value is None or value == "":
return None
parsed = datetime.fromisoformat(value)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed
def _row_to_chunk(row) -> Chunk:
headings = json.loads(row["headings"]) if row["headings"] else []
bboxes_raw = json.loads(row["bboxes"]) if row["bboxes"] else []
doc_items_raw = json.loads(row["doc_items"]) if row["doc_items"] else []
return Chunk(
id=row["id"],
document_id=row["document_id"],
sequence=row["sequence"],
text=row["text"],
headings=headings,
source_page=row["source_page"],
bboxes=[ChunkBbox(page=b["page"], bbox=b["bbox"]) for b in bboxes_raw],
doc_items=[ChunkDocItem(self_ref=d["self_ref"], label=d["label"]) for d in doc_items_raw],
token_count=row["token_count"],
created_at=_parse_iso(row["created_at"]) or datetime.now(UTC),
updated_at=_parse_iso(row["updated_at"]) or datetime.now(UTC),
deleted_at=_parse_iso(row["deleted_at"]),
)
def _chunk_to_params(c: Chunk) -> tuple:
return (
c.id,
c.document_id,
c.sequence,
c.text,
json.dumps(c.headings),
c.source_page,
json.dumps([asdict(b) for b in c.bboxes]),
json.dumps([asdict(d) for d in c.doc_items]),
c.token_count,
str(c.created_at),
str(c.updated_at),
str(c.deleted_at) if c.deleted_at else None,
)
_INSERT_SQL = """INSERT INTO chunks
(id, document_id, sequence, text, headings, source_page,
bboxes, doc_items, token_count, created_at, updated_at, deleted_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)"""
class SqliteChunkRepository:
"""SQLite implementation of the ChunkRepository port."""
async def insert(self, chunk: Chunk) -> None:
async with get_connection() as db:
await db.execute(_INSERT_SQL, _chunk_to_params(chunk))
await db.commit()
async def insert_many(self, chunks: list[Chunk]) -> None:
if not chunks:
return
async with get_connection() as db:
await db.executemany(_INSERT_SQL, [_chunk_to_params(c) for c in chunks])
await db.commit()
async def update(self, chunk: Chunk) -> None:
async with get_connection() as db:
await db.execute(
"""UPDATE chunks SET
sequence = ?, text = ?, headings = ?, source_page = ?,
bboxes = ?, doc_items = ?, token_count = ?,
updated_at = ?, deleted_at = ?
WHERE id = ?""",
(
chunk.sequence,
chunk.text,
json.dumps(chunk.headings),
chunk.source_page,
json.dumps([asdict(b) for b in chunk.bboxes]),
json.dumps([asdict(d) for d in chunk.doc_items]),
chunk.token_count,
str(chunk.updated_at),
str(chunk.deleted_at) if chunk.deleted_at else None,
chunk.id,
),
)
await db.commit()
async def soft_delete(self, chunk_id: str, *, at: datetime) -> bool:
async with get_connection() as db:
cursor = await db.execute(
"UPDATE chunks SET deleted_at = ?, updated_at = ? WHERE id = ?",
(str(at), str(at), chunk_id),
)
await db.commit()
return cursor.rowcount > 0
async def find_for_document(
self,
document_id: str,
*,
include_deleted: bool = False,
) -> list[Chunk]:
clause = "" if include_deleted else " AND deleted_at IS NULL"
async with get_connection() as db:
cursor = await db.execute(
f"""SELECT * FROM chunks
WHERE document_id = ?{clause}
ORDER BY sequence ASC""",
(document_id,),
)
rows = await cursor.fetchall()
return [_row_to_chunk(r) for r in rows]
async def find_by_id(self, chunk_id: str) -> Chunk | None:
async with get_connection() as db:
cursor = await db.execute("SELECT * FROM chunks WHERE id = ?", (chunk_id,))
row = await cursor.fetchone()
return _row_to_chunk(row) if row else None
async def count_for_document(self, document_id: str) -> int:
async with get_connection() as db:
cursor = await db.execute(
"SELECT COUNT(*) AS n FROM chunks WHERE document_id = ? AND deleted_at IS NULL",
(document_id,),
)
row = await cursor.fetchone()
return int(row["n"]) if row else 0

View file

@ -42,25 +42,140 @@ CREATE TABLE IF NOT EXISTS analysis_jobs (
CREATE INDEX IF NOT EXISTS idx_analysis_jobs_status ON analysis_jobs(status);
CREATE INDEX IF NOT EXISTS idx_analysis_jobs_created_at ON analysis_jobs(created_at);
CREATE INDEX IF NOT EXISTS idx_documents_created_at ON documents(created_at);
-- 0.6.0 Per (document, store) ingestion state (#203).
CREATE TABLE IF NOT EXISTS stores (
id TEXT PRIMARY KEY,
name TEXT NOT NULL UNIQUE,
slug TEXT NOT NULL UNIQUE,
kind TEXT NOT NULL,
embedder TEXT NOT NULL,
config TEXT NOT NULL DEFAULT '{}',
is_default INTEGER NOT NULL DEFAULT 0,
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE TABLE IF NOT EXISTS document_store_links (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
store_id TEXT NOT NULL REFERENCES stores(id) ON DELETE CASCADE,
state TEXT NOT NULL,
chunkset_hash TEXT,
last_push_at TEXT,
last_run_id TEXT,
error_message TEXT,
UNIQUE (document_id, store_id)
);
CREATE INDEX IF NOT EXISTS idx_dsl_doc ON document_store_links(document_id);
CREATE INDEX IF NOT EXISTS idx_dsl_store ON document_store_links(store_id);
CREATE INDEX IF NOT EXISTS idx_dsl_state ON document_store_links(state);
-- 0.6.0 Chunks promoted to first-class entities + audit trail (#205).
CREATE TABLE IF NOT EXISTS chunks (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
sequence INTEGER NOT NULL,
text TEXT NOT NULL,
headings TEXT NOT NULL DEFAULT '[]',
source_page INTEGER,
bboxes TEXT NOT NULL DEFAULT '[]',
doc_items TEXT NOT NULL DEFAULT '[]',
token_count INTEGER,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
deleted_at TEXT
);
CREATE INDEX IF NOT EXISTS idx_chunks_doc ON chunks(document_id);
CREATE INDEX IF NOT EXISTS idx_chunks_doc_seq ON chunks(document_id, sequence);
CREATE TABLE IF NOT EXISTS chunk_edits (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
chunk_id TEXT,
action TEXT NOT NULL,
actor TEXT NOT NULL DEFAULT 'system',
at TEXT NOT NULL,
before_json TEXT,
after_json TEXT,
parents_json TEXT NOT NULL DEFAULT '[]',
children_json TEXT NOT NULL DEFAULT '[]',
reason TEXT
);
CREATE INDEX IF NOT EXISTS idx_chunk_edits_doc_at ON chunk_edits(document_id, at);
CREATE INDEX IF NOT EXISTS idx_chunk_edits_chunk ON chunk_edits(chunk_id);
CREATE TABLE IF NOT EXISTS chunk_pushes (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL REFERENCES documents(id) ON DELETE CASCADE,
store_id TEXT NOT NULL REFERENCES stores(id) ON DELETE CASCADE,
chunkset_hash TEXT NOT NULL,
chunk_ids TEXT NOT NULL,
pushed_at TEXT NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_chunk_pushes_doc_store ON chunk_pushes(document_id, store_id);
-- 0.6.0 migration progress (resumability for the 0.6.0 backfill, #206).
CREATE TABLE IF NOT EXISTS migration_progress (
name TEXT PRIMARY KEY,
completed_at TEXT NOT NULL DEFAULT (datetime('now'))
);
"""
_MIGRATIONS = [
("document_json", "ALTER TABLE analysis_jobs ADD COLUMN document_json TEXT"),
("chunks_json", "ALTER TABLE analysis_jobs ADD COLUMN chunks_json TEXT"),
("progress_current", "ALTER TABLE analysis_jobs ADD COLUMN progress_current INTEGER"),
("progress_total", "ALTER TABLE analysis_jobs ADD COLUMN progress_total INTEGER"),
# Column migrations: (table, column_name, ddl). Idempotent — applied only
# when the column is missing from the live schema. Order matters when a
# later migration depends on an earlier one (none today).
_COLUMN_MIGRATIONS: list[tuple[str, str, str]] = [
("analysis_jobs", "document_json", "ALTER TABLE analysis_jobs ADD COLUMN document_json TEXT"),
("analysis_jobs", "chunks_json", "ALTER TABLE analysis_jobs ADD COLUMN chunks_json TEXT"),
(
"analysis_jobs",
"progress_current",
"ALTER TABLE analysis_jobs ADD COLUMN progress_current INTEGER",
),
(
"analysis_jobs",
"progress_total",
"ALTER TABLE analysis_jobs ADD COLUMN progress_total INTEGER",
),
# 0.6.0 — Document lifecycle state machine (#202).
(
"documents",
"lifecycle_state",
"ALTER TABLE documents ADD COLUMN lifecycle_state TEXT NOT NULL DEFAULT 'Uploaded'",
),
(
"documents",
"lifecycle_state_at",
"ALTER TABLE documents ADD COLUMN lifecycle_state_at TEXT",
),
]
# DDL statements run after column migrations — typically CREATE INDEX
# IF NOT EXISTS for indexes on freshly-added columns.
_POST_MIGRATION_DDL: list[str] = [
"CREATE INDEX IF NOT EXISTS idx_documents_lifecycle_state ON documents(lifecycle_state)",
]
async def _run_migrations(db: aiosqlite.Connection) -> None:
"""Add columns that may be missing in older databases."""
cursor = await db.execute("PRAGMA table_info(analysis_jobs)")
existing = {row[1] for row in await cursor.fetchall()}
for col_name, ddl in _MIGRATIONS:
if col_name not in existing:
"""Apply additive column migrations, then any post-migration DDL.
Existing columns are detected via PRAGMA `table_info`, so re-running
the migration on an already-up-to-date DB is a no-op.
"""
columns_by_table: dict[str, set[str]] = {}
for table, col_name, ddl in _COLUMN_MIGRATIONS:
if table not in columns_by_table:
cursor = await db.execute(f"PRAGMA table_info({table})")
columns_by_table[table] = {row[1] for row in await cursor.fetchall()}
if col_name not in columns_by_table[table]:
await db.execute(ddl)
logger.info("Migration: added column %s to analysis_jobs", col_name)
columns_by_table[table].add(col_name)
logger.info("Migration: added column %s to %s", col_name, table)
for ddl in _POST_MIGRATION_DDL:
await db.execute(ddl)
await db.commit()
@ -70,10 +185,27 @@ async def init_db() -> None:
async with aiosqlite.connect(DB_PATH) as db:
await db.executescript(_SCHEMA)
await _run_migrations(db)
await _seed_default_store(db)
await db.commit()
logger.info("Database initialized at %s", DB_PATH)
async def _seed_default_store(db: aiosqlite.Connection) -> None:
"""Insert the canonical `default` store on first boot.
Idempotent uses INSERT OR IGNORE keyed on the unique slug. The
embedder is read from the DEFAULT_EMBEDDER env var with a sensible
fallback so existing single-index deployments keep working.
"""
embedder = os.environ.get("DEFAULT_EMBEDDER", "bge-m3")
await db.execute(
"""INSERT OR IGNORE INTO stores
(id, name, slug, kind, embedder, config, is_default, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, datetime('now'))""",
("default", "default", "default", "opensearch", embedder, "{}", 1),
)
async def get_db() -> aiosqlite.Connection:
"""Open a new database connection with row factory and FK enforcement."""
db = await aiosqlite.connect(DB_PATH)

View file

@ -5,15 +5,34 @@ from __future__ import annotations
from datetime import UTC, datetime
from domain.models import Document
from domain.value_objects import DocumentLifecycleState
from persistence.database import get_connection
def _parse_iso(value: str | None) -> datetime | None:
if value is None or value == "":
return None
parsed = datetime.fromisoformat(value)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed
def _row_to_document(row) -> Document:
created = row["created_at"]
if isinstance(created, str):
created = datetime.fromisoformat(created)
if created.tzinfo is None:
created = created.replace(tzinfo=UTC)
# `_run_migrations` guarantees both lifecycle columns exist by the time
# any read happens, so direct access is safe.
lifecycle_value = row["lifecycle_state"]
lifecycle_state = (
DocumentLifecycleState(lifecycle_value)
if lifecycle_value
else DocumentLifecycleState.UPLOADED
)
lifecycle_state_at = _parse_iso(row["lifecycle_state_at"])
return Document(
id=row["id"],
filename=row["filename"],
@ -22,6 +41,8 @@ def _row_to_document(row) -> Document:
page_count=row["page_count"],
storage_path=row["storage_path"],
created_at=created,
lifecycle_state=lifecycle_state,
lifecycle_state_at=lifecycle_state_at,
)
@ -32,8 +53,10 @@ class SqliteDocumentRepository:
"""Persist a new document record."""
async with get_connection() as db:
await db.execute(
"""INSERT INTO documents (id, filename, content_type, file_size, page_count, storage_path, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?)""",
"""INSERT INTO documents (
id, filename, content_type, file_size, page_count,
storage_path, created_at, lifecycle_state, lifecycle_state_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)""",
(
doc.id,
doc.filename,
@ -42,6 +65,8 @@ class SqliteDocumentRepository:
doc.page_count,
doc.storage_path,
str(doc.created_at),
doc.lifecycle_state.value,
str(doc.lifecycle_state_at) if doc.lifecycle_state_at else None,
),
)
await db.commit()
@ -72,6 +97,25 @@ class SqliteDocumentRepository:
)
await db.commit()
async def update_lifecycle(
self,
doc_id: str,
state: DocumentLifecycleState,
at: datetime,
) -> None:
"""Persist a lifecycle state transition for a document.
Callers must have already validated the transition via
`Document.transition_to()` this method does not check the
transition graph; it just writes.
"""
async with get_connection() as db:
await db.execute(
"UPDATE documents SET lifecycle_state = ?, lifecycle_state_at = ? WHERE id = ?",
(state.value, str(at), doc_id),
)
await db.commit()
async def delete(self, doc_id: str) -> bool:
"""Delete a document by ID. Returns True if a row was removed."""
async with get_connection() as db:

View file

@ -0,0 +1,98 @@
"""Document-Store link repository — SQLite CRUD on `document_store_links`."""
from __future__ import annotations
from datetime import UTC, datetime
from domain.models import DocumentStoreLink
from domain.value_objects import DocumentStoreLinkState
from persistence.database import get_connection
def _parse_iso(value: str | None) -> datetime | None:
if value is None or value == "":
return None
parsed = datetime.fromisoformat(value)
if parsed.tzinfo is None:
parsed = parsed.replace(tzinfo=UTC)
return parsed
def _row_to_link(row) -> DocumentStoreLink:
return DocumentStoreLink(
id=row["id"],
document_id=row["document_id"],
store_id=row["store_id"],
state=DocumentStoreLinkState(row["state"]),
chunkset_hash=row["chunkset_hash"],
last_push_at=_parse_iso(row["last_push_at"]),
last_run_id=row["last_run_id"],
error_message=row["error_message"],
)
class SqliteDocumentStoreLinkRepository:
"""SQLite implementation of the DocumentStoreLinkRepository port."""
async def upsert(self, link: DocumentStoreLink) -> None:
"""Insert a new link or update the existing (document_id, store_id) row."""
async with get_connection() as db:
await db.execute(
"""INSERT INTO document_store_links
(id, document_id, store_id, state, chunkset_hash,
last_push_at, last_run_id, error_message)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(document_id, store_id) DO UPDATE SET
state = excluded.state,
chunkset_hash = excluded.chunkset_hash,
last_push_at = excluded.last_push_at,
last_run_id = excluded.last_run_id,
error_message = excluded.error_message""",
(
link.id,
link.document_id,
link.store_id,
link.state.value,
link.chunkset_hash,
str(link.last_push_at) if link.last_push_at else None,
link.last_run_id,
link.error_message,
),
)
await db.commit()
async def find_for_document(self, document_id: str) -> list[DocumentStoreLink]:
async with get_connection() as db:
cursor = await db.execute(
"SELECT * FROM document_store_links WHERE document_id = ?",
(document_id,),
)
rows = await cursor.fetchall()
return [_row_to_link(r) for r in rows]
async def find_for_store(self, store_id: str) -> list[DocumentStoreLink]:
async with get_connection() as db:
cursor = await db.execute(
"SELECT * FROM document_store_links WHERE store_id = ?",
(store_id,),
)
rows = await cursor.fetchall()
return [_row_to_link(r) for r in rows]
async def find_one(self, document_id: str, store_id: str) -> DocumentStoreLink | None:
async with get_connection() as db:
cursor = await db.execute(
"SELECT * FROM document_store_links WHERE document_id = ? AND store_id = ?",
(document_id, store_id),
)
row = await cursor.fetchone()
return _row_to_link(row) if row else None
async def delete(self, document_id: str, store_id: str) -> bool:
async with get_connection() as db:
cursor = await db.execute(
"DELETE FROM document_store_links WHERE document_id = ? AND store_id = ?",
(document_id, store_id),
)
await db.commit()
return cursor.rowcount > 0

View file

@ -0,0 +1,126 @@
"""Store repository — SQLite CRUD for the `stores` table."""
from __future__ import annotations
import json
from datetime import UTC, datetime
from domain.models import Store
from domain.value_objects import StoreKind
from persistence.database import get_connection
def _row_to_store(row) -> Store:
created = row["created_at"]
if isinstance(created, str):
created = datetime.fromisoformat(created)
if created.tzinfo is None:
created = created.replace(tzinfo=UTC)
config_raw = row["config"] or "{}"
try:
config = json.loads(config_raw)
except (TypeError, ValueError):
config = {}
return Store(
id=row["id"],
name=row["name"],
slug=row["slug"],
kind=StoreKind(row["kind"]),
embedder=row["embedder"],
config=config,
is_default=bool(row["is_default"]),
created_at=created,
)
class SqliteStoreRepository:
"""SQLite implementation of the StoreRepository port."""
async def insert(self, store: Store) -> None:
async with get_connection() as db:
await db.execute(
"""INSERT INTO stores
(id, name, slug, kind, embedder, config, is_default, created_at)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)""",
(
store.id,
store.name,
store.slug,
store.kind.value,
store.embedder,
json.dumps(store.config),
1 if store.is_default else 0,
str(store.created_at),
),
)
await db.commit()
async def find_all(self) -> list[Store]:
async with get_connection() as db:
cursor = await db.execute("SELECT * FROM stores ORDER BY is_default DESC, name ASC")
rows = await cursor.fetchall()
return [_row_to_store(r) for r in rows]
async def find_by_slug(self, slug: str) -> Store | None:
async with get_connection() as db:
cursor = await db.execute("SELECT * FROM stores WHERE slug = ?", (slug,))
row = await cursor.fetchone()
return _row_to_store(row) if row else None
async def find_by_id(self, store_id: str) -> Store | None:
async with get_connection() as db:
cursor = await db.execute("SELECT * FROM stores WHERE id = ?", (store_id,))
row = await cursor.fetchone()
return _row_to_store(row) if row else None
async def find_by_name(self, name: str) -> Store | None:
async with get_connection() as db:
cursor = await db.execute("SELECT * FROM stores WHERE name = ?", (name,))
row = await cursor.fetchone()
return _row_to_store(row) if row else None
async def get_default(self) -> Store | None:
"""Return the seeded `default` store, if any."""
async with get_connection() as db:
cursor = await db.execute(
"SELECT * FROM stores WHERE is_default = 1 ORDER BY created_at ASC LIMIT 1"
)
row = await cursor.fetchone()
return _row_to_store(row) if row else None
async def update(self, store: Store) -> None:
"""Replace all mutable fields of an existing store. `id` and `created_at`
are not touched."""
async with get_connection() as db:
await db.execute(
"""UPDATE stores
SET name = ?, slug = ?, kind = ?, embedder = ?,
config = ?, is_default = ?
WHERE id = ?""",
(
store.name,
store.slug,
store.kind.value,
store.embedder,
json.dumps(store.config),
1 if store.is_default else 0,
store.id,
),
)
await db.commit()
async def clear_default_except(self, store_id: str) -> None:
"""Reset `is_default = 0` on every store except the given id. Used to
enforce single-default invariant when promoting a new default."""
async with get_connection() as db:
await db.execute(
"UPDATE stores SET is_default = 0 WHERE id != ?",
(store_id,),
)
await db.commit()
async def delete(self, store_id: str) -> bool:
async with get_connection() as db:
cursor = await db.execute("DELETE FROM stores WHERE id = ?", (store_id,))
await db.commit()
return cursor.rowcount > 0

View file

@ -16,6 +16,7 @@ from typing import TYPE_CHECKING
import pypdfium2 as pdfium
from domain.exceptions import InvalidLifecycleTransitionError
from domain.models import AnalysisJob, AnalysisStatus
from domain.services import classify_error, merge_results
from domain.value_objects import (
@ -23,6 +24,7 @@ from domain.value_objects import (
ChunkResult,
ConversionOptions,
ConversionResult,
DocumentLifecycleState,
)
if TYPE_CHECKING:
@ -97,6 +99,21 @@ class AnalysisService:
self._background_tasks: set[asyncio.Task] = set()
self._config = config or AnalysisConfig()
self._neo4j = neo4j_driver
# Duck-typed callable injected at startup. Wired in main.py to
# `ChunkService.promote_from_analysis_if_empty` so the canonical
# chunkset (#205) is populated on the first successful analysis,
# making the Doc workspace tab functional immediately (#256).
# Optional: when None, analysis behaviour is unchanged.
self._chunk_promoter = None
def set_chunk_promoter(self, chunk_service) -> None:
"""Inject the canonical-chunk promoter (post-construction wiring).
Kept loosely typed to avoid an import cycle between
`analysis_service` and `chunk_service`. The contract is a
coroutine `(document_id: str, chunks_json: str) -> int`.
"""
self._chunk_promoter = chunk_service
async def create(
self,
@ -132,6 +149,10 @@ class AnalysisService:
"""Return all analysis jobs, newest first."""
return await self._analysis_repo.find_all()
async def find_by_document(self, document_id: str) -> list[AnalysisJob]:
"""Return analysis jobs for a given document, newest first."""
return await self._analysis_repo.find_by_document(document_id)
async def find_by_id(self, job_id: str) -> AnalysisJob | None:
"""Find an analysis job by ID, or return None."""
return await self._analysis_repo.find_by_id(job_id)
@ -162,6 +183,10 @@ class AnalysisService:
chunks_json = json.dumps([_chunk_to_dict(c) for c in chunks])
await self._analysis_repo.update_chunks(job_id, chunks_json)
# Re-chunk drives the document into Chunked (idempotent if already
# Chunked; #204 will mark per-store links Stale separately).
await self._transition_document(job.document_id, DocumentLifecycleState.CHUNKED)
return chunks
async def update_chunk_text(self, job_id: str, chunk_index: int, text: str) -> list[dict]:
@ -292,11 +317,47 @@ class AnalysisService:
if job:
job.mark_failed(error)
await self._analysis_repo.update_status(job)
await self._transition_document(job.document_id, DocumentLifecycleState.FAILED)
except OSError:
logger.exception("Database I/O error marking job %s as failed", job_id)
except Exception:
logger.exception("Unexpected error marking job %s as failed", job_id)
async def _transition_document(
self,
document_id: str,
target: DocumentLifecycleState,
) -> None:
"""Drive a Document lifecycle transition (#202).
Idempotent on the target if the document is already in the
requested state, no write happens. Invalid transitions are
logged at WARNING and swallowed so a lifecycle hiccup does not
crash an otherwise-successful pipeline run.
"""
doc = await self._document_repo.find_by_id(document_id)
if doc is None:
return
if doc.lifecycle_state == target:
return
try:
event = doc.transition_to(target)
except InvalidLifecycleTransitionError:
logger.warning(
"Skipped invalid lifecycle transition for doc %s: %s -> %s",
document_id,
doc.lifecycle_state.value,
target.value,
)
return
await self._document_repo.update_lifecycle(document_id, event.current, event.at)
logger.info(
"lifecycle_changed doc_id=%s from=%s to=%s",
document_id,
event.previous.value,
event.current.value,
)
async def _run_analysis(
self,
job_id: str,
@ -379,6 +440,29 @@ class AnalysisService:
if result.page_count:
await self._document_repo.update_page_count(job.document_id, result.page_count)
# Promote chunks into the canonical doc-centric chunkset on the
# first analysis (#256). Idempotent: a doc that already has
# canonical chunks is left untouched, so subsequent ad-hoc
# analyses (Studio / OCR Debug) never overwrite hand-edited state.
if chunks_json is not None and self._chunk_promoter is not None:
try:
await self._chunk_promoter.promote_from_analysis_if_empty(
job.document_id, chunks_json
)
except Exception:
# Promotion is a best-effort hook — never fail an analysis
# because the canonical promotion hit a snag.
logger.exception("Canonical chunk promotion failed for doc %s", job.document_id)
# Drive the document lifecycle (#202): chunks present → Chunked,
# otherwise → Parsed.
target_state = (
DocumentLifecycleState.CHUNKED
if chunks_json is not None
else DocumentLifecycleState.PARSED
)
await self._transition_document(job.document_id, target_state)
await self._write_tree_to_neo4j(job, result.document_json)
logger.info("Analysis completed: %s (%d pages)", job_id, result.page_count)

View file

@ -0,0 +1,700 @@
"""Chunk service — canonical chunk lifecycle for a document (#256).
Sits between the API layer and the chunk / chunk_edit / chunk_push
repositories. Owns the invariants of the canonical chunkset:
- ordering by `sequence` (dense ascending, gaps allowed after split)
- soft-delete (audit log keeps before/after pointers valid)
- atomic mutation + audit row (one ChunkEdit per mutation)
- promotion from the first completed analysis (idempotent)
Re-uses `DocumentChunker` for rechunk (same port that
`AnalysisService.rechunk` uses), so chunking strategy logic is not
duplicated.
"""
from __future__ import annotations
import json
import logging
import uuid
from dataclasses import asdict
from datetime import UTC, datetime
from typing import TYPE_CHECKING
from domain.models import Chunk, ChunkEdit, ChunkPush
from domain.value_objects import (
ChunkBbox,
ChunkDocItem,
ChunkEditAction,
ChunkingOptions,
)
if TYPE_CHECKING:
from domain.ports import (
AnalysisRepository,
ChunkEditRepository,
ChunkPushRepository,
ChunkRepository,
DocumentChunker,
DocumentRepository,
)
from services.ingestion_service import IngestionService
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Errors — carry an http_status hint, mirrors store_service.py convention.
# ---------------------------------------------------------------------------
class ChunkServiceError(Exception):
http_status: int = 400
def __init__(self, message: str, *, http_status: int | None = None):
super().__init__(message)
if http_status is not None:
self.http_status = http_status
class ChunkNotFoundError(ChunkServiceError):
http_status = 404
class DocumentNotFoundError(ChunkServiceError):
http_status = 404
class ChunkConflictError(ChunkServiceError):
http_status = 409
class ChunkValidationError(ChunkServiceError):
http_status = 400
# ---------------------------------------------------------------------------
# Helpers — chunk ↔ dict conversions for audit log + analysis chunks_json.
# ---------------------------------------------------------------------------
def _utcnow() -> datetime:
return datetime.now(UTC)
def _new_id() -> str:
return str(uuid.uuid4())
def _chunk_to_audit_dict(c: Chunk) -> dict:
"""Serializable snapshot for ChunkEdit.before / .after."""
return {
"id": c.id,
"sequence": c.sequence,
"text": c.text,
"headings": list(c.headings),
"sourcePage": c.source_page,
"tokenCount": c.token_count,
"bboxes": [asdict(b) for b in c.bboxes],
"docItems": [asdict(d) for d in c.doc_items],
}
def _bbox_from_dict(d: dict) -> ChunkBbox:
return ChunkBbox(page=d["page"], bbox=list(d["bbox"]))
def _doc_item_from_dict(d: dict) -> ChunkDocItem:
return ChunkDocItem(
self_ref=d.get("selfRef") or d.get("self_ref", ""), label=d.get("label", "")
)
def _analysis_chunk_to_canonical(
document_id: str,
sequence: int,
raw: dict,
) -> Chunk:
"""Convert an entry from `AnalysisJob.chunks_json` (camelCase) into a
canonical `Chunk`. Used by `_promote_from_analysis`."""
return Chunk(
document_id=document_id,
sequence=sequence,
text=raw.get("text", ""),
headings=list(raw.get("headings", [])),
source_page=raw.get("sourcePage"),
bboxes=[_bbox_from_dict(b) for b in raw.get("bboxes", [])],
doc_items=[_doc_item_from_dict(d) for d in raw.get("docItems", [])],
token_count=raw.get("tokenCount"),
)
# ---------------------------------------------------------------------------
# Service
# ---------------------------------------------------------------------------
class ChunkService:
"""Orchestrates canonical chunk operations for a document."""
def __init__(
self,
chunk_repo: ChunkRepository,
chunk_edit_repo: ChunkEditRepository,
chunk_push_repo: ChunkPushRepository,
document_repo: DocumentRepository,
analysis_repo: AnalysisRepository,
chunker: DocumentChunker | None = None,
ingestion_service: IngestionService | None = None,
actor: str = "user",
) -> None:
self._chunks = chunk_repo
self._edits = chunk_edit_repo
self._pushes = chunk_push_repo
self._documents = document_repo
self._analyses = analysis_repo
self._chunker = chunker
self._ingestion = ingestion_service
self._actor = actor
# -- promotion (called by AnalysisService after first successful analysis)
async def promote_from_analysis_if_empty(self, document_id: str, chunks_json: str) -> int:
"""Populate the canonical chunkset from an analysis result, ONLY if
the document has no canonical chunks yet. Idempotent.
Returns the number of chunks promoted (0 if skipped).
"""
if await self._chunks.count_for_document(document_id) > 0:
return 0
try:
raw_chunks = json.loads(chunks_json)
except json.JSONDecodeError:
logger.exception("Invalid chunks_json for doc %s — skipping promotion", document_id)
return 0
if not isinstance(raw_chunks, list) or not raw_chunks:
return 0
canonical = [
_analysis_chunk_to_canonical(document_id, seq, raw)
for seq, raw in enumerate(raw_chunks)
if not raw.get("deleted")
]
if not canonical:
return 0
await self._chunks.insert_many(canonical)
for c in canonical:
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=c.id,
action=ChunkEditAction.INSERT,
actor="system:initial-analysis",
at=_utcnow(),
after=_chunk_to_audit_dict(c),
)
)
logger.info(
"chunk.promote docId=%s count=%d (initial-analysis)", document_id, len(canonical)
)
return len(canonical)
# -- read
async def list_chunks(self, document_id: str) -> list[Chunk]:
await self._require_doc(document_id)
return await self._chunks.find_for_document(document_id)
# -- mutations
async def add_chunk(self, document_id: str, *, text: str, after_id: str | None = None) -> Chunk:
await self._require_doc(document_id)
existing = await self._chunks.find_for_document(document_id)
sequence = self._sequence_after(existing, after_id)
await self._shift_sequences(existing, from_sequence=sequence)
new_chunk = Chunk(
document_id=document_id,
sequence=sequence,
text=text,
created_at=_utcnow(),
updated_at=_utcnow(),
)
await self._chunks.insert(new_chunk)
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=new_chunk.id,
action=ChunkEditAction.INSERT,
actor=self._actor,
at=_utcnow(),
after=_chunk_to_audit_dict(new_chunk),
)
)
logger.info(
"chunk.add docId=%s chunkId=%s sequence=%d", document_id, new_chunk.id, sequence
)
return new_chunk
async def update_chunk(
self,
document_id: str,
chunk_id: str,
*,
text: str | None = None,
headings: list[str] | None = None,
) -> Chunk:
chunk = await self._require_chunk(document_id, chunk_id)
before = _chunk_to_audit_dict(chunk)
if text is not None:
chunk.text = text
if headings is not None:
chunk.headings = list(headings)
chunk.updated_at = _utcnow()
await self._chunks.update(chunk)
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=chunk.id,
action=ChunkEditAction.UPDATE,
actor=self._actor,
at=_utcnow(),
before=before,
after=_chunk_to_audit_dict(chunk),
)
)
logger.info("chunk.update docId=%s chunkId=%s", document_id, chunk.id)
return chunk
async def delete_chunk(self, document_id: str, chunk_id: str) -> None:
chunk = await self._require_chunk(document_id, chunk_id)
before = _chunk_to_audit_dict(chunk)
deleted = await self._chunks.soft_delete(chunk_id, at=_utcnow())
if not deleted:
raise ChunkNotFoundError(f"Chunk not found: {chunk_id}")
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=chunk_id,
action=ChunkEditAction.DELETE,
actor=self._actor,
at=_utcnow(),
before=before,
)
)
logger.info("chunk.delete docId=%s chunkId=%s", document_id, chunk.id)
async def split_chunk(self, document_id: str, chunk_id: str, cursor_offset: int) -> list[Chunk]:
source = await self._require_chunk(document_id, chunk_id)
if cursor_offset <= 0 or cursor_offset >= len(source.text):
raise ChunkValidationError(
f"cursorOffset {cursor_offset} out of range for chunk of length {len(source.text)}"
)
existing = await self._chunks.find_for_document(document_id)
before = _chunk_to_audit_dict(source)
# Both halves inherit headings, source_page, bboxes, doc_items.
# Token counts are unknown post-split; leave as None.
head_text = source.text[:cursor_offset]
tail_text = source.text[cursor_offset:]
head = Chunk(
document_id=document_id,
sequence=source.sequence,
text=head_text,
headings=list(source.headings),
source_page=source.source_page,
bboxes=list(source.bboxes),
doc_items=list(source.doc_items),
)
tail = Chunk(
document_id=document_id,
sequence=source.sequence + 1,
text=tail_text,
headings=list(source.headings),
source_page=source.source_page,
bboxes=list(source.bboxes),
doc_items=list(source.doc_items),
)
# Push subsequent sequences by 1 to make room for `tail`.
await self._shift_sequences(existing, from_sequence=source.sequence + 1)
await self._chunks.soft_delete(source.id, at=_utcnow())
await self._chunks.insert_many([head, tail])
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=source.id,
action=ChunkEditAction.SPLIT,
actor=self._actor,
at=_utcnow(),
before=before,
children=[head.id, tail.id],
)
)
logger.info(
"chunk.split docId=%s sourceId=%s newIds=[%s,%s]",
document_id,
source.id,
head.id,
tail.id,
)
return [head, tail]
async def merge_chunks(self, document_id: str, ids: list[str]) -> Chunk:
if len(ids) < 2:
raise ChunkValidationError("merge requires at least 2 chunk ids")
existing = await self._chunks.find_for_document(document_id)
by_id = {c.id: c for c in existing}
targets = [by_id.get(i) for i in ids]
if any(t is None for t in targets):
missing = [i for i, t in zip(ids, targets, strict=True) if t is None]
raise ChunkNotFoundError(f"Chunks not found: {missing}")
ordered = sorted(targets, key=lambda c: c.sequence)
sequences = [c.sequence for c in ordered]
if sequences != list(range(sequences[0], sequences[0] + len(sequences))):
raise ChunkConflictError("merge requires contiguous chunks (by sequence)")
merged_text = "\n".join(c.text for c in ordered)
bboxes: list[ChunkBbox] = []
doc_items: list[ChunkDocItem] = []
for c in ordered:
bboxes.extend(c.bboxes)
doc_items.extend(c.doc_items)
token_total = sum((c.token_count or 0) for c in ordered) or None
merged = Chunk(
document_id=document_id,
sequence=ordered[0].sequence,
text=merged_text,
headings=list(ordered[0].headings),
source_page=ordered[0].source_page,
bboxes=bboxes,
doc_items=doc_items,
token_count=token_total,
)
for c in ordered:
await self._chunks.soft_delete(c.id, at=_utcnow())
await self._chunks.insert(merged)
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=merged.id,
action=ChunkEditAction.MERGE,
actor=self._actor,
at=_utcnow(),
parents=[c.id for c in ordered],
after=_chunk_to_audit_dict(merged),
)
)
logger.info(
"chunk.merge docId=%s sourceIds=%s newId=%s",
document_id,
[c.id for c in ordered],
merged.id,
)
return merged
async def rechunk_document(self, document_id: str, options: dict | None = None) -> list[Chunk]:
"""Re-run the chunker on the latest completed analysis JSON and
replace the canonical chunkset.
Emits one INSERT edit per new chunk and one DELETE per old chunk
keeps the audit log within the existing `ChunkEditAction` enum.
"""
await self._require_doc(document_id)
if not self._chunker:
raise ChunkServiceError("Chunker not configured", http_status=503)
job = await self._analyses.find_latest_completed_by_document(document_id)
if not job or not job.document_json:
raise ChunkServiceError(
"No completed analysis with document_json available for rechunk",
http_status=409,
)
chunk_opts = ChunkingOptions(**options) if options else ChunkingOptions()
new_results = await self._chunker.chunk(job.document_json, chunk_opts)
existing = await self._chunks.find_for_document(document_id)
now = _utcnow()
for c in existing:
await self._chunks.soft_delete(c.id, at=now)
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=c.id,
action=ChunkEditAction.DELETE,
actor="system:rechunk",
at=now,
before=_chunk_to_audit_dict(c),
)
)
new_chunks = [
Chunk(
document_id=document_id,
sequence=seq,
text=r.text,
headings=list(r.headings),
source_page=r.source_page,
bboxes=list(r.bboxes),
doc_items=[], # ChunkResult has no doc_items currently
token_count=r.token_count or None,
)
for seq, r in enumerate(new_results)
]
if new_chunks:
await self._chunks.insert_many(new_chunks)
for c in new_chunks:
await self._edits.insert(
ChunkEdit(
id=_new_id(),
document_id=document_id,
chunk_id=c.id,
action=ChunkEditAction.INSERT,
actor="system:rechunk",
at=now,
after=_chunk_to_audit_dict(c),
)
)
logger.info(
"chunk.rechunk docId=%s oldCount=%d newCount=%d",
document_id,
len(existing),
len(new_chunks),
)
return new_chunks
# -- diff (against last push to a store)
async def diff_against_store(self, document_id: str, store_id: str) -> list[dict]:
"""Compare the canonical chunkset to the last push for `store_id`.
Returns a list of `ChunkDiff`-shaped dicts (camelCase) covering:
- canonical chunks not in the last push status "added"
- canonical chunks updated since last push status "modified"
- canonical chunks unchanged since push status "unchanged"
- chunk ids in last push absent from canonical status "removed"
Coarse-grained does not produce a textDiff today (follow-up).
"""
await self._require_doc(document_id)
canonical = await self._chunks.find_for_document(document_id)
last_push = await self._pushes.find_latest(document_id, store_id)
if last_push is None:
return [{"chunkId": c.id, "status": "added", "textDiff": None} for c in canonical]
pushed_ids = set(last_push.chunk_ids)
diffs: list[dict] = []
for c in canonical:
if c.id not in pushed_ids:
diffs.append({"chunkId": c.id, "status": "added", "textDiff": None})
continue
if c.updated_at > last_push.pushed_at:
diffs.append({"chunkId": c.id, "status": "modified", "textDiff": None})
else:
diffs.append({"chunkId": c.id, "status": "unchanged", "textDiff": None})
canonical_ids = {c.id for c in canonical}
for cid in pushed_ids - canonical_ids:
diffs.append({"chunkId": cid, "status": "removed", "textDiff": None})
return diffs
# -- push (delegates to IngestionService; per-store dispatch is a follow-up)
async def push_to_store(self, document_id: str, store_id: str) -> dict:
"""Push the canonical chunkset to a store and record a `ChunkPush`.
Today this delegates to the globally-configured `IngestionService`
(single index). Per-store dispatch by slug is a follow-up issue
the `store_id` argument is recorded on the `ChunkPush` row so the
diff endpoint can answer "what was last pushed to store X" even
once the dispatch lands.
"""
await self._require_doc(document_id)
if self._ingestion is None:
raise ChunkServiceError(
"Ingestion not available (EMBEDDING_URL and OPENSEARCH_URL required)",
http_status=503,
)
doc = await self._documents.find_by_id(document_id)
canonical = await self._chunks.find_for_document(document_id)
if not canonical:
raise ChunkServiceError(
"No canonical chunks to push — run analysis or rechunk first",
http_status=409,
)
chunks_payload = [_chunk_to_ingestion_dict(c) for c in canonical]
chunks_json_payload = json.dumps(chunks_payload)
ingestion_result = await self._ingestion.ingest(
doc_id=document_id,
filename=(doc.filename if doc else document_id),
chunks_json=chunks_json_payload,
)
chunk_ids = [c.id for c in canonical]
push = ChunkPush(
id=_new_id(),
document_id=document_id,
store_id=store_id,
chunkset_hash=_compute_chunkset_hash(canonical),
chunk_ids=chunk_ids,
pushed_at=_utcnow(),
)
await self._pushes.insert(push)
token_total = sum((c.token_count or 0) for c in canonical)
logger.info(
"chunk.push docId=%s store=%s count=%d tokens=%d",
document_id,
store_id,
ingestion_result.chunks_indexed,
token_total,
)
return {
"jobId": push.id,
"summary": {
"embeds": ingestion_result.chunks_indexed,
"tokens": token_total,
},
}
# -- tree (read from latest analysis document_json)
async def get_tree(self, document_id: str) -> list[dict]:
"""Build a doc tree from the latest completed analysis.
Returns a list of `DocTreeNode`-shaped dicts (camelCase). Empty
list if no analysis is available yet caller decides if that is
an error or just "not parsed yet".
"""
await self._require_doc(document_id)
job = await self._analyses.find_latest_completed_by_document(document_id)
if not job or not job.document_json:
return []
try:
doc_data = json.loads(job.document_json)
except json.JSONDecodeError:
logger.exception("Invalid document_json for analysis %s", job.id)
return []
return _build_tree_nodes(doc_data)
# -- guards
async def _require_doc(self, document_id: str) -> None:
doc = await self._documents.find_by_id(document_id)
if not doc:
raise DocumentNotFoundError(f"Document not found: {document_id}")
async def _require_chunk(self, document_id: str, chunk_id: str) -> Chunk:
chunk = await self._chunks.find_by_id(chunk_id)
if not chunk or chunk.document_id != document_id or chunk.deleted_at is not None:
raise ChunkNotFoundError(f"Chunk not found: {chunk_id}")
return chunk
# -- sequence helpers
@staticmethod
def _sequence_after(existing: list[Chunk], after_id: str | None) -> int:
if after_id is None:
return (max((c.sequence for c in existing), default=-1)) + 1
anchor = next((c for c in existing if c.id == after_id), None)
if anchor is None:
raise ChunkNotFoundError(f"Anchor chunk not found: {after_id}")
return anchor.sequence + 1
async def _shift_sequences(self, existing: list[Chunk], *, from_sequence: int) -> None:
"""Push chunks at >= from_sequence one slot up to make room."""
affected = [c for c in existing if c.sequence >= from_sequence]
for c in affected:
c.sequence += 1
c.updated_at = _utcnow()
await self._chunks.update(c)
# ---------------------------------------------------------------------------
# Tree projection — extract a hierarchical outline from a DoclingDocument.
# Kept module-level so it stays cheap to test in isolation.
# ---------------------------------------------------------------------------
def _chunk_to_ingestion_dict(c: Chunk) -> dict:
"""Convert a canonical `Chunk` into the legacy chunks_json shape that
`IngestionService.ingest` consumes (camelCase, modeled after
`analysis_service._chunk_to_dict`)."""
return {
"text": c.text,
"headings": list(c.headings),
"sourcePage": c.source_page,
"tokenCount": c.token_count or 0,
"bboxes": [{"page": b.page, "bbox": list(b.bbox)} for b in c.bboxes],
"docItems": [{"selfRef": d.self_ref, "label": d.label} for d in c.doc_items],
}
def _compute_chunkset_hash(chunks: list[Chunk]) -> str:
"""Stable hash of the canonical chunkset content.
Used by `ChunkPush` snapshots so we can answer 'is the store in sync
with the current canonical state' without listing chunks from the
vector store.
"""
import hashlib
h = hashlib.sha256()
for c in chunks:
h.update(c.id.encode("utf-8"))
h.update(b"\x00")
h.update(c.text.encode("utf-8"))
h.update(b"\x00")
h.update(str(c.updated_at).encode("utf-8"))
h.update(b"\x01")
return h.hexdigest()
def _build_tree_nodes(doc_data: dict) -> list[dict]:
"""Project a Docling document JSON into a `[DocTreeNode]` outline.
The Docling document layout has top-level lists like `texts`, `tables`,
`pictures`, `groups`, etc. Each entry carries a `self_ref` and a
`label`. We surface a flat outline grouped by label families, which
is enough for the Inspect tab full hierarchy reconstruction is a
follow-up (#216).
"""
sections = (
("section_header", "Sections", []),
("title", "Titles", []),
("text", "Paragraphs", []),
("table", "Tables", []),
("picture", "Pictures", []),
)
section_map = {label: bucket for label, _, bucket in sections}
for collection in ("texts", "tables", "pictures", "groups"):
for item in doc_data.get(collection, []) or []:
label = item.get("label") or collection.rstrip("s")
ref = item.get("self_ref") or item.get("selfRef") or ""
display = item.get("text") or item.get("name") or label
bucket = section_map.get(label)
if bucket is None:
continue
bucket.append({"ref": ref, "type": label, "label": display, "children": []})
return [
{"ref": f"#group/{key}", "type": "group", "label": title, "children": children}
for key, title, children in sections
if children
]

View file

@ -0,0 +1,285 @@
"""Store service — CRUD orchestration for ingestion targets (#251).
Sits between the API layer and the SQLite repositories. Owns:
- input validation (per-kind config schema, slug shape, name uniqueness)
- single-default invariant (only one Store can be `is_default = True`)
- delete safety (refuse on seeded `default` slug or non-empty links)
- list enrichment (per-store document counts read from
`document_store_links`)
"""
from __future__ import annotations
import logging
import re
import uuid
from dataclasses import dataclass
from typing import TYPE_CHECKING
from domain.models import Store
from domain.value_objects import StoreKind
if TYPE_CHECKING:
from persistence.document_repo import SqliteDocumentRepository
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from persistence.store_repo import SqliteStoreRepository
logger = logging.getLogger(__name__)
_SLUG_PATTERN = re.compile(r"^[a-z0-9][a-z0-9-]*[a-z0-9]$|^[a-z0-9]$")
class StoreServiceError(Exception):
"""Base service error. Carries an `http_status` hint for the API layer."""
http_status: int = 400
def __init__(self, message: str, *, http_status: int | None = None):
super().__init__(message)
if http_status is not None:
self.http_status = http_status
class StoreNotFoundError(StoreServiceError):
http_status = 404
class StoreConflictError(StoreServiceError):
http_status = 409
class StoreValidationError(StoreServiceError):
http_status = 422
@dataclass
class StoreInfoView:
"""Read model for `GET /api/stores`. Mirrors the frontend `StoreInfo`."""
name: str
slug: str
kind: str
embedder: str
is_default: bool
document_count: int
chunk_count: int
connected: bool
error_message: str | None = None
@dataclass
class StoreDocEntryView:
"""Read model for `GET /api/stores/{slug}/documents`."""
doc_id: str
filename: str
state: str
chunk_count: int
pushed_at: str | None
def _validate_slug(slug: str) -> None:
if not slug or not _SLUG_PATTERN.match(slug):
raise StoreValidationError(
"slug must be lowercase alphanumeric with optional dashes (e.g. 'rh-corpus-v3')"
)
def _validate_config_for_kind(kind: StoreKind, config: dict) -> None:
"""Per-kind config schema. New kinds plug in here without touching the
rest of the pipeline. Authentication for remote stores comes from the
deployment env (see `infra/settings.py`)."""
if kind is StoreKind.OPENSEARCH:
index_name = config.get("index_name") or config.get("indexName")
if not isinstance(index_name, str) or not index_name.strip():
raise StoreValidationError("OpenSearch config requires a non-empty 'index_name'")
elif kind is StoreKind.NEO4J:
index_name = config.get("index_name") or config.get("indexName")
if not isinstance(index_name, str) or not index_name.strip():
raise StoreValidationError("Neo4j config requires a non-empty 'index_name'")
class StoreService:
"""Orchestrates store CRUD on top of SQLite repositories."""
def __init__(
self,
store_repo: SqliteStoreRepository,
link_repo: SqliteDocumentStoreLinkRepository,
document_repo: SqliteDocumentRepository | None = None,
):
self._stores = store_repo
self._links = link_repo
self._documents = document_repo
async def list_stores(self) -> list[StoreInfoView]:
stores = await self._stores.find_all()
views: list[StoreInfoView] = []
for store in stores:
links = await self._links.find_for_store(store.id)
views.append(
StoreInfoView(
name=store.name,
slug=store.slug,
kind=store.kind.value,
embedder=store.embedder,
is_default=store.is_default,
document_count=len(links),
chunk_count=0,
connected=True,
)
)
return views
async def get_by_slug(self, slug: str) -> Store:
store = await self._stores.find_by_slug(slug)
if store is None:
raise StoreNotFoundError(f"Store '{slug}' not found")
return store
async def create_store(
self,
*,
name: str,
slug: str,
kind: StoreKind,
embedder: str,
config: dict,
is_default: bool = False,
) -> Store:
name = (name or "").strip()
slug = (slug or "").strip().lower()
embedder = (embedder or "").strip()
if not name:
raise StoreValidationError("name is required")
if not embedder:
raise StoreValidationError("embedder is required")
_validate_slug(slug)
_validate_config_for_kind(kind, config)
if await self._stores.find_by_slug(slug) is not None:
raise StoreConflictError(f"slug '{slug}' is already in use")
if await self._stores.find_by_name(name) is not None:
raise StoreConflictError(f"name '{name}' is already in use")
store = Store(
id=str(uuid.uuid4()),
name=name,
slug=slug,
kind=kind,
embedder=embedder,
config=config,
is_default=is_default,
)
await self._stores.insert(store)
if is_default:
await self._stores.clear_default_except(store.id)
return store
async def update_store(
self,
slug: str,
*,
name: str | None = None,
new_slug: str | None = None,
kind: StoreKind | None = None,
embedder: str | None = None,
config: dict | None = None,
is_default: bool | None = None,
) -> Store:
store = await self.get_by_slug(slug)
if name is not None:
name = name.strip()
if not name:
raise StoreValidationError("name cannot be empty")
other = await self._stores.find_by_name(name)
if other is not None and other.id != store.id:
raise StoreConflictError(f"name '{name}' is already in use")
store.name = name
if new_slug is not None:
new_slug = new_slug.strip().lower()
_validate_slug(new_slug)
if new_slug != store.slug:
other = await self._stores.find_by_slug(new_slug)
if other is not None and other.id != store.id:
raise StoreConflictError(f"slug '{new_slug}' is already in use")
store.slug = new_slug
if kind is not None:
store.kind = kind
if embedder is not None:
embedder = embedder.strip()
if not embedder:
raise StoreValidationError("embedder cannot be empty")
store.embedder = embedder
if config is not None:
store.config = config
# Validate the (kind, config) pair as a whole — even when only one of
# the two changed, the combination must still satisfy the schema.
_validate_config_for_kind(store.kind, store.config)
promote_default = False
if is_default is not None:
store.is_default = is_default
promote_default = is_default
await self._stores.update(store)
if promote_default:
await self._stores.clear_default_except(store.id)
return store
async def list_documents(self, slug: str) -> list[StoreDocEntryView]:
store = await self.get_by_slug(slug)
links = await self._links.find_for_store(store.id)
if self._documents is None:
return [
StoreDocEntryView(
doc_id=link.document_id,
filename=link.document_id,
state=link.state.value,
chunk_count=0,
pushed_at=str(link.last_push_at) if link.last_push_at else None,
)
for link in links
]
entries: list[StoreDocEntryView] = []
for link in links:
doc = await self._documents.find_by_id(link.document_id)
entries.append(
StoreDocEntryView(
doc_id=link.document_id,
filename=doc.filename if doc else link.document_id,
state=link.state.value,
chunk_count=0,
pushed_at=str(link.last_push_at) if link.last_push_at else None,
)
)
return entries
async def remove_document(self, slug: str, doc_id: str) -> None:
store = await self.get_by_slug(slug)
removed = await self._links.delete(doc_id, store.id)
if not removed:
raise StoreNotFoundError(f"document '{doc_id}' is not linked to store '{slug}'")
async def delete_store(self, slug: str) -> None:
store = await self.get_by_slug(slug)
if store.slug == "default":
raise StoreConflictError(
"the seeded 'default' store cannot be deleted",
http_status=409,
)
links = await self._links.find_for_store(store.id)
if links:
raise StoreConflictError(
f"store '{slug}' has {len(links)} linked document(s); remove the documents first",
http_status=409,
)
await self._stores.delete(store.id)

View file

@ -68,6 +68,28 @@ class TestHealthEndpoint:
data = resp.json()
assert data["ingestionAvailable"] is True
def test_health_exposes_surface_flags(self, client):
"""0.6.1 (#257): /api/health surfaces studio + rag_pipeline master flags."""
resp = client.get("/api/health")
data = resp.json()
assert "studioModeEnabled" in data
assert "ragPipelineEnabled" in data
# Defaults: studio off, rag pipeline on (production target).
assert data["studioModeEnabled"] is False
assert data["ragPipelineEnabled"] is True
def test_health_exposes_doc_mode_flags(self, client):
"""0.6.0 (#210, renamed in #257): /api/health surfaces inspect/linked/ask sub-flags."""
resp = client.get("/api/health")
data = resp.json()
assert "inspectModeEnabled" in data
assert "linkedModeEnabled" in data
assert "askModeEnabled" in data
# Sub-flag defaults preserve current behaviour (all enabled).
assert data["inspectModeEnabled"] is True
assert data["linkedModeEnabled"] is True
assert data["askModeEnabled"] is True
class TestDocumentEndpoints:
def test_list_documents(self, client, mock_document_service):
@ -187,6 +209,22 @@ class TestAnalysisEndpoints:
assert data[0]["documentFilename"] == "test.pdf"
assert data[0]["status"] == "PENDING"
def test_list_analyses_filtered_by_document(self, client, mock_analysis_service):
mock_analysis_service.find_all = AsyncMock(return_value=[])
mock_analysis_service.find_by_document = AsyncMock(
return_value=[
AnalysisJob(id="j2", document_id="d42", document_filename="paper.pdf"),
]
)
resp = client.get("/api/analyses?documentId=d42")
assert resp.status_code == 200
data = resp.json()
assert len(data) == 1
assert data[0]["documentId"] == "d42"
mock_analysis_service.find_by_document.assert_awaited_once_with("d42")
mock_analysis_service.find_all.assert_not_awaited()
def test_get_analysis(self, client, mock_analysis_service):
job = AnalysisJob(id="j1", document_id="d1", document_filename="test.pdf")
job.mark_running()

View file

@ -0,0 +1,200 @@
"""Tests for `/api/stores/*` HTTP endpoints (#251)."""
from __future__ import annotations
import pytest
from fastapi.testclient import TestClient
from domain.models import Document, DocumentStoreLink
from domain.value_objects import DocumentStoreLinkState
from main import app
from persistence.database import init_db
from persistence.document_repo import SqliteDocumentRepository
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from persistence.store_repo import SqliteStoreRepository
from services.store_service import StoreService
@pytest.fixture(autouse=True)
async def setup_db_and_state(monkeypatch, tmp_path):
db_path = str(tmp_path / "test.db")
monkeypatch.setattr("persistence.database.DB_PATH", db_path)
await init_db()
store_repo = SqliteStoreRepository()
link_repo = SqliteDocumentStoreLinkRepository()
document_repo = SqliteDocumentRepository()
original = getattr(app.state, "store_service", None)
app.state.store_service = StoreService(store_repo, link_repo, document_repo)
yield
app.state.store_service = original
@pytest.fixture
def client():
return TestClient(app, raise_server_exceptions=False)
VALID_OS = {
"name": "rh",
"slug": "rh",
"kind": "opensearch",
"embedder": "bge-m3",
"config": {"indexName": "rh"},
}
class TestList:
def test_list_returns_seeded_default(self, client):
resp = client.get("/api/stores")
assert resp.status_code == 200
slugs = [s["slug"] for s in resp.json()]
assert "default" in slugs
def test_list_uses_camel_case(self, client):
resp = client.get("/api/stores")
body = resp.json()[0]
assert "documentCount" in body
assert "isDefault" in body
class TestCreate:
def test_create_201(self, client):
resp = client.post("/api/stores", json=VALID_OS)
assert resp.status_code == 201, resp.text
body = resp.json()
assert body["slug"] == "rh"
assert body["isDefault"] is False
def test_create_duplicate_slug_409(self, client):
client.post("/api/stores", json=VALID_OS)
resp = client.post("/api/stores", json={**VALID_OS, "name": "rh-2"})
assert resp.status_code == 409
def test_create_invalid_kind_422(self, client):
resp = client.post("/api/stores", json={**VALID_OS, "kind": "pinecone"})
assert resp.status_code == 422
def test_create_missing_index_422(self, client):
resp = client.post("/api/stores", json={**VALID_OS, "config": {}})
assert resp.status_code == 422
def test_create_bad_slug_422(self, client):
resp = client.post("/api/stores", json={**VALID_OS, "slug": "RH Corp"})
assert resp.status_code == 422
class TestRead:
def test_get_by_slug(self, client):
client.post("/api/stores", json=VALID_OS)
resp = client.get("/api/stores/rh")
assert resp.status_code == 200
body = resp.json()
assert body["slug"] == "rh"
assert body["embedder"] == "bge-m3"
def test_get_unknown_404(self, client):
resp = client.get("/api/stores/missing")
assert resp.status_code == 404
class TestUpdate:
def test_patch_embedder(self, client):
client.post("/api/stores", json=VALID_OS)
resp = client.patch("/api/stores/rh", json={"embedder": "bge-large"})
assert resp.status_code == 200
assert resp.json()["embedder"] == "bge-large"
def test_patch_rename_slug(self, client):
client.post("/api/stores", json=VALID_OS)
resp = client.patch("/api/stores/rh", json={"slug": "rh-v2"})
assert resp.status_code == 200
assert resp.json()["slug"] == "rh-v2"
assert client.get("/api/stores/rh").status_code == 404
assert client.get("/api/stores/rh-v2").status_code == 200
def test_patch_unknown_404(self, client):
resp = client.patch("/api/stores/missing", json={"embedder": "x"})
assert resp.status_code == 404
class TestDelete:
def test_delete_204(self, client):
client.post("/api/stores", json=VALID_OS)
resp = client.delete("/api/stores/rh")
assert resp.status_code == 204
assert client.get("/api/stores/rh").status_code == 404
def test_delete_default_409(self, client):
resp = client.delete("/api/stores/default")
assert resp.status_code == 409
async def test_delete_with_links_409(self, client):
client.post("/api/stores", json=VALID_OS)
# Seed a link from outside (service-level access).
await SqliteDocumentRepository().insert(
Document(id="d-1", filename="t.pdf", storage_path="/tmp/t.pdf")
)
store = await SqliteStoreRepository().find_by_slug("rh")
assert store is not None
await SqliteDocumentStoreLinkRepository().upsert(
DocumentStoreLink(
id="l-1",
document_id="d-1",
store_id=store.id,
state=DocumentStoreLinkState.INGESTED,
)
)
resp = client.delete("/api/stores/rh")
assert resp.status_code == 409
class TestStoreDocuments:
async def test_list_documents_for_store(self, client):
client.post("/api/stores", json=VALID_OS)
await SqliteDocumentRepository().insert(
Document(id="d-1", filename="hr.pdf", storage_path="/tmp/hr.pdf")
)
store = await SqliteStoreRepository().find_by_slug("rh")
assert store is not None
await SqliteDocumentStoreLinkRepository().upsert(
DocumentStoreLink(
id="l-1",
document_id="d-1",
store_id=store.id,
state=DocumentStoreLinkState.INGESTED,
)
)
resp = client.get("/api/stores/rh/documents")
assert resp.status_code == 200
body = resp.json()
assert len(body) == 1
assert body[0]["docId"] == "d-1"
assert body[0]["filename"] == "hr.pdf"
async def test_remove_document_204(self, client):
client.post("/api/stores", json=VALID_OS)
await SqliteDocumentRepository().insert(
Document(id="d-1", filename="hr.pdf", storage_path="/tmp/hr.pdf")
)
store = await SqliteStoreRepository().find_by_slug("rh")
assert store is not None
await SqliteDocumentStoreLinkRepository().upsert(
DocumentStoreLink(
id="l-1",
document_id="d-1",
store_id=store.id,
state=DocumentStoreLinkState.INGESTED,
)
)
resp = client.delete("/api/stores/rh/documents/d-1")
assert resp.status_code == 204
# Now empty.
listing = client.get("/api/stores/rh/documents")
assert listing.json() == []
def test_remove_unknown_doc_404(self, client):
client.post("/api/stores", json=VALID_OS)
resp = client.delete("/api/stores/rh/documents/missing")
assert resp.status_code == 404

View file

@ -0,0 +1,196 @@
"""Tests for the pure-domain chunk-editing operations (#205)."""
from __future__ import annotations
import pytest
from domain.chunk_editing import (
ChunkEditingError,
delete,
insert,
merge,
split,
update,
)
from domain.models import Chunk
def _make(document_id: str = "doc-1", count: int = 3) -> list[Chunk]:
out: list[Chunk] = []
for i in range(count):
out.append(
Chunk(
id=f"c-{i}",
document_id=document_id,
sequence=i,
text=f"text {i}",
headings=[f"H{i}"],
source_page=i + 1,
)
)
return out
# ---------------------------------------------------------------------------
# insert
# ---------------------------------------------------------------------------
def test_insert_at_end_appends_chunk_with_correct_sequence() -> None:
chunks = _make(count=2)
out, new = insert(chunks, at_position=2, text="new", document_id="doc-1")
assert len(out) == 3
assert out[2].id == new.id
assert new.sequence == 2
# Existing chunks untouched.
assert out[0].sequence == 0
assert out[1].sequence == 1
def test_insert_in_middle_shifts_subsequent_sequences() -> None:
chunks = _make(count=3)
out, new = insert(chunks, at_position=1, text="middle", document_id="doc-1")
sequences = [c.sequence for c in out]
assert sequences == [0, 1, 2, 3]
assert out[1].id == new.id
# Original chunk-1 was shifted to sequence 2.
assert next(c for c in out if c.id == "c-1").sequence == 2
def test_insert_out_of_range_raises() -> None:
chunks = _make(count=2)
with pytest.raises(ChunkEditingError):
insert(chunks, at_position=99, text="x", document_id="doc-1")
# ---------------------------------------------------------------------------
# update
# ---------------------------------------------------------------------------
def test_update_changes_text_in_place() -> None:
chunks = _make()
out, modified = update(chunks, "c-1", text="edited")
assert modified.id == "c-1"
assert modified.text == "edited"
# Identity preserved.
assert next(c for c in out if c.id == "c-1").text == "edited"
def test_update_can_change_headings_only() -> None:
chunks = _make()
_, modified = update(chunks, "c-0", headings=["X"])
assert modified.headings == ["X"]
def test_update_unknown_chunk_raises() -> None:
chunks = _make()
with pytest.raises(ChunkEditingError):
update(chunks, "missing", text="x")
def test_update_deleted_chunk_raises() -> None:
chunks = _make()
delete(chunks, "c-1")
with pytest.raises(ChunkEditingError):
update(chunks, "c-1", text="x")
# ---------------------------------------------------------------------------
# delete
# ---------------------------------------------------------------------------
def test_delete_marks_deleted_at() -> None:
chunks = _make()
out, deleted = delete(chunks, "c-1")
target = next(c for c in out if c.id == "c-1")
assert target.deleted_at is not None
assert deleted is target
def test_delete_is_idempotent() -> None:
chunks = _make()
out_a, _ = delete(chunks, "c-1")
out_b, _ = delete(out_a, "c-1")
target_a = next(c for c in out_a if c.id == "c-1")
target_b = next(c for c in out_b if c.id == "c-1")
assert target_a.deleted_at == target_b.deleted_at
# ---------------------------------------------------------------------------
# merge
# ---------------------------------------------------------------------------
def test_merge_removes_sources_and_returns_new_chunk() -> None:
chunks = _make(count=3)
out, merged = merge(chunks, ["c-0", "c-1"])
ids = {c.id for c in out}
assert "c-0" not in ids
assert "c-1" not in ids
assert merged.id in ids
assert merged.text == "text 0\ntext 1"
# Merged chunk inherits headings from first source.
assert merged.headings == ["H0"]
# Source page is the min.
assert merged.source_page == 1
def test_merge_requires_at_least_two_chunks() -> None:
chunks = _make()
with pytest.raises(ChunkEditingError):
merge(chunks, ["c-0"])
def test_merge_across_documents_raises() -> None:
chunks = _make(document_id="doc-1", count=2)
other = Chunk(id="x-0", document_id="doc-2", sequence=10, text="other")
chunks.append(other)
with pytest.raises(ChunkEditingError):
merge(chunks, ["c-0", "x-0"])
# ---------------------------------------------------------------------------
# split
# ---------------------------------------------------------------------------
def test_split_creates_two_new_chunks() -> None:
chunks = _make(count=2)
out, left, right = split(chunks, "c-0", at_offset=4)
assert left.text == "text"
assert right.text == " 0"
ids = {c.id for c in out}
# Source replaced by two new chunks.
assert "c-0" not in ids
assert left.id in ids
assert right.id in ids
def test_split_preserves_headings_on_both_halves() -> None:
chunks = _make(count=1)
_, left, right = split(chunks, "c-0", at_offset=2)
assert left.headings == ["H0"]
assert right.headings == ["H0"]
def test_split_shifts_subsequent_sequences() -> None:
chunks = _make(count=3)
out, _, _ = split(chunks, "c-0", at_offset=2)
# After split, c-1 (was seq=1) and c-2 (was seq=2) shift up by 1.
seqs = sorted(c.sequence for c in out)
assert seqs == [0, 1, 2, 3]
def test_split_at_zero_offset_raises() -> None:
chunks = _make()
with pytest.raises(ChunkEditingError):
split(chunks, "c-0", at_offset=0)
def test_split_beyond_text_length_raises() -> None:
chunks = _make()
target = next(c for c in chunks if c.id == "c-0")
with pytest.raises(ChunkEditingError):
split(chunks, "c-0", at_offset=len(target.text))

View file

@ -0,0 +1,212 @@
"""Tests for the chunk / chunk_edit / chunk_push repositories (#205)."""
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from domain.models import Chunk, ChunkEdit, ChunkPush, Document
from domain.value_objects import ChunkBbox, ChunkEditAction
from persistence.chunk_edit_repo import (
SqliteChunkEditRepository,
SqliteChunkPushRepository,
)
from persistence.chunk_repo import SqliteChunkRepository
from persistence.database import init_db
from persistence.document_repo import SqliteDocumentRepository
@pytest.fixture(autouse=True)
async def setup_db(monkeypatch, tmp_path):
db_path = str(tmp_path / "test.db")
monkeypatch.setattr("persistence.database.DB_PATH", db_path)
await init_db()
yield
@pytest.fixture
async def doc():
repo = SqliteDocumentRepository()
document = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
await repo.insert(document)
return document
@pytest.fixture
def chunk_repo():
return SqliteChunkRepository()
@pytest.fixture
def edit_repo():
return SqliteChunkEditRepository()
@pytest.fixture
def push_repo():
return SqliteChunkPushRepository()
class TestChunkRepo:
async def test_insert_and_find_for_document(self, doc, chunk_repo):
c = Chunk(
id="c-1",
document_id=doc.id,
sequence=0,
text="alpha",
headings=["A"],
source_page=1,
bboxes=[ChunkBbox(page=1, bbox=[0, 0, 10, 10])],
)
await chunk_repo.insert(c)
out = await chunk_repo.find_for_document(doc.id)
assert len(out) == 1
assert out[0].id == "c-1"
assert out[0].text == "alpha"
assert out[0].headings == ["A"]
assert out[0].bboxes[0].bbox == [0, 0, 10, 10]
async def test_insert_many_round_trips(self, doc, chunk_repo):
chunks = [
Chunk(id=f"c-{i}", document_id=doc.id, sequence=i, text=f"t{i}") for i in range(5)
]
await chunk_repo.insert_many(chunks)
out = await chunk_repo.find_for_document(doc.id)
assert len(out) == 5
# Returned in sequence order.
assert [c.id for c in out] == [f"c-{i}" for i in range(5)]
async def test_soft_delete_excludes_chunk_unless_requested(self, doc, chunk_repo):
await chunk_repo.insert(Chunk(id="c-1", document_id=doc.id, sequence=0, text="a"))
await chunk_repo.soft_delete("c-1", at=datetime(2026, 4, 29, 12, tzinfo=UTC))
active = await chunk_repo.find_for_document(doc.id)
assert active == []
all_chunks = await chunk_repo.find_for_document(doc.id, include_deleted=True)
assert len(all_chunks) == 1
assert all_chunks[0].deleted_at is not None
async def test_update_persists_text_and_sequence(self, doc, chunk_repo):
c = Chunk(id="c-1", document_id=doc.id, sequence=0, text="old")
await chunk_repo.insert(c)
c.text = "new"
c.sequence = 5
c.updated_at = datetime(2026, 4, 29, 12, tzinfo=UTC)
await chunk_repo.update(c)
out = await chunk_repo.find_by_id("c-1")
assert out is not None
assert out.text == "new"
assert out.sequence == 5
async def test_cascade_delete_with_document(self, doc, chunk_repo):
await chunk_repo.insert(Chunk(id="c-1", document_id=doc.id, sequence=0, text="a"))
await SqliteDocumentRepository().delete(doc.id)
out = await chunk_repo.find_for_document(doc.id, include_deleted=True)
assert out == []
class TestChunkEditRepo:
async def test_insert_and_history(self, doc, edit_repo):
edit = ChunkEdit(
id="e-1",
document_id=doc.id,
chunk_id="c-1",
action=ChunkEditAction.UPDATE,
actor="user@example",
at=datetime(2026, 4, 29, 12, tzinfo=UTC),
before={"text": "old"},
after={"text": "new"},
reason="typo",
)
await edit_repo.insert(edit)
out = await edit_repo.find_for_document(doc.id)
assert len(out) == 1
assert out[0].action == ChunkEditAction.UPDATE
assert out[0].before == {"text": "old"}
assert out[0].after == {"text": "new"}
assert out[0].reason == "typo"
async def test_history_orders_newest_first(self, doc, edit_repo):
for i in range(3):
await edit_repo.insert(
ChunkEdit(
id=f"e-{i}",
document_id=doc.id,
chunk_id=f"c-{i}",
action=ChunkEditAction.INSERT,
actor="system",
at=datetime(2026, 4, 29, 12 + i, tzinfo=UTC),
)
)
out = await edit_repo.find_for_document(doc.id)
ids = [e.id for e in out]
assert ids == ["e-2", "e-1", "e-0"]
async def test_find_for_chunk_filters(self, doc, edit_repo):
for i in range(3):
await edit_repo.insert(
ChunkEdit(
id=f"e-{i}",
document_id=doc.id,
chunk_id="c-target" if i == 1 else f"c-{i}",
action=ChunkEditAction.UPDATE,
actor="system",
at=datetime(2026, 4, 29, 12 + i, tzinfo=UTC),
)
)
out = await edit_repo.find_for_chunk("c-target")
assert [e.id for e in out] == ["e-1"]
async def test_lineage_round_trips(self, doc, edit_repo):
edit = ChunkEdit(
id="e-merge",
document_id=doc.id,
chunk_id="c-merged",
action=ChunkEditAction.MERGE,
actor="system",
at=datetime(2026, 4, 29, 12, tzinfo=UTC),
parents=["c-1", "c-2"],
children=["c-merged"],
)
await edit_repo.insert(edit)
out = await edit_repo.find_for_document(doc.id)
assert out[0].parents == ["c-1", "c-2"]
assert out[0].children == ["c-merged"]
class TestChunkPushRepo:
async def test_insert_and_find_latest(self, doc, push_repo):
first = ChunkPush(
id="p-1",
document_id=doc.id,
store_id="default",
chunkset_hash="hash-old",
chunk_ids=["c-1", "c-2"],
pushed_at=datetime(2026, 4, 29, 10, tzinfo=UTC),
)
latest = ChunkPush(
id="p-2",
document_id=doc.id,
store_id="default",
chunkset_hash="hash-new",
chunk_ids=["c-1", "c-3"],
pushed_at=datetime(2026, 4, 29, 11, tzinfo=UTC),
)
await push_repo.insert(first)
await push_repo.insert(latest)
found = await push_repo.find_latest(doc.id, "default")
assert found is not None
assert found.id == "p-2"
assert found.chunkset_hash == "hash-new"
assert found.chunk_ids == ["c-1", "c-3"]
async def test_find_latest_returns_none_when_absent(self, doc, push_repo):
assert await push_repo.find_latest(doc.id, "default") is None

View file

@ -0,0 +1,378 @@
"""Tests for ChunkService — canonical chunk lifecycle (#256)."""
from __future__ import annotations
import json
from datetime import UTC, datetime
from unittest.mock import AsyncMock, MagicMock
import pytest
from domain.models import AnalysisJob, AnalysisStatus, Chunk, Document
from domain.value_objects import ChunkEditAction, ChunkResult
from persistence.analysis_repo import SqliteAnalysisRepository
from persistence.chunk_edit_repo import (
SqliteChunkEditRepository,
SqliteChunkPushRepository,
)
from persistence.chunk_repo import SqliteChunkRepository
from persistence.database import init_db
from persistence.document_repo import SqliteDocumentRepository
from services.chunk_service import (
ChunkConflictError,
ChunkNotFoundError,
ChunkService,
ChunkServiceError,
ChunkValidationError,
DocumentNotFoundError,
)
# ---------------------------------------------------------------------------
# Fixtures
# ---------------------------------------------------------------------------
@pytest.fixture(autouse=True)
async def setup_db(monkeypatch, tmp_path):
db_path = str(tmp_path / "test.db")
monkeypatch.setattr("persistence.database.DB_PATH", db_path)
await init_db()
yield
@pytest.fixture
async def doc():
repo = SqliteDocumentRepository()
document = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
await repo.insert(document)
return document
@pytest.fixture
def repos():
return {
"chunks": SqliteChunkRepository(),
"edits": SqliteChunkEditRepository(),
"pushes": SqliteChunkPushRepository(),
"documents": SqliteDocumentRepository(),
"analyses": SqliteAnalysisRepository(),
}
@pytest.fixture
def service(repos):
return ChunkService(
chunk_repo=repos["chunks"],
chunk_edit_repo=repos["edits"],
chunk_push_repo=repos["pushes"],
document_repo=repos["documents"],
analysis_repo=repos["analyses"],
chunker=None,
ingestion_service=None,
)
# ---------------------------------------------------------------------------
# list_chunks
# ---------------------------------------------------------------------------
class TestListChunks:
async def test_list_empty(self, service, doc):
assert await service.list_chunks(doc.id) == []
async def test_list_filters_deleted(self, service, repos, doc):
a = Chunk(document_id=doc.id, sequence=0, text="alpha")
b = Chunk(document_id=doc.id, sequence=1, text="beta", deleted_at=datetime.now(UTC))
await repos["chunks"].insert_many([a, b])
chunks = await service.list_chunks(doc.id)
assert [c.id for c in chunks] == [a.id]
async def test_404_on_missing_doc(self, service):
with pytest.raises(DocumentNotFoundError):
await service.list_chunks("no-such")
# ---------------------------------------------------------------------------
# add_chunk
# ---------------------------------------------------------------------------
class TestAddChunk:
async def test_append_to_empty(self, service, repos, doc):
new = await service.add_chunk(doc.id, text="hello")
assert new.sequence == 0
chunks = await service.list_chunks(doc.id)
assert [c.text for c in chunks] == ["hello"]
edits = await repos["edits"].find_for_document(doc.id)
assert len(edits) == 1
assert edits[0].action == ChunkEditAction.INSERT
assert edits[0].after is not None
async def test_append_after_anchor_shifts_sequences(self, service, repos, doc):
first = await service.add_chunk(doc.id, text="a")
last = await service.add_chunk(doc.id, text="c")
middle = await service.add_chunk(doc.id, text="b", after_id=first.id)
chunks = await service.list_chunks(doc.id)
# Order should now be a, b, c
assert [c.text for c in chunks] == ["a", "b", "c"]
# Sequences must be dense ascending
assert [c.sequence for c in chunks] == [0, 1, 2]
assert chunks[1].id == middle.id
# `last` was shifted from sequence=1 to sequence=2
refreshed_last = next(c for c in chunks if c.id == last.id)
assert refreshed_last.sequence == 2
async def test_anchor_not_found(self, service, doc):
with pytest.raises(ChunkNotFoundError):
await service.add_chunk(doc.id, text="x", after_id="no-such")
# ---------------------------------------------------------------------------
# update_chunk
# ---------------------------------------------------------------------------
class TestUpdateChunk:
async def test_update_text_records_audit(self, service, repos, doc):
new = await service.add_chunk(doc.id, text="hi")
updated = await service.update_chunk(doc.id, new.id, text="hi there")
assert updated.text == "hi there"
edits = await repos["edits"].find_for_document(doc.id)
assert {e.action for e in edits} == {ChunkEditAction.INSERT, ChunkEditAction.UPDATE}
update_edit = next(e for e in edits if e.action == ChunkEditAction.UPDATE)
assert update_edit.before["text"] == "hi"
assert update_edit.after["text"] == "hi there"
async def test_404_on_missing_chunk(self, service, doc):
with pytest.raises(ChunkNotFoundError):
await service.update_chunk(doc.id, "no-such", text="x")
async def test_404_on_chunk_from_other_doc(self, service, repos, doc):
other = Document(id="doc-2", filename="o.pdf", storage_path="/tmp/o.pdf")
await repos["documents"].insert(other)
new = await service.add_chunk(other.id, text="x")
with pytest.raises(ChunkNotFoundError):
await service.update_chunk(doc.id, new.id, text="y")
# ---------------------------------------------------------------------------
# delete_chunk
# ---------------------------------------------------------------------------
class TestDeleteChunk:
async def test_soft_deletes_and_records_audit(self, service, repos, doc):
new = await service.add_chunk(doc.id, text="x")
await service.delete_chunk(doc.id, new.id)
# Soft-deleted: still in repo with deleted_at set, list filters it.
assert await service.list_chunks(doc.id) == []
edits = await repos["edits"].find_for_document(doc.id)
assert any(e.action == ChunkEditAction.DELETE for e in edits)
# ---------------------------------------------------------------------------
# split_chunk
# ---------------------------------------------------------------------------
class TestSplitChunk:
async def test_split_produces_two_chunks(self, service, repos, doc):
src = await service.add_chunk(doc.id, text="abcdef")
head, tail = await service.split_chunk(doc.id, src.id, cursor_offset=3)
assert head.text == "abc"
assert tail.text == "def"
chunks = await service.list_chunks(doc.id)
assert [c.text for c in chunks] == ["abc", "def"]
edits = await repos["edits"].find_for_document(doc.id)
split_edit = next(e for e in edits if e.action == ChunkEditAction.SPLIT)
assert split_edit.children == [head.id, tail.id]
assert split_edit.before is not None
async def test_split_400_on_offset_out_of_range(self, service, doc):
src = await service.add_chunk(doc.id, text="abc")
with pytest.raises(ChunkValidationError):
await service.split_chunk(doc.id, src.id, cursor_offset=0)
with pytest.raises(ChunkValidationError):
await service.split_chunk(doc.id, src.id, cursor_offset=99)
async def test_split_shifts_subsequent_chunks(self, service, doc):
a = await service.add_chunk(doc.id, text="abcdef")
await service.add_chunk(doc.id, text="next")
await service.split_chunk(doc.id, a.id, cursor_offset=3)
chunks = await service.list_chunks(doc.id)
# New head, new tail, then `next` at sequence 2
assert [c.text for c in chunks] == ["abc", "def", "next"]
assert [c.sequence for c in chunks] == [0, 1, 2]
# ---------------------------------------------------------------------------
# merge_chunks
# ---------------------------------------------------------------------------
class TestMergeChunks:
async def test_merge_contiguous(self, service, repos, doc):
a = await service.add_chunk(doc.id, text="a")
b = await service.add_chunk(doc.id, text="b")
await service.add_chunk(doc.id, text="c")
merged = await service.merge_chunks(doc.id, [b.id, a.id]) # order irrelevant
# Merged contains a,b
assert merged.text == "a\nb"
chunks = await service.list_chunks(doc.id)
# New chunk at sequence 0, then `c` still at 2 (gap is allowed)
assert {c2.text for c2 in chunks} == {"a\nb", "c"}
edit = next(
e
for e in await repos["edits"].find_for_document(doc.id)
if e.action == ChunkEditAction.MERGE
)
assert set(edit.parents) == {a.id, b.id}
async def test_merge_409_on_non_contiguous(self, service, doc):
a = await service.add_chunk(doc.id, text="a")
await service.add_chunk(doc.id, text="b")
c = await service.add_chunk(doc.id, text="c")
with pytest.raises(ChunkConflictError):
await service.merge_chunks(doc.id, [a.id, c.id])
async def test_merge_validates_min_two(self, service, doc):
a = await service.add_chunk(doc.id, text="a")
with pytest.raises(ChunkValidationError):
await service.merge_chunks(doc.id, [a.id])
# ---------------------------------------------------------------------------
# rechunk_document
# ---------------------------------------------------------------------------
class TestRechunkDocument:
async def test_rechunk_replaces_canonical(self, service, repos, doc):
# Seed an existing canonical chunk
await service.add_chunk(doc.id, text="old")
# Seed a completed analysis with document_json
job = AnalysisJob(document_id=doc.id, status=AnalysisStatus.COMPLETED)
await repos["analyses"].insert(job)
job.document_json = json.dumps({"texts": []})
job.completed_at = datetime.now(UTC)
await repos["analyses"].update_status(job)
chunker = MagicMock()
chunker.chunk = AsyncMock(
return_value=[
ChunkResult(text="new1", source_page=1, token_count=4),
ChunkResult(text="new2", source_page=2, token_count=4),
]
)
service._chunker = chunker
result = await service.rechunk_document(doc.id)
assert [c.text for c in result] == ["new1", "new2"]
chunks = await service.list_chunks(doc.id)
assert [c.text for c in chunks] == ["new1", "new2"]
async def test_rechunk_409_when_no_completed_analysis(self, service, doc):
service._chunker = MagicMock()
with pytest.raises(ChunkServiceError):
await service.rechunk_document(doc.id)
async def test_rechunk_503_when_no_chunker(self, service, doc):
with pytest.raises(ChunkServiceError) as exc:
await service.rechunk_document(doc.id)
assert exc.value.http_status == 503
# ---------------------------------------------------------------------------
# promote_from_analysis_if_empty
# ---------------------------------------------------------------------------
class TestPromote:
async def test_promotes_when_canonical_empty(self, service, repos, doc):
chunks_json = json.dumps(
[
{"text": "first", "headings": ["H"], "sourcePage": 1, "tokenCount": 2},
{"text": "second", "sourcePage": 2, "tokenCount": 3},
]
)
promoted = await service.promote_from_analysis_if_empty(doc.id, chunks_json)
assert promoted == 2
chunks = await service.list_chunks(doc.id)
assert [c.text for c in chunks] == ["first", "second"]
# Audit should record both INSERTs
edits = await repos["edits"].find_for_document(doc.id)
assert sum(1 for e in edits if e.action == ChunkEditAction.INSERT) == 2
async def test_idempotent_when_canonical_not_empty(self, service, doc):
await service.add_chunk(doc.id, text="manual")
promoted = await service.promote_from_analysis_if_empty(
doc.id, json.dumps([{"text": "auto"}])
)
assert promoted == 0
chunks = await service.list_chunks(doc.id)
assert [c.text for c in chunks] == ["manual"]
async def test_skips_invalid_json(self, service, doc):
promoted = await service.promote_from_analysis_if_empty(doc.id, "not-json")
assert promoted == 0
async def test_skips_deleted_chunks_from_analysis(self, service, doc):
chunks_json = json.dumps(
[
{"text": "keep"},
{"text": "drop", "deleted": True},
]
)
promoted = await service.promote_from_analysis_if_empty(doc.id, chunks_json)
assert promoted == 1
chunks = await service.list_chunks(doc.id)
assert [c.text for c in chunks] == ["keep"]
# ---------------------------------------------------------------------------
# diff_against_store
# ---------------------------------------------------------------------------
class TestDiff:
async def test_no_push_history_returns_all_added(self, service, doc):
c1 = await service.add_chunk(doc.id, text="a")
c2 = await service.add_chunk(doc.id, text="b")
diffs = await service.diff_against_store(doc.id, "store-1")
statuses = {d["chunkId"]: d["status"] for d in diffs}
assert statuses == {c1.id: "added", c2.id: "added"}
# ---------------------------------------------------------------------------
# get_tree
# ---------------------------------------------------------------------------
class TestGetTree:
async def test_tree_empty_when_no_analysis(self, service, doc):
assert await service.get_tree(doc.id) == []
async def test_tree_groups_by_label(self, service, repos, doc):
job = AnalysisJob(document_id=doc.id, status=AnalysisStatus.COMPLETED)
await repos["analyses"].insert(job)
job.document_json = json.dumps(
{
"texts": [
{"self_ref": "#/texts/0", "label": "title", "text": "Hi"},
{"self_ref": "#/texts/1", "label": "text", "text": "Body"},
],
"tables": [],
"pictures": [],
}
)
job.completed_at = datetime.now(UTC)
await repos["analyses"].update_status(job)
tree = await service.get_tree(doc.id)
labels = {n["type"] for n in tree}
assert labels == {"group"}
# Both Titles and Paragraphs groups should be present
group_titles = {n["label"] for n in tree}
assert "Titles" in group_titles
assert "Paragraphs" in group_titles

View file

@ -10,7 +10,7 @@ import pytest
from fastapi.testclient import TestClient
from api.schemas import ChunkBboxResponse, ChunkingOptionsRequest, ChunkResponse, RechunkRequest
from domain.models import AnalysisJob, AnalysisStatus
from domain.models import AnalysisJob, AnalysisStatus, Document
from domain.value_objects import ChunkBbox, ChunkingOptions, ChunkResult
from main import app
@ -529,6 +529,13 @@ class TestRemoteChunkingPath:
)
analysis_repo.find_by_id = AsyncMock(return_value=job)
analysis_repo.update_chunks = AsyncMock(return_value=True)
# rechunk() now drives a Document lifecycle transition (#202), so
# the document_repo must return a real Document and accept the
# update_lifecycle write.
document_repo.find_by_id = AsyncMock(
return_value=Document(id="d1", filename="test.pdf", storage_path="/tmp/test.pdf")
)
document_repo.update_lifecycle = AsyncMock()
chunks = await service.rechunk(
"j-remote",

View file

@ -0,0 +1,225 @@
"""Integration tests for the /api/documents/{id}/chunks router (#256)."""
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock
import pytest
from fastapi.testclient import TestClient
from domain.models import Chunk
from main import app
from services.chunk_service import (
ChunkConflictError,
ChunkNotFoundError,
ChunkValidationError,
DocumentNotFoundError,
)
@pytest.fixture
def client():
return TestClient(app, raise_server_exceptions=False)
@pytest.fixture
def mock_service(client):
svc = MagicMock()
original = getattr(app.state, "chunk_service", None)
app.state.chunk_service = svc
yield svc
app.state.chunk_service = original
def _chunk(*, id="c-1", doc_id="d-1", sequence=0, text="hi") -> Chunk:
return Chunk(id=id, document_id=doc_id, sequence=sequence, text=text)
class TestListChunks:
def test_200(self, client, mock_service):
mock_service.list_chunks = AsyncMock(return_value=[_chunk(id="c1"), _chunk(id="c2")])
resp = client.get("/api/documents/d-1/chunks")
assert resp.status_code == 200
data = resp.json()
assert [c["id"] for c in data] == ["c1", "c2"]
# camelCase serialization
assert "docId" in data[0]
assert "sourcePage" in data[0]
def test_404_when_doc_missing(self, client, mock_service):
mock_service.list_chunks = AsyncMock(side_effect=DocumentNotFoundError("nope"))
resp = client.get("/api/documents/no-such/chunks")
assert resp.status_code == 404
class TestAddChunk:
def test_201(self, client, mock_service):
mock_service.add_chunk = AsyncMock(return_value=_chunk(id="new"))
resp = client.post("/api/documents/d-1/chunks", json={"text": "hello"})
assert resp.status_code == 201
assert resp.json()["id"] == "new"
mock_service.add_chunk.assert_awaited_once_with("d-1", text="hello", after_id=None)
def test_400_on_empty_body(self, client, mock_service):
resp = client.post("/api/documents/d-1/chunks", json={"text": ""})
assert resp.status_code == 400
def test_passes_after_id(self, client, mock_service):
mock_service.add_chunk = AsyncMock(return_value=_chunk(id="new"))
client.post(
"/api/documents/d-1/chunks",
json={"text": "x", "afterId": "c-prev"},
)
mock_service.add_chunk.assert_awaited_once_with("d-1", text="x", after_id="c-prev")
class TestUpdateChunk:
def test_200(self, client, mock_service):
mock_service.update_chunk = AsyncMock(return_value=_chunk(text="new"))
resp = client.patch(
"/api/documents/d-1/chunks/c-1",
json={"text": "new"},
)
assert resp.status_code == 200
assert resp.json()["text"] == "new"
def test_400_when_no_field_set(self, client, mock_service):
resp = client.patch("/api/documents/d-1/chunks/c-1", json={})
assert resp.status_code == 400
def test_404_when_chunk_missing(self, client, mock_service):
mock_service.update_chunk = AsyncMock(side_effect=ChunkNotFoundError("nope"))
resp = client.patch("/api/documents/d-1/chunks/c-1", json={"text": "x"})
assert resp.status_code == 404
def test_title_maps_to_first_heading(self, client, mock_service):
mock_service.update_chunk = AsyncMock(return_value=_chunk())
client.patch("/api/documents/d-1/chunks/c-1", json={"title": "Section A"})
mock_service.update_chunk.assert_awaited_once_with(
"d-1", "c-1", text=None, headings=["Section A"]
)
class TestDeleteChunk:
def test_204(self, client, mock_service):
mock_service.delete_chunk = AsyncMock(return_value=None)
resp = client.delete("/api/documents/d-1/chunks/c-1")
assert resp.status_code == 204
def test_404(self, client, mock_service):
mock_service.delete_chunk = AsyncMock(side_effect=ChunkNotFoundError("nope"))
resp = client.delete("/api/documents/d-1/chunks/c-1")
assert resp.status_code == 404
class TestSplit:
def test_200(self, client, mock_service):
mock_service.split_chunk = AsyncMock(
return_value=[_chunk(id="head"), _chunk(id="tail", sequence=1)]
)
resp = client.post(
"/api/documents/d-1/chunks/c-1/split",
json={"cursorOffset": 3},
)
assert resp.status_code == 200
assert [c["id"] for c in resp.json()] == ["head", "tail"]
def test_400_on_invalid_offset(self, client, mock_service):
mock_service.split_chunk = AsyncMock(side_effect=ChunkValidationError("oor"))
resp = client.post(
"/api/documents/d-1/chunks/c-1/split",
json={"cursorOffset": 999},
)
assert resp.status_code == 400
class TestMerge:
def test_200(self, client, mock_service):
mock_service.merge_chunks = AsyncMock(return_value=_chunk(id="merged"))
resp = client.post(
"/api/documents/d-1/chunks/merge",
json={"ids": ["a", "b"]},
)
assert resp.status_code == 200
assert resp.json()["id"] == "merged"
def test_409_on_non_contiguous(self, client, mock_service):
mock_service.merge_chunks = AsyncMock(side_effect=ChunkConflictError("nope"))
resp = client.post(
"/api/documents/d-1/chunks/merge",
json={"ids": ["a", "z"]},
)
assert resp.status_code == 409
class TestRechunk:
def test_200_no_options(self, client, mock_service):
mock_service.rechunk_document = AsyncMock(return_value=[_chunk(id="r1")])
resp = client.post("/api/documents/d-1/rechunk", json={})
assert resp.status_code == 200
mock_service.rechunk_document.assert_awaited_once_with("d-1", None)
def test_200_with_options(self, client, mock_service):
mock_service.rechunk_document = AsyncMock(return_value=[])
resp = client.post(
"/api/documents/d-1/rechunk",
json={"chunkingOptions": {"chunkerType": "hybrid", "maxTokens": 256}},
)
assert resp.status_code == 200
call_args = mock_service.rechunk_document.await_args
assert call_args.args[0] == "d-1"
passed_opts = call_args.args[1]
# snake_case dump from Pydantic ChunkingOptionsRequest
assert passed_opts["chunker_type"] == "hybrid"
assert passed_opts["max_tokens"] == 256
class TestTree:
def test_200_empty(self, client, mock_service):
mock_service.get_tree = AsyncMock(return_value=[])
resp = client.get("/api/documents/d-1/tree")
assert resp.status_code == 200
assert resp.json() == []
def test_200_groups(self, client, mock_service):
mock_service.get_tree = AsyncMock(
return_value=[
{"ref": "#group/title", "type": "group", "label": "Titles", "children": []}
]
)
resp = client.get("/api/documents/d-1/tree")
assert resp.status_code == 200
data = resp.json()
assert data[0]["label"] == "Titles"
class TestDiff:
def test_200(self, client, mock_service):
mock_service.diff_against_store = AsyncMock(
return_value=[
{"chunkId": "c1", "status": "added", "textDiff": None},
]
)
resp = client.get("/api/documents/d-1/diff?store=mystore")
assert resp.status_code == 200
assert resp.json()[0]["chunkId"] == "c1"
def test_400_when_store_missing(self, client, mock_service):
resp = client.get("/api/documents/d-1/diff")
# FastAPI auto-422 on missing required query
assert resp.status_code in (400, 422)
class TestPush:
def test_200(self, client, mock_service):
mock_service.push_to_store = AsyncMock(
return_value={"jobId": "p1", "summary": {"embeds": 3, "tokens": 30}}
)
resp = client.post(
"/api/documents/d-1/chunks/push",
json={"store": "mystore"},
)
assert resp.status_code == 200
body = resp.json()
assert body["jobId"] == "p1"
assert body["summary"]["embeds"] == 3

View file

@ -0,0 +1,102 @@
"""Tests for the chunkset hash function (#204)."""
from __future__ import annotations
from domain.hashing import chunkset_hash
from domain.value_objects import ChunkBbox, ChunkDocItem, ChunkResult
def _chunk(text: str, *, page: int | None = 1, headings=()) -> ChunkResult:
return ChunkResult(text=text, headings=list(headings), source_page=page)
def test_empty_chunkset_returns_empty_sha256() -> None:
"""Empty input has a stable, well-known hash."""
h = chunkset_hash([])
# SHA-256 of nothing is the well-known constant.
assert h == "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
def test_determinism_across_invocations() -> None:
chunks = [_chunk("a"), _chunk("b"), _chunk("c")]
assert chunkset_hash(chunks) == chunkset_hash(chunks)
def test_text_change_changes_hash() -> None:
base = [_chunk("alpha")]
edited = [_chunk("alpha!")]
assert chunkset_hash(base) != chunkset_hash(edited)
def test_page_change_changes_hash() -> None:
a = [_chunk("alpha", page=1)]
b = [_chunk("alpha", page=2)]
assert chunkset_hash(a) != chunkset_hash(b)
def test_headings_change_changes_hash() -> None:
a = [_chunk("alpha", headings=("Section A",))]
b = [_chunk("alpha", headings=("Section B",))]
assert chunkset_hash(a) != chunkset_hash(b)
def test_excluded_fields_do_not_change_hash() -> None:
"""token_count, bboxes, doc_items are deliberately excluded."""
a = ChunkResult(text="alpha", headings=[], source_page=1, token_count=100, bboxes=[])
b = ChunkResult(
text="alpha",
headings=[],
source_page=1,
token_count=999, # different
bboxes=[ChunkBbox(page=1, bbox=[0, 0, 10, 10])], # different
doc_items=[ChunkDocItem(self_ref="#/x", label="text")], # different
)
assert chunkset_hash([a]) == chunkset_hash([b])
def test_join_attack_resistance() -> None:
"""Splitting one chunk into two with the same combined content must
produce a different hash from the original single-chunk version."""
one = [_chunk("HelloWorld")]
two = [_chunk("Hello"), _chunk("World")]
assert chunkset_hash(one) != chunkset_hash(two)
def test_order_matters() -> None:
a = [_chunk("alpha"), _chunk("bravo")]
b = [_chunk("bravo"), _chunk("alpha")]
assert chunkset_hash(a) != chunkset_hash(b)
def test_locked_fixture_three_chunks() -> None:
"""Locked fixture: a hand-built 3-chunk input has a fixed expected
hash. CI fails loud if anyone changes the canonical input list
without updating the fixture deliberately.
The expected hash below was computed once with the function as
pinned in this commit. To regenerate after a deliberate canonical
change, run:
python -c 'from domain.hashing import chunkset_hash; \\
from domain.value_objects import ChunkResult; \\
print(chunkset_hash([
ChunkResult(text="Intro paragraph.", source_page=1,
headings=["Intro"]),
ChunkResult(text="Body of section A.", source_page=2,
headings=["A"]),
ChunkResult(text="Conclusion.", source_page=3,
headings=["Conclusion"]),
]))'
"""
chunks = [
ChunkResult(text="Intro paragraph.", source_page=1, headings=["Intro"]),
ChunkResult(text="Body of section A.", source_page=2, headings=["A"]),
ChunkResult(text="Conclusion.", source_page=3, headings=["Conclusion"]),
]
expected = "6ac365ae403e53675e57884b69b0629684f2209c39730093231caa11a40e5225"
actual = chunkset_hash(chunks)
assert actual == expected, (
f"Hash drift detected. Expected {expected}, got {actual}. "
"If you intentionally changed the canonical inputs, update this "
"fixture and document the breaking change in the release notes."
)

View file

@ -0,0 +1,129 @@
"""Tests for the Document lifecycle state machine (#202)."""
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from domain.events import DocumentLifecycleChanged
from domain.exceptions import InvalidLifecycleTransitionError
from domain.lifecycle import assert_transition, is_allowed_transition
from domain.models import Document
from domain.value_objects import DocumentLifecycleState
ALLOWED_TRANSITIONS = [
# Uploaded
(DocumentLifecycleState.UPLOADED, DocumentLifecycleState.PARSED),
(DocumentLifecycleState.UPLOADED, DocumentLifecycleState.CHUNKED),
(DocumentLifecycleState.UPLOADED, DocumentLifecycleState.FAILED),
# Parsed
(DocumentLifecycleState.PARSED, DocumentLifecycleState.PARSED),
(DocumentLifecycleState.PARSED, DocumentLifecycleState.CHUNKED),
(DocumentLifecycleState.PARSED, DocumentLifecycleState.FAILED),
# Chunked
(DocumentLifecycleState.CHUNKED, DocumentLifecycleState.CHUNKED),
(DocumentLifecycleState.CHUNKED, DocumentLifecycleState.INGESTED),
(DocumentLifecycleState.CHUNKED, DocumentLifecycleState.FAILED),
# Ingested
(DocumentLifecycleState.INGESTED, DocumentLifecycleState.STALE),
(DocumentLifecycleState.INGESTED, DocumentLifecycleState.CHUNKED),
(DocumentLifecycleState.INGESTED, DocumentLifecycleState.INGESTED),
(DocumentLifecycleState.INGESTED, DocumentLifecycleState.FAILED),
# Stale
(DocumentLifecycleState.STALE, DocumentLifecycleState.INGESTED),
(DocumentLifecycleState.STALE, DocumentLifecycleState.CHUNKED),
(DocumentLifecycleState.STALE, DocumentLifecycleState.FAILED),
# Failed (recoverable)
(DocumentLifecycleState.FAILED, DocumentLifecycleState.UPLOADED),
(DocumentLifecycleState.FAILED, DocumentLifecycleState.PARSED),
(DocumentLifecycleState.FAILED, DocumentLifecycleState.CHUNKED),
]
@pytest.mark.parametrize(("source", "target"), ALLOWED_TRANSITIONS)
def test_allowed_transitions_pass(
source: DocumentLifecycleState, target: DocumentLifecycleState
) -> None:
assert is_allowed_transition(source, target)
# assert_transition does not raise.
assert_transition(source, target)
def test_disallowed_transitions_are_rejected() -> None:
"""Spot-check the most surprising disallowed pairs.
Exhaustive enumeration of every disallowed pair would just mirror the
transition table; this list captures the cases a reader is most
likely to argue about.
"""
forbidden = [
# Uploaded cannot skip directly to Ingested or Stale.
(DocumentLifecycleState.UPLOADED, DocumentLifecycleState.INGESTED),
(DocumentLifecycleState.UPLOADED, DocumentLifecycleState.STALE),
# Parsed cannot become Ingested without going through Chunked.
(DocumentLifecycleState.PARSED, DocumentLifecycleState.INGESTED),
# Stale is not directly reachable from Parsed (no per-store yet).
(DocumentLifecycleState.PARSED, DocumentLifecycleState.STALE),
# Failed cannot be reached as a "self-loop" — it always reflects a
# new failure, never an idempotent re-mark.
(DocumentLifecycleState.FAILED, DocumentLifecycleState.FAILED),
# Failed cannot jump directly to Ingested or Stale — must re-do
# the relevant pipeline step first.
(DocumentLifecycleState.FAILED, DocumentLifecycleState.INGESTED),
(DocumentLifecycleState.FAILED, DocumentLifecycleState.STALE),
]
for source, target in forbidden:
assert not is_allowed_transition(source, target)
with pytest.raises(InvalidLifecycleTransitionError) as excinfo:
assert_transition(source, target)
assert excinfo.value.source == source
assert excinfo.value.target == target
def test_document_default_state_is_uploaded() -> None:
doc = Document(filename="test.pdf", storage_path="/tmp/test.pdf")
assert doc.lifecycle_state == DocumentLifecycleState.UPLOADED
assert doc.lifecycle_state_at is None
def test_transition_returns_event_and_mutates_state() -> None:
doc = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
before = datetime(2026, 4, 29, 10, tzinfo=UTC)
event = doc.transition_to(DocumentLifecycleState.PARSED, now=before)
assert isinstance(event, DocumentLifecycleChanged)
assert event.document_id == "doc-1"
assert event.previous == DocumentLifecycleState.UPLOADED
assert event.current == DocumentLifecycleState.PARSED
assert event.at == before
# State mutated on the dataclass.
assert doc.lifecycle_state == DocumentLifecycleState.PARSED
assert doc.lifecycle_state_at == before
def test_invalid_transition_does_not_mutate_state() -> None:
doc = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
assert doc.lifecycle_state == DocumentLifecycleState.UPLOADED
with pytest.raises(InvalidLifecycleTransitionError):
doc.transition_to(DocumentLifecycleState.STALE)
# State is untouched.
assert doc.lifecycle_state == DocumentLifecycleState.UPLOADED
assert doc.lifecycle_state_at is None
def test_idempotent_self_transitions_emit_event() -> None:
"""Self-loops (re-parse, re-chunk, re-push) are explicitly allowed
so the pipeline can drive transitions without checking the source
state first. They still emit an event for observability."""
doc = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
doc.transition_to(DocumentLifecycleState.CHUNKED)
assert doc.lifecycle_state == DocumentLifecycleState.CHUNKED
event = doc.transition_to(DocumentLifecycleState.CHUNKED)
assert event.previous == DocumentLifecycleState.CHUNKED
assert event.current == DocumentLifecycleState.CHUNKED

View file

@ -0,0 +1,72 @@
"""Tests for the document lifecycle aggregation rule (#203)."""
from __future__ import annotations
from domain.lifecycle_aggregation import aggregate_lifecycle
from domain.models import DocumentStoreLink
from domain.value_objects import DocumentLifecycleState, DocumentStoreLinkState
def _link(state: DocumentStoreLinkState) -> DocumentStoreLink:
return DocumentStoreLink(
id=f"link-{state.value}",
document_id="doc-1",
store_id=f"store-{state.value}",
state=state,
)
def test_no_links_returns_fallback() -> None:
assert (
aggregate_lifecycle([], fallback=DocumentLifecycleState.CHUNKED)
== DocumentLifecycleState.CHUNKED
)
def test_single_ingested_link_makes_doc_ingested() -> None:
assert (
aggregate_lifecycle(
[_link(DocumentStoreLinkState.INGESTED)],
fallback=DocumentLifecycleState.CHUNKED,
)
== DocumentLifecycleState.INGESTED
)
def test_any_stale_link_makes_doc_stale() -> None:
"""Stale outranks Ingested — a doc with one stale store is considered stale."""
assert (
aggregate_lifecycle(
[
_link(DocumentStoreLinkState.INGESTED),
_link(DocumentStoreLinkState.STALE),
],
fallback=DocumentLifecycleState.CHUNKED,
)
== DocumentLifecycleState.STALE
)
def test_any_failed_link_makes_doc_failed() -> None:
"""Failed outranks every other link state."""
assert (
aggregate_lifecycle(
[
_link(DocumentStoreLinkState.INGESTED),
_link(DocumentStoreLinkState.STALE),
_link(DocumentStoreLinkState.FAILED),
],
fallback=DocumentLifecycleState.CHUNKED,
)
== DocumentLifecycleState.FAILED
)
def test_only_failed_link_makes_doc_failed() -> None:
assert (
aggregate_lifecycle(
[_link(DocumentStoreLinkState.FAILED)],
fallback=DocumentLifecycleState.CHUNKED,
)
== DocumentLifecycleState.FAILED
)

View file

@ -0,0 +1,185 @@
"""Tests for the 0.6.0 migration script (#206)."""
from __future__ import annotations
import json
import pytest
from domain.value_objects import DocumentLifecycleState, DocumentStoreLinkState
from persistence.chunk_repo import SqliteChunkRepository
from persistence.database import get_connection, init_db
from persistence.document_repo import SqliteDocumentRepository
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from tools.migrate_06 import _DocFacts, _infer_lifecycle, run
@pytest.fixture(autouse=True)
async def setup_db(monkeypatch, tmp_path):
db_path = str(tmp_path / "test.db")
monkeypatch.setattr("persistence.database.DB_PATH", db_path)
await init_db()
yield
# ---------------------------------------------------------------------------
# Inference rule unit tests
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
("has_completed", "has_failed", "has_chunks_json", "expected"),
[
(False, False, False, DocumentLifecycleState.UPLOADED),
(False, True, False, DocumentLifecycleState.FAILED),
(True, False, False, DocumentLifecycleState.PARSED),
(True, False, True, DocumentLifecycleState.CHUNKED),
(True, True, True, DocumentLifecycleState.CHUNKED), # any completed wins
],
)
def test_infer_lifecycle(
has_completed: bool,
has_failed: bool,
has_chunks_json: bool,
expected: DocumentLifecycleState,
) -> None:
facts = _DocFacts(
document_id="doc",
has_completed=has_completed,
has_failed=has_failed,
has_chunks_json=has_chunks_json,
)
assert _infer_lifecycle(facts) == expected
# ---------------------------------------------------------------------------
# Integration: end-to-end migration on a hand-built pre-state
# ---------------------------------------------------------------------------
async def _seed_pre_06_state() -> None:
"""Seed a tenant snapshot with three documents covering the three
main inference cases:
doc-uploaded only an `Uploaded` doc row, nothing else
doc-parsed completed analysis, no chunks
doc-chunked completed analysis with chunks_json
"""
repo = SqliteDocumentRepository()
async with get_connection() as db:
await db.execute(
"INSERT INTO documents (id, filename, storage_path, created_at) "
"VALUES ('doc-uploaded', 'a.pdf', '/tmp/a.pdf', datetime('now'))"
)
await db.execute(
"INSERT INTO documents (id, filename, storage_path, created_at) "
"VALUES ('doc-parsed', 'b.pdf', '/tmp/b.pdf', datetime('now'))"
)
await db.execute(
"INSERT INTO documents (id, filename, storage_path, created_at) "
"VALUES ('doc-chunked', 'c.pdf', '/tmp/c.pdf', datetime('now'))"
)
await db.execute(
"INSERT INTO analysis_jobs (id, document_id, status) "
"VALUES ('a-parsed', 'doc-parsed', 'COMPLETED')"
)
chunks_json = json.dumps(
[
{"text": "Intro paragraph.", "headings": ["Intro"], "sourcePage": 1},
{"text": "Body of section A.", "headings": ["A"], "sourcePage": 2},
]
)
await db.execute(
"INSERT INTO analysis_jobs (id, document_id, status, chunks_json) "
"VALUES ('a-chunked', 'doc-chunked', 'COMPLETED', ?)",
(chunks_json,),
)
await db.commit()
# Touch the document repo so the type checker is satisfied.
assert await repo.find_by_id("doc-chunked") is not None
async def test_full_migration_writes_expected_state() -> None:
await _seed_pre_06_state()
results = await run(dry_run=False)
by_step = {r.name: r for r in results}
assert by_step["backfill_lifecycle"].written >= 2 # parsed + chunked transition
assert by_step["materialize_chunks"].written == 2 # two chunks
assert by_step["backfill_links"].written == 1 # only doc-chunked has chunks
repo = SqliteDocumentRepository()
chunks_repo = SqliteChunkRepository()
link_repo = SqliteDocumentStoreLinkRepository()
uploaded = await repo.find_by_id("doc-uploaded")
assert uploaded is not None
assert uploaded.lifecycle_state == DocumentLifecycleState.UPLOADED
parsed = await repo.find_by_id("doc-parsed")
assert parsed is not None
assert parsed.lifecycle_state == DocumentLifecycleState.PARSED
chunked = await repo.find_by_id("doc-chunked")
assert chunked is not None
# After link backfill + reaggregation, doc-chunked is Ingested in default store.
assert chunked.lifecycle_state == DocumentLifecycleState.INGESTED
chunks = await chunks_repo.find_for_document("doc-chunked")
assert len(chunks) == 2
assert chunks[0].text == "Intro paragraph."
assert chunks[0].sequence == 0
links = await link_repo.find_for_document("doc-chunked")
assert len(links) == 1
assert links[0].store_id == "default"
assert links[0].state == DocumentStoreLinkState.INGESTED
assert links[0].chunkset_hash is not None
async def test_migration_is_idempotent() -> None:
await _seed_pre_06_state()
first = await run(dry_run=False)
second = await run(dry_run=False)
# Second run sees every step as already-done.
assert all(r.skipped for r in second)
assert all(not r.skipped for r in first)
async def test_dry_run_writes_nothing() -> None:
await _seed_pre_06_state()
repo = SqliteDocumentRepository()
chunks_repo = SqliteChunkRepository()
await run(dry_run=True)
# No state changed.
parsed = await repo.find_by_id("doc-parsed")
assert parsed is not None
assert parsed.lifecycle_state == DocumentLifecycleState.UPLOADED # unchanged
chunked_chunks = await chunks_repo.find_for_document("doc-chunked")
assert chunked_chunks == []
async def test_chunk_ids_are_deterministic_across_runs() -> None:
"""Stable id derivation lets a re-run after a manual reset land on
the same chunk ids important for the link-table foreign keys."""
await _seed_pre_06_state()
chunks_repo = SqliteChunkRepository()
await run(dry_run=False, only_step="materialize_chunks")
first_ids = sorted(c.id for c in await chunks_repo.find_for_document("doc-chunked"))
# Wipe progress + chunks, run again — same ids.
async with get_connection() as db:
await db.execute("DELETE FROM migration_progress WHERE name = 'materialize_chunks'")
await db.execute("DELETE FROM chunks WHERE document_id = 'doc-chunked'")
await db.commit()
await run(dry_run=False, only_step="materialize_chunks")
second_ids = sorted(c.id for c in await chunks_repo.find_for_document("doc-chunked"))
assert first_ids == second_ids

View file

@ -1,10 +1,11 @@
"""Tests for persistence repositories using a temporary SQLite database."""
from datetime import datetime
from datetime import UTC, datetime
import pytest
from domain.models import AnalysisJob, AnalysisStatus, Document
from domain.value_objects import DocumentLifecycleState
from persistence.analysis_repo import SqliteAnalysisRepository
from persistence.database import init_db
from persistence.document_repo import SqliteDocumentRepository
@ -81,6 +82,46 @@ class TestDocumentRepo:
deleted = await document_repo.delete("nonexistent")
assert deleted is False
async def test_default_lifecycle_state_is_uploaded(self, document_repo):
"""Fresh document round-trip preserves the default Uploaded state."""
doc = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
await document_repo.insert(doc)
found = await document_repo.find_by_id("doc-1")
assert found is not None
assert found.lifecycle_state == DocumentLifecycleState.UPLOADED
assert found.lifecycle_state_at is None
async def test_update_lifecycle_persists_state_and_timestamp(self, document_repo):
doc = Document(id="doc-1", filename="t.pdf", storage_path="/tmp/t.pdf")
await document_repo.insert(doc)
when = datetime(2026, 4, 29, 12, 0, tzinfo=UTC)
await document_repo.update_lifecycle("doc-1", DocumentLifecycleState.PARSED, when)
found = await document_repo.find_by_id("doc-1")
assert found is not None
assert found.lifecycle_state == DocumentLifecycleState.PARSED
assert found.lifecycle_state_at is not None
assert found.lifecycle_state_at == when
async def test_lifecycle_state_round_trips_for_each_value(self, document_repo):
"""Every enum value must serialize cleanly into and out of SQLite."""
when = datetime(2026, 4, 29, 12, 0, tzinfo=UTC)
for value in DocumentLifecycleState:
doc = Document(
id=f"doc-{value.value}",
filename="t.pdf",
storage_path="/tmp/t.pdf",
lifecycle_state=value,
lifecycle_state_at=when,
)
await document_repo.insert(doc)
found = await document_repo.find_by_id(f"doc-{value.value}")
assert found is not None
assert found.lifecycle_state == value
class TestAnalysisRepo:
async def _insert_doc(self, document_repo):

View file

@ -0,0 +1,260 @@
"""Tests for the Store and DocumentStoreLink repositories (#203)."""
from __future__ import annotations
from datetime import UTC, datetime
import pytest
from domain.models import Document, DocumentStoreLink, Store
from domain.value_objects import DocumentStoreLinkState, StoreKind
from persistence.database import init_db
from persistence.document_repo import SqliteDocumentRepository
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from persistence.store_repo import SqliteStoreRepository
@pytest.fixture(autouse=True)
async def setup_db(monkeypatch, tmp_path):
db_path = str(tmp_path / "test.db")
monkeypatch.setattr("persistence.database.DB_PATH", db_path)
await init_db()
yield
@pytest.fixture
def store_repo():
return SqliteStoreRepository()
@pytest.fixture
def link_repo():
return SqliteDocumentStoreLinkRepository()
@pytest.fixture
def document_repo():
return SqliteDocumentRepository()
class TestStoreRepo:
async def test_default_store_seeded_on_init(self, store_repo):
"""init_db() must seed exactly one `default` store on a fresh DB."""
default = await store_repo.get_default()
assert default is not None
assert default.slug == "default"
assert default.is_default is True
assert default.kind == StoreKind.OPENSEARCH
async def test_seed_is_idempotent(self, store_repo):
"""Re-running init_db() must not create a second default store."""
await init_db()
await init_db()
all_stores = await store_repo.find_all()
slugs = [s.slug for s in all_stores]
assert slugs.count("default") == 1
async def test_insert_and_find_by_slug(self, store_repo):
store = Store(
id="s-rh",
name="rh-corpus-v3",
slug="rh-corpus-v3",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={"index_name": "rh-corpus-v3"},
)
await store_repo.insert(store)
found = await store_repo.find_by_slug("rh-corpus-v3")
assert found is not None
assert found.embedder == "bge-m3"
assert found.config == {"index_name": "rh-corpus-v3"}
async def test_find_all_orders_default_first(self, store_repo):
await store_repo.insert(
Store(
id="s-rh",
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
)
)
await store_repo.insert(
Store(
id="s-legal",
name="legal",
slug="legal",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
)
)
all_stores = await store_repo.find_all()
assert all_stores[0].slug == "default" # seeded default comes first
async def test_find_by_name(self, store_repo):
await store_repo.insert(
Store(
id="s-rh",
name="rh-corpus",
slug="rh-corpus",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
)
)
found = await store_repo.find_by_name("rh-corpus")
assert found is not None
assert found.slug == "rh-corpus"
assert await store_repo.find_by_name("missing") is None
async def test_update_replaces_mutable_fields(self, store_repo):
await store_repo.insert(
Store(
id="s-rh",
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={"index_name": "rh"},
)
)
store = await store_repo.find_by_id("s-rh")
assert store is not None
store.embedder = "bge-large"
store.config = {"index_name": "rh-v2"}
store.name = "rh-v2"
await store_repo.update(store)
reloaded = await store_repo.find_by_id("s-rh")
assert reloaded is not None
assert reloaded.embedder == "bge-large"
assert reloaded.config == {"index_name": "rh-v2"}
assert reloaded.name == "rh-v2"
async def test_clear_default_except_promotes_single_winner(self, store_repo):
# Seeded default is the original; insert another candidate as default.
await store_repo.insert(
Store(
id="s-new",
name="new",
slug="new",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
is_default=True,
)
)
await store_repo.clear_default_except("s-new")
defaults = [s for s in await store_repo.find_all() if s.is_default]
assert len(defaults) == 1
assert defaults[0].id == "s-new"
async def test_delete_returns_true_when_removed(self, store_repo):
await store_repo.insert(
Store(
id="s-tmp",
name="tmp",
slug="tmp",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
)
)
assert await store_repo.delete("s-tmp") is True
assert await store_repo.find_by_id("s-tmp") is None
assert await store_repo.delete("s-tmp") is False
class TestDocumentStoreLinkRepo:
async def _seed_doc(self, document_repo, doc_id: str = "doc-1") -> Document:
doc = Document(id=doc_id, filename="t.pdf", storage_path="/tmp/t.pdf")
await document_repo.insert(doc)
return doc
async def test_upsert_creates_then_updates(self, link_repo, document_repo):
await self._seed_doc(document_repo)
link = DocumentStoreLink(
id="l-1",
document_id="doc-1",
store_id="default",
state=DocumentStoreLinkState.INGESTED,
)
await link_repo.upsert(link)
# Update same (doc, store) — no second row.
link.mark_ingested(
hash_="abc123",
at=datetime(2026, 4, 29, 12, tzinfo=UTC),
run_id="run-1",
)
await link_repo.upsert(link)
all_for_doc = await link_repo.find_for_document("doc-1")
assert len(all_for_doc) == 1
assert all_for_doc[0].chunkset_hash == "abc123"
assert all_for_doc[0].last_run_id == "run-1"
async def test_unique_constraint_on_doc_store_pair(self, link_repo, document_repo):
"""Two links with the same (doc, store) collapse to one via upsert."""
await self._seed_doc(document_repo)
await link_repo.upsert(
DocumentStoreLink(
id="l-1",
document_id="doc-1",
store_id="default",
state=DocumentStoreLinkState.INGESTED,
)
)
await link_repo.upsert(
DocumentStoreLink(
id="l-2",
document_id="doc-1",
store_id="default",
state=DocumentStoreLinkState.STALE,
)
)
rows = await link_repo.find_for_document("doc-1")
assert len(rows) == 1
assert rows[0].state == DocumentStoreLinkState.STALE
async def test_find_one_returns_none_when_absent(self, link_repo, document_repo):
await self._seed_doc(document_repo)
assert await link_repo.find_one("doc-1", "default") is None
async def test_cascade_delete_when_doc_removed(self, link_repo, document_repo):
await self._seed_doc(document_repo)
await link_repo.upsert(
DocumentStoreLink(
id="l-1",
document_id="doc-1",
store_id="default",
state=DocumentStoreLinkState.INGESTED,
)
)
await document_repo.delete("doc-1")
rows = await link_repo.find_for_document("doc-1")
assert rows == []
async def test_state_round_trips(self, link_repo, document_repo):
await self._seed_doc(document_repo)
for state in DocumentStoreLinkState:
link = DocumentStoreLink(
id=f"l-{state.value}",
document_id="doc-1",
store_id=f"store-{state.value}",
state=state,
)
# Need a store row for FK; seed minimal stores.
from persistence.store_repo import SqliteStoreRepository
await SqliteStoreRepository().insert(
Store(
id=f"store-{state.value}",
name=f"s-{state.value}",
slug=f"s-{state.value}",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
)
)
await link_repo.upsert(link)
found = await link_repo.find_one("doc-1", f"store-{state.value}")
assert found is not None
assert found.state == state

View file

@ -0,0 +1,283 @@
"""Tests for StoreService — CRUD + validations (#251)."""
from __future__ import annotations
import pytest
from domain.models import Document, DocumentStoreLink
from domain.value_objects import DocumentStoreLinkState, StoreKind
from persistence.database import init_db
from persistence.document_repo import SqliteDocumentRepository
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from persistence.store_repo import SqliteStoreRepository
from services.store_service import (
StoreConflictError,
StoreNotFoundError,
StoreService,
StoreValidationError,
)
@pytest.fixture(autouse=True)
async def setup_db(monkeypatch, tmp_path):
db_path = str(tmp_path / "test.db")
monkeypatch.setattr("persistence.database.DB_PATH", db_path)
await init_db()
yield
@pytest.fixture
def service():
return StoreService(SqliteStoreRepository(), SqliteDocumentStoreLinkRepository())
VALID_OS_CONFIG = {"index_name": "rh-corpus-v3"}
class TestCreate:
async def test_create_ok(self, service):
store = await service.create_store(
name="rh-corpus",
slug="rh-corpus",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
assert store.id
assert store.slug == "rh-corpus"
async def test_slug_normalized_to_lowercase(self, service):
store = await service.create_store(
name="RH",
slug="RH-Corpus",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
assert store.slug == "rh-corpus"
async def test_create_rejects_duplicate_slug(self, service):
await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
with pytest.raises(StoreConflictError):
await service.create_store(
name="rh-2",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
async def test_create_rejects_duplicate_name(self, service):
await service.create_store(
name="rh",
slug="rh-1",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
with pytest.raises(StoreConflictError):
await service.create_store(
name="rh",
slug="rh-2",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
async def test_create_rejects_bad_slug(self, service):
with pytest.raises(StoreValidationError):
await service.create_store(
name="rh",
slug="RH Corpus",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
async def test_create_rejects_missing_index_name(self, service):
with pytest.raises(StoreValidationError):
await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={},
)
async def test_create_neo4j_ok(self, service):
store = await service.create_store(
name="kg",
slug="kg",
kind=StoreKind.NEO4J,
embedder="bge-m3",
config={"index_name": "chunks-vec", "database": "neo4j"},
)
assert store.kind is StoreKind.NEO4J
assert store.config["index_name"] == "chunks-vec"
async def test_create_neo4j_rejects_missing_index_name(self, service):
with pytest.raises(StoreValidationError):
await service.create_store(
name="kg",
slug="kg",
kind=StoreKind.NEO4J,
embedder="bge-m3",
config={},
)
async def test_create_default_clears_others(self, service):
# The seeded 'default' store starts as is_default=True.
new_default = await service.create_store(
name="new",
slug="new",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={"index_name": "new"},
is_default=True,
)
all_stores = await service.list_stores()
defaults = [s for s in all_stores if s.is_default]
assert len(defaults) == 1
assert defaults[0].slug == new_default.slug
class TestRead:
async def test_get_by_slug_404(self, service):
with pytest.raises(StoreNotFoundError):
await service.get_by_slug("missing")
async def test_list_includes_seeded_default(self, service):
stores = await service.list_stores()
slugs = [s.slug for s in stores]
assert "default" in slugs
async def test_list_counts_linked_documents(self, service):
# Seed a doc + a link to the seeded default store.
doc_repo = SqliteDocumentRepository()
await doc_repo.insert(Document(id="d-1", filename="t.pdf", storage_path="/tmp/t.pdf"))
await SqliteDocumentStoreLinkRepository().upsert(
DocumentStoreLink(
id="l-1",
document_id="d-1",
store_id="default",
state=DocumentStoreLinkState.INGESTED,
)
)
views = await service.list_stores()
default_view = next(v for v in views if v.slug == "default")
assert default_view.document_count == 1
class TestUpdate:
async def test_partial_update(self, service):
await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
updated = await service.update_store("rh", embedder="bge-large")
assert updated.embedder == "bge-large"
# Other fields untouched.
assert updated.name == "rh"
async def test_update_rename_slug(self, service):
await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
updated = await service.update_store("rh", new_slug="rh-v2")
assert updated.slug == "rh-v2"
with pytest.raises(StoreNotFoundError):
await service.get_by_slug("rh")
async def test_update_rejects_slug_collision(self, service):
await service.create_store(
name="a",
slug="a",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={"index_name": "a"},
)
await service.create_store(
name="b",
slug="b",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={"index_name": "b"},
)
with pytest.raises(StoreConflictError):
await service.update_store("a", new_slug="b")
async def test_update_promote_default_demotes_others(self, service):
await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
await service.update_store("rh", is_default=True)
defaults = [s for s in await service.list_stores() if s.is_default]
assert len(defaults) == 1
assert defaults[0].slug == "rh"
async def test_update_rejects_invalid_config(self, service):
await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
with pytest.raises(StoreValidationError):
await service.update_store("rh", config={})
class TestDelete:
async def test_delete_ok(self, service):
await service.create_store(
name="tmp",
slug="tmp",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config={"index_name": "tmp"},
)
await service.delete_store("tmp")
with pytest.raises(StoreNotFoundError):
await service.get_by_slug("tmp")
async def test_delete_seeded_default_refused(self, service):
with pytest.raises(StoreConflictError):
await service.delete_store("default")
async def test_delete_with_links_refused(self, service):
store = await service.create_store(
name="rh",
slug="rh",
kind=StoreKind.OPENSEARCH,
embedder="bge-m3",
config=VALID_OS_CONFIG,
)
doc_repo = SqliteDocumentRepository()
await doc_repo.insert(Document(id="d-1", filename="t.pdf", storage_path="/tmp/t.pdf"))
await SqliteDocumentStoreLinkRepository().upsert(
DocumentStoreLink(
id="l-1",
document_id="d-1",
store_id=store.id,
state=DocumentStoreLinkState.INGESTED,
)
)
with pytest.raises(StoreConflictError):
await service.delete_store("rh")

View file

@ -0,0 +1,433 @@
"""0.6.0 migration — backfill the doc-centric data model.
Run after deploying the 0.6.0 release code, before sending traffic. The
script is idempotent and resumable re-running on an already-migrated
database is a no-op. Each step records itself in `migration_progress`.
Inference rules for `documents.lifecycle_state`:
has any FAILED analysis_jobs row, no completed Failed
has any COMPLETED analysis_jobs row + chunks_json Chunked
has any COMPLETED analysis_jobs row, no chunks_json Parsed
else Uploaded
After chunks are materialized into the new `chunks` table and links are
backfilled (one per existing default-store ingestion), the document
state is recomputed using the standard #203 aggregation rule. This
covers Stale / Ingested upgrades where applicable.
Usage:
python -m tools.migrate_06 [--dry-run] [--only-step <name>]
The CLI is intentionally minimal see docs/design/206-lifecycle-state-migration.md
for the full set of flags considered.
"""
from __future__ import annotations
import argparse
import asyncio
import json
import logging
import sys
import uuid
from dataclasses import dataclass
from typing import TYPE_CHECKING
from domain.hashing import chunkset_hash
if TYPE_CHECKING:
from collections.abc import Callable, Iterable
import aiosqlite
from domain.lifecycle_aggregation import aggregate_lifecycle
from domain.models import Chunk, DocumentStoreLink
from domain.value_objects import (
ChunkBbox,
ChunkDocItem,
ChunkResult,
DocumentLifecycleState,
DocumentStoreLinkState,
)
from persistence.chunk_repo import SqliteChunkRepository
from persistence.database import get_connection, init_db
from persistence.document_store_link_repo import SqliteDocumentStoreLinkRepository
from persistence.store_repo import SqliteStoreRepository
logger = logging.getLogger("migrate_06")
# ---------------------------------------------------------------------------
# Progress tracking
# ---------------------------------------------------------------------------
async def _is_done(db: aiosqlite.Connection, name: str) -> bool:
cursor = await db.execute("SELECT 1 FROM migration_progress WHERE name = ?", (name,))
return await cursor.fetchone() is not None
async def _mark_done(db: aiosqlite.Connection, name: str) -> None:
await db.execute(
"INSERT OR IGNORE INTO migration_progress (name) VALUES (?)",
(name,),
)
await db.commit()
# ---------------------------------------------------------------------------
# Step: backfill_document_lifecycle_state
# ---------------------------------------------------------------------------
@dataclass
class _DocFacts:
document_id: str
has_completed: bool
has_failed: bool
has_chunks_json: bool
async def _gather_document_facts(db: aiosqlite.Connection) -> list[_DocFacts]:
"""One row per document with the analysis-state booleans we care about."""
cursor = await db.execute(
"""SELECT d.id AS document_id,
MAX(CASE WHEN a.status = 'COMPLETED' THEN 1 ELSE 0 END) AS has_completed,
MAX(CASE WHEN a.status = 'FAILED' THEN 1 ELSE 0 END) AS has_failed,
MAX(CASE WHEN a.chunks_json IS NOT NULL AND a.chunks_json != ''
THEN 1 ELSE 0 END) AS has_chunks_json
FROM documents d
LEFT JOIN analysis_jobs a ON a.document_id = d.id
GROUP BY d.id"""
)
rows = await cursor.fetchall()
return [
_DocFacts(
document_id=r["document_id"],
has_completed=bool(r["has_completed"]),
has_failed=bool(r["has_failed"]),
has_chunks_json=bool(r["has_chunks_json"]),
)
for r in rows
]
def _infer_lifecycle(facts: _DocFacts) -> DocumentLifecycleState:
if facts.has_completed and facts.has_chunks_json:
return DocumentLifecycleState.CHUNKED
if facts.has_completed:
return DocumentLifecycleState.PARSED
if facts.has_failed:
return DocumentLifecycleState.FAILED
return DocumentLifecycleState.UPLOADED
async def _step_backfill_lifecycle(db: aiosqlite.Connection, *, dry_run: bool) -> int:
facts = await _gather_document_facts(db)
written = 0
for f in facts:
target = _infer_lifecycle(f)
cursor = await db.execute(
"SELECT lifecycle_state FROM documents WHERE id = ?",
(f.document_id,),
)
row = await cursor.fetchone()
if row and row["lifecycle_state"] == target.value:
continue
logger.info(
"lifecycle: doc=%s -> %s",
f.document_id,
target.value,
)
if not dry_run:
await db.execute(
"UPDATE documents SET lifecycle_state = ?, "
"lifecycle_state_at = datetime('now') WHERE id = ?",
(target.value, f.document_id),
)
written += 1
if not dry_run:
await db.commit()
return written
# ---------------------------------------------------------------------------
# Step: materialize_chunks_from_chunks_json
# ---------------------------------------------------------------------------
def _stable_chunk_id(document_id: str, sequence: int, text: str) -> str:
"""Deterministic id so re-running the migration doesn't duplicate rows."""
seed = f"{document_id}|{sequence}|{text}"
return uuid.uuid5(uuid.NAMESPACE_OID, seed).hex
def _chunk_dicts_for_doc(
db: aiosqlite.Connection,
) -> Callable[[str], asyncio.Future[list[dict]]]:
"""Return an async-compatible accessor over the latest chunks_json
for a document. Returns an empty list when no chunks_json is available.
"""
async def _read(document_id: str) -> list[dict]:
cursor = await db.execute(
"""SELECT chunks_json FROM analysis_jobs
WHERE document_id = ? AND chunks_json IS NOT NULL AND chunks_json != ''
ORDER BY completed_at DESC, created_at DESC
LIMIT 1""",
(document_id,),
)
row = await cursor.fetchone()
if row is None:
return []
try:
return list(json.loads(row["chunks_json"]))
except (TypeError, ValueError):
logger.warning("Invalid chunks_json for doc %s — skipping", document_id)
return []
return _read
def _chunk_dict_to_chunk(d: dict, *, document_id: str, sequence: int) -> Chunk:
text = d.get("text", "")
return Chunk(
id=_stable_chunk_id(document_id, sequence, text),
document_id=document_id,
sequence=sequence,
text=text,
headings=list(d.get("headings", [])),
source_page=d.get("sourcePage"),
bboxes=[
ChunkBbox(page=b.get("page", 0), bbox=list(b.get("bbox", [])))
for b in d.get("bboxes", [])
],
doc_items=[
ChunkDocItem(
self_ref=di.get("selfRef", ""),
label=di.get("label", ""),
)
for di in d.get("docItems", [])
],
token_count=d.get("tokenCount"),
)
async def _step_materialize_chunks(db: aiosqlite.Connection, *, dry_run: bool) -> int:
"""For each document with chunks_json, insert chunk rows if absent."""
chunks_repo = SqliteChunkRepository()
cursor = await db.execute("SELECT id FROM documents")
doc_rows = await cursor.fetchall()
written = 0
read = _chunk_dicts_for_doc(db)
for doc_row in doc_rows:
document_id = doc_row["id"]
existing = await chunks_repo.find_for_document(document_id, include_deleted=True)
if existing:
continue # already materialized
chunk_dicts = await read(document_id)
if not chunk_dicts:
continue
chunks = [
_chunk_dict_to_chunk(d, document_id=document_id, sequence=i)
for i, d in enumerate(chunk_dicts)
]
logger.info("chunks: doc=%s materializing %d", document_id, len(chunks))
if not dry_run:
await chunks_repo.insert_many(chunks)
written += len(chunks)
return written
# ---------------------------------------------------------------------------
# Step: backfill_default_store_links
# ---------------------------------------------------------------------------
def _chunks_to_results(chunks: Iterable[Chunk]) -> list[ChunkResult]:
return [
ChunkResult(
text=c.text,
headings=list(c.headings),
source_page=c.source_page,
bboxes=[ChunkBbox(page=b.page, bbox=list(b.bbox)) for b in c.bboxes],
doc_items=[ChunkDocItem(self_ref=d.self_ref, label=d.label) for d in c.doc_items],
)
for c in chunks
]
async def _step_backfill_links(db: aiosqlite.Connection, *, dry_run: bool) -> int:
"""For each doc that already has chunks materialized, create a
`default`-store link in `Ingested` state with a freshly-computed
chunkset hash. This treats "chunks exist" as a proxy for "the
document has been ingested into the default store" — which is true
for tenants with the legacy single-index deployment that 0.6.0
formalises into the `default` store.
This step does NOT call OpenSearch. Operators with non-trivial
OpenSearch state should run a separate reindex / verification
after this script (see runbook).
"""
store_repo = SqliteStoreRepository()
chunks_repo = SqliteChunkRepository()
link_repo = SqliteDocumentStoreLinkRepository()
default_store = await store_repo.get_default()
if default_store is None:
logger.warning(
"No default store — skipping link backfill (run init_db first)",
)
return 0
cursor = await db.execute("SELECT id FROM documents")
doc_rows = await cursor.fetchall()
written = 0
for doc_row in doc_rows:
document_id = doc_row["id"]
existing = await link_repo.find_one(document_id, default_store.id)
if existing is not None:
continue
chunks = await chunks_repo.find_for_document(document_id)
if not chunks:
continue
h = chunkset_hash(_chunks_to_results(chunks))
link = DocumentStoreLink(
document_id=document_id,
store_id=default_store.id,
state=DocumentStoreLinkState.INGESTED,
chunkset_hash=h,
)
logger.info(
"links: doc=%s -> store=%s ingested (hash=%s…)",
document_id,
default_store.id,
h[:8],
)
if not dry_run:
await link_repo.upsert(link)
written += 1
return written
# ---------------------------------------------------------------------------
# Step: reaggregate_document_lifecycle
# ---------------------------------------------------------------------------
async def _step_reaggregate(db: aiosqlite.Connection, *, dry_run: bool) -> int:
"""Recompute the doc lifecycle state from per-store links + the
fallback inferred in step 1."""
link_repo = SqliteDocumentStoreLinkRepository()
cursor = await db.execute("SELECT id, lifecycle_state FROM documents")
rows = await cursor.fetchall()
written = 0
for row in rows:
doc_id = row["id"]
current = DocumentLifecycleState(row["lifecycle_state"])
links = await link_repo.find_for_document(doc_id)
target = aggregate_lifecycle(links, fallback=current)
if target == current:
continue
logger.info(
"reaggregate: doc=%s %s -> %s",
doc_id,
current.value,
target.value,
)
if not dry_run:
await db.execute(
"UPDATE documents SET lifecycle_state = ?, "
"lifecycle_state_at = datetime('now') WHERE id = ?",
(target.value, doc_id),
)
written += 1
if not dry_run:
await db.commit()
return written
# ---------------------------------------------------------------------------
# Orchestration
# ---------------------------------------------------------------------------
_STEPS = [
("backfill_lifecycle", _step_backfill_lifecycle),
("materialize_chunks", _step_materialize_chunks),
("backfill_links", _step_backfill_links),
("reaggregate", _step_reaggregate),
]
@dataclass
class StepResult:
name: str
skipped: bool
written: int
async def run(
*,
dry_run: bool = False,
only_step: str | None = None,
) -> list[StepResult]:
"""Run the migration. Returns one StepResult per step (skipped or not)."""
await init_db() # ensures the schema and migration_progress table exist
results: list[StepResult] = []
async with get_connection() as db:
for name, step in _STEPS:
if only_step and only_step != name:
continue
if await _is_done(db, name):
results.append(StepResult(name=name, skipped=True, written=0))
logger.info("step %s: already done — skipping", name)
continue
logger.info("step %s: running (dry_run=%s)", name, dry_run)
written = await step(db, dry_run=dry_run)
if not dry_run:
await _mark_done(db, name)
results.append(StepResult(name=name, skipped=False, written=written))
return results
def _print_summary(results: list[StepResult], *, dry_run: bool) -> None:
print()
print(f"{'step':35s} {'wrote':>10s} {'skipped':>10s}")
print("-" * 60)
total_written = 0
for r in results:
print(f"{r.name:35s} {r.written:>10d} {'yes' if r.skipped else 'no':>10s}")
total_written += r.written
print("-" * 60)
print(f"{'total':35s} {total_written:>10d}")
print()
print(f"dry_run={dry_run}")
def main(argv: list[str] | None = None) -> int:
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s: %(message)s",
)
parser = argparse.ArgumentParser(prog="migrate_06")
parser.add_argument(
"--dry-run",
action="store_true",
help="Print the plan without writing.",
)
parser.add_argument(
"--only-step",
choices=[name for name, _ in _STEPS],
help="Run a single named step (useful when re-doing one phase).",
)
args = parser.parse_args(argv)
results = asyncio.run(run(dry_run=args.dry_run, only_step=args.only_step))
_print_summary(results, dry_run=args.dry_run)
return 0
if __name__ == "__main__":
sys.exit(main())

View file

@ -0,0 +1,53 @@
@ui @critical
Feature: UI — Doc tab chunk mode renders canonical chunks (#256)
# Regression coverage for the bug where opening the chunks tab on a
# document returned 404 because /api/documents/{id}/chunks was not
# implemented backend-side. Setup is API-driven (upload + analyse) so
# the test is fast and deterministic; the UI portion only exercises
# navigation + chunk rendering, not the parse pipeline.
Background:
* url baseUrl
Scenario: Open the chunks tab and see the canonical chunkset (no 404)
# 1. Setup via API: upload + run an analysis (with chunking) and wait
# until the doc-centric chunkset gets promoted by the analysis hook.
* def upload = call read('classpath:common/helpers/upload.feature') { file: 'small.pdf' }
* def docId = upload.docId
Given url baseUrl
And path '/api/analyses'
And request { documentId: '#(docId)', chunkingOptions: { chunkerType: 'hybrid', maxTokens: 256, mergePeers: true, repeatTableHeader: true } }
When method POST
Then status 200
* def analysisId = response.id
# Poll until the analysis is COMPLETED (chunks get promoted in the
# same step via AnalysisService → ChunkService.promote_from_analysis).
Given url baseUrl
And path '/api/analyses', analysisId
And retry until response.status == 'COMPLETED' || response.status == 'FAILED'
When method GET
Then status 200
And match response.status == 'COMPLETED'
# 2. Sanity-check the new endpoint over HTTP — proves the 404 is gone.
Given url baseUrl
And path '/api/documents', docId, 'chunks'
When method GET
Then status 200
And assert karate.sizeOf(response) > 0
# 3. Drive the UI: open the workspace page and click the chunks tab.
* driver uiBaseUrl + '/docs/' + docId
* waitFor('[data-e2e=tab-strip]')
* click('[data-e2e=tab-chunks]')
# 4. Verify the chunks list rendered (no 404 banner, chunk-list visible).
* waitFor('[data-e2e=chunks-editor]')
* waitFor('[data-e2e=chunk-list]')
* assert karate.sizeOf(locateAll('[data-e2e=chunk-list] .chunk-card')) > 0
# 5. Cleanup via API.
* call read('classpath:common/helpers/cleanup-by-name.feature') { filename: 'small.pdf' }

View file

@ -1,12 +1,12 @@
{
"name": "docling-studio",
"version": "0.4.0",
"version": "0.5.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "docling-studio",
"version": "0.4.0",
"version": "0.5.0",
"dependencies": {
"cytoscape": "^3.30.0",
"cytoscape-dagre": "^2.5.0",

View file

@ -40,6 +40,7 @@
<div class="app-body">
<AppSidebar :open="sidebarOpen" />
<main class="main">
<AppBreadcrumb :crumbs="breadcrumbStore.crumbs" />
<RouterView />
</main>
</div>
@ -50,6 +51,8 @@
import { ref, computed } from 'vue'
import { RouterView, useRouter } from 'vue-router'
import { AppSidebar } from '../shared/ui/index'
import AppBreadcrumb from '../shared/breadcrumb/AppBreadcrumb.vue'
import { useBreadcrumbStore } from '../shared/breadcrumb/store'
import { useSettingsStore } from '../features/settings/store'
import { useDocumentStore } from '../features/document/store'
import { useFeatureFlag } from '../features/feature-flags'
@ -58,6 +61,7 @@ import { useI18n } from '../shared/i18n'
useSettingsStore()
const flagStore = useFeatureFlagStore()
const breadcrumbStore = useBreadcrumbStore()
const { t } = useI18n()
const router = useRouter()
const documentStore = useDocumentStore()

View file

@ -1,61 +1,58 @@
import { createRouter, createWebHistory } from 'vue-router'
import type { RouteRecordRaw } from 'vue-router'
const routes: RouteRecordRaw[] = [
{
path: '/',
name: 'home',
component: () => import('../../pages/HomePage.vue'),
},
{
path: '/studio',
name: 'studio',
component: () => import('../../pages/StudioPage.vue'),
},
{
path: '/history',
name: 'history',
component: () => import('../../pages/HistoryPage.vue'),
},
{
path: '/documents',
name: 'documents',
component: () => import('../../pages/DocumentsPage.vue'),
},
{
path: '/search',
name: 'search',
component: () => import('../../pages/SearchPage.vue'),
},
{
// Reasoning-trace tunnel. Route is always registered; the page shows
// an empty state when the `reasoning` feature flag is off (same pattern
// as /search does for ingestion).
path: '/reasoning',
name: 'reasoning',
component: () => import('../../pages/ReasoningPage.vue'),
},
{
// Deep-link into a specific document's reasoning workspace, e.g. shared
// by Peter to a teammate.
path: '/reasoning/:docId',
name: 'reasoning-doc',
component: () => import('../../pages/ReasoningPage.vue'),
props: true,
},
{
path: '/settings',
name: 'settings',
component: () => import('../../pages/SettingsPage.vue'),
},
{
path: '/:pathMatch(.*)*',
name: 'not-found',
redirect: '/',
},
]
import { useFeatureFlagStore } from '../../features/feature-flags/store'
import { isDocMode } from '../../shared/routing/modes'
import { ROUTES } from '../../shared/routing/names'
import { resolveMode } from '../../shared/routing/resolveMode'
import { resolveSurface } from '../../shared/routing/resolveSurface'
import { routes } from './routes'
export { routes }
export const router = createRouter({
history: createWebHistory(),
routes,
})
/**
* Surface gating (#257) + mode gating (#210).
*
* Surface: two master flags select which UI surface is exposed. When a
* route from a disabled surface is requested, redirect to the other
* surface's landing page or to `/` if both are off (defensive net;
* backend refuses to start in that state).
*
* Mode: inside the doc workspace, the requested mode falls back to the
* first enabled mode when the deep-linked one is off.
*/
router.beforeEach((to) => {
const flagStore = useFeatureFlagStore()
const name = String(to.name ?? '')
const surfaceRedirect = resolveSurface(name, {
studio: flagStore.studioModeEnabled,
rag: flagStore.ragPipelineEnabled,
})
if (surfaceRedirect !== null) return { name: surfaceRedirect }
if (to.name !== ROUTES.DOC_WORKSPACE) return true
const flags = flagStore.modeFlags()
const requestedRaw = Array.isArray(to.query.mode) ? to.query.mode[0] : to.query.mode
const requested = isDocMode(requestedRaw) ? requestedRaw : undefined
const resolved = resolveMode(requested, flags)
if (resolved === null) {
return {
name: ROUTES.DOCS_LIBRARY,
query: { reason: 'no-mode-enabled' },
}
}
if (resolved !== requested) {
return {
...to,
query: { ...to.query, mode: resolved },
}
}
return true
})

View file

@ -0,0 +1,81 @@
import { createMemoryHistory, createRouter } from 'vue-router'
import { describe, expect, it } from 'vitest'
import { routes } from './routes'
import { ROUTES } from '../../shared/routing/names'
/**
* Router test uses `createMemoryHistory` so we don't need `window`
* (vitest defaults to the node environment for performance). The
* production router builds the same `routes` table on top of
* `createWebHistory` in `index.ts`.
*/
const buildRouter = () => createRouter({ history: createMemoryHistory(), routes })
describe('router', () => {
it('resolves every 0.6.0 doc-centric route to a component', () => {
const router = buildRouter()
const cases: Array<{ path: string; name: string }> = [
{ path: '/docs', name: ROUTES.DOCS_LIBRARY },
{ path: '/docs/new', name: ROUTES.DOCS_NEW },
{ path: '/docs/abc', name: ROUTES.DOC_WORKSPACE },
{ path: '/index', name: ROUTES.STORES_LIST },
{ path: '/index/foo', name: ROUTES.STORE_DETAIL },
{ path: '/index/foo/query', name: ROUTES.STORE_QUERY },
{ path: '/runs', name: ROUTES.RUNS },
{ path: '/runs/run-42', name: ROUTES.RUN_DETAIL },
]
for (const c of cases) {
const resolved = router.resolve(c.path)
expect(resolved.name, `route ${c.path}`).toBe(c.name)
expect(resolved.matched.length, `route ${c.path} has a component`).toBeGreaterThan(0)
}
})
it('keeps legacy routes functional', () => {
const router = buildRouter()
expect(router.resolve('/').name).toBe(ROUTES.HOME)
expect(router.resolve('/studio').name).toBe(ROUTES.STUDIO)
expect(router.resolve('/documents').name).toBe(ROUTES.DOCUMENTS)
expect(router.resolve('/history').name).toBe(ROUTES.HISTORY)
expect(router.resolve('/search').name).toBe(ROUTES.SEARCH)
expect(router.resolve('/reasoning').name).toBe(ROUTES.REASONING)
expect(router.resolve('/reasoning/abc').name).toBe(ROUTES.REASONING_DOC)
expect(router.resolve('/settings').name).toBe(ROUTES.SETTINGS)
})
it('passes id and parsed mode to the doc workspace as props', () => {
const router = buildRouter()
const route = router.resolve({
name: ROUTES.DOC_WORKSPACE,
params: { id: 'abc' },
query: { mode: 'chunks' },
})
const propsFn = route.matched[0]?.props as
| { default?: (r: typeof route) => unknown }
| undefined
const computed = (propsFn?.default ?? (() => null))(route) as { id: string; mode: string }
expect(computed.id).toBe('abc')
expect(computed.mode).toBe('chunks')
})
it('falls back to ask when mode is unknown', () => {
const router = buildRouter()
const route = router.resolve({
name: ROUTES.DOC_WORKSPACE,
params: { id: 'abc' },
query: { mode: 'garbage' },
})
const propsFn = route.matched[0]?.props as
| { default?: (r: typeof route) => unknown }
| undefined
const computed = (propsFn?.default ?? (() => null))(route) as { mode: string }
expect(computed.mode).toBe('ask')
})
it('redirects unknown paths to /', () => {
const router = buildRouter()
const resolved = router.resolve('/nope/this/does/not/exist')
expect(resolved.matched[0]?.redirect).toBeDefined()
})
})

View file

@ -0,0 +1,136 @@
import type { RouteLocationNormalized, RouteRecordRaw } from 'vue-router'
import { parseMode } from '../../shared/routing/modes'
import { ROUTES } from '../../shared/routing/names'
/**
* Route table used by the production router and by tests.
*
* Lives in its own file so importing the router definition doesn't
* trigger `createWebHistory()` (which needs `window` and breaks the
* default node-environment vitest tests). Tests build a router with
* `createMemoryHistory()` over this same table.
*/
export const routes: RouteRecordRaw[] = [
// ---------------------------------------------------------------------------
// Legacy routes — kept functional during the 0.6.0 transition.
// ---------------------------------------------------------------------------
{
path: '/',
name: ROUTES.HOME,
component: () => import('../../pages/HomePage.vue'),
},
{
path: '/studio',
name: ROUTES.STUDIO,
component: () => import('../../pages/StudioPage.vue'),
},
{
path: '/history',
name: ROUTES.HISTORY,
component: () => import('../../pages/HistoryPage.vue'),
},
{
path: '/documents',
name: ROUTES.DOCUMENTS,
component: () => import('../../pages/DocumentsPage.vue'),
},
{
path: '/search',
name: ROUTES.SEARCH,
component: () => import('../../pages/SearchPage.vue'),
},
{
// Reasoning-trace tunnel. Route is always registered; the page shows
// an empty state when the `reasoning` feature flag is off (same pattern
// as /search does for ingestion).
path: '/reasoning',
name: ROUTES.REASONING,
component: () => import('../../pages/ReasoningPage.vue'),
},
{
// Deep-link into a specific document's reasoning workspace, e.g. shared
// by Peter to a teammate.
path: '/reasoning/:docId',
name: ROUTES.REASONING_DOC,
component: () => import('../../pages/ReasoningPage.vue'),
props: true,
},
{
path: '/settings',
name: ROUTES.SETTINGS,
component: () => import('../../pages/SettingsPage.vue'),
},
// ---------------------------------------------------------------------------
// 0.6.0 — Document-centric routes (#207). Placeholder pages until E3/E4/E5
// implement them; the legacy routes above keep working in parallel.
// ---------------------------------------------------------------------------
{
path: '/docs',
name: ROUTES.DOCS_LIBRARY,
component: () => import('../../pages/DocsLibraryPage.vue'),
},
{
path: '/docs/new',
name: ROUTES.DOCS_NEW,
component: () => import('../../pages/DocsNewPage.vue'),
},
{
path: '/docs/:id',
name: ROUTES.DOC_WORKSPACE,
component: () => import('../../pages/DocWorkspacePage.vue'),
props: (route: RouteLocationNormalized) => ({
id: String(route.params.id),
mode: parseMode(route.query.mode),
}),
},
{
path: '/index',
name: ROUTES.STORES_LIST,
component: () => import('../../pages/StoresListPage.vue'),
},
{
path: '/index/new',
name: ROUTES.STORE_CREATE,
component: () => import('../../pages/StoreCreatePage.vue'),
},
{
path: '/index/:store',
name: ROUTES.STORE_DETAIL,
component: () => import('../../pages/StoreDetailPage.vue'),
props: true,
},
{
path: '/index/:store/edit',
name: ROUTES.STORE_EDIT,
component: () => import('../../pages/StoreEditPage.vue'),
props: true,
},
{
path: '/index/:store/query',
name: ROUTES.STORE_QUERY,
component: () => import('../../pages/StoreQueryPage.vue'),
props: true,
},
{
path: '/runs',
name: ROUTES.RUNS,
component: () => import('../../pages/RunsPage.vue'),
},
{
path: '/runs/:id',
name: ROUTES.RUN_DETAIL,
component: () => import('../../pages/RunDetailPage.vue'),
props: true,
},
// ---------------------------------------------------------------------------
// 404 — must come last.
// ---------------------------------------------------------------------------
{
path: '/:pathMatch(.*)*',
name: ROUTES.NOT_FOUND,
redirect: '/',
},
]

View file

@ -1,5 +1,11 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { createAnalysis, fetchAnalyses, fetchAnalysis, deleteAnalysis } from './api'
import {
createAnalysis,
fetchAnalyses,
fetchAnalysis,
deleteAnalysis,
fetchDocumentAnalyses,
} from './api'
vi.mock('../../shared/api/http', () => ({
apiFetch: vi.fn(),
@ -66,4 +72,14 @@ describe('analysis API', () => {
expect(apiFetch).toHaveBeenCalledWith('/api/analyses/42', { method: 'DELETE' })
})
it('fetchDocumentAnalyses calls GET /api/analyses?documentId=:id', async () => {
const analyses = [{ id: '1', documentId: 'doc-42', status: 'COMPLETED' }]
apiFetch.mockResolvedValue(analyses)
const result = await fetchDocumentAnalyses('doc-42')
expect(apiFetch).toHaveBeenCalledWith('/api/analyses?documentId=doc-42')
expect(result).toEqual(analyses)
})
})

View file

@ -37,3 +37,7 @@ export function fetchAnalysis(id: string): Promise<Analysis> {
export function deleteAnalysis(id: string): Promise<unknown> {
return apiFetch(`/api/analyses/${id}`, { method: 'DELETE' })
}
export function fetchDocumentAnalyses(docId: string): Promise<Analysis[]> {
return apiFetch<Analysis[]>(`/api/analyses?documentId=${encodeURIComponent(docId)}`)
}

View file

@ -0,0 +1,131 @@
<template>
<div class="inspect-result-tabs">
<div class="irt-tab-strip">
<button
v-for="tab in TABS"
:key="tab.id"
class="irt-tab-btn"
:class="{ active: activeTab === tab.id }"
@click="activeTab = tab.id"
>
{{ t(tab.labelKey) }}
</button>
</div>
<div class="irt-content">
<!-- Markdown full document -->
<div v-if="activeTab === 'markdown'" class="irt-markdown">
<MarkdownViewer :content="analysis.contentMarkdown ?? undefined" />
</div>
<!-- Elements StructureViewer (page images + bbox overlay) -->
<div v-else-if="activeTab === 'elements'" class="irt-elements">
<div v-if="pages.length === 0" class="irt-empty">
{{ t('inspect.noElements') }}
</div>
<StructureViewer v-else :pages="pages" :document-id="docId" />
</div>
<!-- Images picture elements extracted from all pages -->
<div v-else-if="activeTab === 'images'" class="irt-images">
<ImageGallery :pages="pages" />
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, computed } from 'vue'
import type { Analysis, Page } from '../../../shared/types'
import MarkdownViewer from './MarkdownViewer.vue'
import StructureViewer from './StructureViewer.vue'
import ImageGallery from './ImageGallery.vue'
import { useI18n } from '../../../shared/i18n'
const props = defineProps<{
analysis: Analysis
docId: string
}>()
const { t } = useI18n()
const TABS = [
{ id: 'markdown', labelKey: 'inspect.tabMarkdown' },
{ id: 'elements', labelKey: 'inspect.tabElements' },
{ id: 'images', labelKey: 'inspect.tabImages' },
] as const
type TabId = (typeof TABS)[number]['id']
const activeTab = ref<TabId>('elements')
const pages = computed<Page[]>(() => {
if (!props.analysis.pagesJson) return []
try {
return JSON.parse(props.analysis.pagesJson) as Page[]
} catch {
return []
}
})
</script>
<style scoped>
.inspect-result-tabs {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
}
.irt-tab-strip {
display: flex;
border-bottom: 1px solid var(--border);
padding: 0 16px;
flex-shrink: 0;
background: var(--bg-surface);
}
.irt-tab-btn {
padding: 10px 14px;
font-size: 13px;
font-weight: 500;
color: var(--text-muted);
background: none;
border: none;
border-bottom: 2px solid transparent;
cursor: pointer;
transition: all var(--transition);
margin-bottom: -1px;
}
.irt-tab-btn:hover {
color: var(--text-secondary);
}
.irt-tab-btn.active {
color: var(--accent);
border-bottom-color: var(--accent);
}
.irt-content {
flex: 1;
overflow: hidden;
}
.irt-markdown,
.irt-elements,
.irt-images {
height: 100%;
overflow-y: auto;
padding: 16px 20px;
}
.irt-empty {
display: flex;
align-items: center;
justify-content: center;
height: 100%;
color: var(--text-muted);
font-size: 13px;
}
</style>

View file

@ -0,0 +1,177 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import {
fetchChunks,
updateChunk,
mergeChunks,
splitChunk,
dropChunk,
addChunk,
fetchChunkDiff,
pushChunksToStore,
} from './api'
vi.mock('../../shared/api/http', () => ({
apiFetch: vi.fn(),
}))
import { apiFetch } from '../../shared/api/http'
describe('chunks API', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('fetchChunks calls GET /api/documents/:id/chunks', async () => {
const chunks = [{ id: 'c1', docId: 'd1', text: 'hello' }]
apiFetch.mockResolvedValue(chunks)
const result = await fetchChunks('d1')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks')
expect(result).toEqual(chunks)
})
it('updateChunk calls PATCH with patch body', async () => {
const chunk = { id: 'c1', docId: 'd1', text: 'updated' }
apiFetch.mockResolvedValue(chunk)
const result = await updateChunk('d1', 'c1', { text: 'updated' })
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks/c1', {
method: 'PATCH',
body: JSON.stringify({ text: 'updated' }),
})
expect(result).toEqual(chunk)
})
it('mergeChunks calls POST /chunks/merge with ids', async () => {
const merged = [{ id: 'cm', docId: 'd1', text: 'merged' }]
apiFetch.mockResolvedValue(merged)
const result = await mergeChunks('d1', ['c1', 'c2'])
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks/merge', {
method: 'POST',
body: JSON.stringify({ ids: ['c1', 'c2'] }),
})
expect(result).toEqual(merged)
})
it('splitChunk calls POST /chunks/:id/split with cursorOffset', async () => {
const split = [
{ id: 'c1a', docId: 'd1', text: 'part1' },
{ id: 'c1b', docId: 'd1', text: 'part2' },
]
apiFetch.mockResolvedValue(split)
const result = await splitChunk('d1', 'c1', 10)
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks/c1/split', {
method: 'POST',
body: JSON.stringify({ cursorOffset: 10 }),
})
expect(result).toEqual(split)
})
it('dropChunk calls DELETE /api/documents/:id/chunks/:chunkId', async () => {
apiFetch.mockResolvedValue(undefined)
await dropChunk('d1', 'c1')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks/c1', { method: 'DELETE' })
})
it('addChunk calls POST /chunks with text and afterId', async () => {
const newChunk = { id: 'cnew', docId: 'd1', text: 'new' }
apiFetch.mockResolvedValue(newChunk)
const result = await addChunk('d1', 'new', 'c1')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks', {
method: 'POST',
body: JSON.stringify({ text: 'new', afterId: 'c1' }),
})
expect(result).toEqual(newChunk)
})
it('addChunk works without afterId', async () => {
const newChunk = { id: 'cnew', docId: 'd1', text: 'new' }
apiFetch.mockResolvedValue(newChunk)
await addChunk('d1', 'new')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks', {
method: 'POST',
body: JSON.stringify({ text: 'new', afterId: undefined }),
})
})
it('fetchChunkDiff calls GET /diff with encoded store param', async () => {
const diffs = [{ chunkId: 'c1', status: 'modified', textDiff: 'diff' }]
apiFetch.mockResolvedValue(diffs)
const result = await fetchChunkDiff('d1', 'my store')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/diff?store=my%20store')
expect(result).toEqual(diffs)
})
it('pushChunksToStore calls POST /chunks/push with store', async () => {
const response = { jobId: 'j1', summary: { embeds: 5, tokens: 1000 } }
apiFetch.mockResolvedValue(response)
const result = await pushChunksToStore('d1', 'my-store')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/d1/chunks/push', {
method: 'POST',
body: JSON.stringify({ store: 'my-store' }),
})
expect(result).toEqual(response)
})
// ---- Error propagation: each call lets the apiFetch error bubble up so
// the store can map it to a user-visible message. The original 404 bug
// (#256) silently swallowed the error — these guard tests would have
// caught it.
describe('error propagation', () => {
it('fetchChunks rejects with apiFetch error (404)', async () => {
apiFetch.mockRejectedValue(new Error('404: Not Found'))
await expect(fetchChunks('d1')).rejects.toThrow('404')
})
it('updateChunk rejects with apiFetch error (404)', async () => {
apiFetch.mockRejectedValue(new Error('404: Not Found'))
await expect(updateChunk('d1', 'c1', { text: 'x' })).rejects.toThrow('404')
})
it('mergeChunks rejects with apiFetch error (409)', async () => {
apiFetch.mockRejectedValue(new Error('409: Conflict'))
await expect(mergeChunks('d1', ['a', 'b'])).rejects.toThrow('409')
})
it('splitChunk rejects with apiFetch error (400)', async () => {
apiFetch.mockRejectedValue(new Error('400: Bad Request'))
await expect(splitChunk('d1', 'c1', 0)).rejects.toThrow('400')
})
it('dropChunk rejects with apiFetch error (404)', async () => {
apiFetch.mockRejectedValue(new Error('404: Not Found'))
await expect(dropChunk('d1', 'c1')).rejects.toThrow('404')
})
it('addChunk rejects with apiFetch error (500)', async () => {
apiFetch.mockRejectedValue(new Error('500: Internal Server Error'))
await expect(addChunk('d1', 'x')).rejects.toThrow('500')
})
it('fetchChunkDiff rejects with apiFetch error (404)', async () => {
apiFetch.mockRejectedValue(new Error('404: Not Found'))
await expect(fetchChunkDiff('d1', 'store')).rejects.toThrow('404')
})
it('pushChunksToStore rejects with apiFetch error (503)', async () => {
apiFetch.mockRejectedValue(new Error('503: Service Unavailable'))
await expect(pushChunksToStore('d1', 'store')).rejects.toThrow('503')
})
})
})

View file

@ -0,0 +1,60 @@
import type { DocChunk, ChunkDiff, PushSummary } from '../../shared/types'
import { apiFetch } from '../../shared/api/http'
export function fetchChunks(docId: string): Promise<DocChunk[]> {
return apiFetch<DocChunk[]>(`/api/documents/${docId}/chunks`)
}
export function updateChunk(
docId: string,
chunkId: string,
patch: { text?: string; title?: string },
): Promise<DocChunk> {
return apiFetch<DocChunk>(`/api/documents/${docId}/chunks/${chunkId}`, {
method: 'PATCH',
body: JSON.stringify(patch),
})
}
export function mergeChunks(docId: string, ids: string[]): Promise<DocChunk[]> {
return apiFetch<DocChunk[]>(`/api/documents/${docId}/chunks/merge`, {
method: 'POST',
body: JSON.stringify({ ids }),
})
}
export function splitChunk(
docId: string,
chunkId: string,
cursorOffset: number,
): Promise<DocChunk[]> {
return apiFetch<DocChunk[]>(`/api/documents/${docId}/chunks/${chunkId}/split`, {
method: 'POST',
body: JSON.stringify({ cursorOffset }),
})
}
export function dropChunk(docId: string, chunkId: string): Promise<void> {
return apiFetch(`/api/documents/${docId}/chunks/${chunkId}`, { method: 'DELETE' })
}
export function addChunk(docId: string, text: string, afterId?: string): Promise<DocChunk> {
return apiFetch<DocChunk>(`/api/documents/${docId}/chunks`, {
method: 'POST',
body: JSON.stringify({ text, afterId }),
})
}
export function fetchChunkDiff(docId: string, store: string): Promise<ChunkDiff[]> {
return apiFetch<ChunkDiff[]>(`/api/documents/${docId}/diff?store=${encodeURIComponent(store)}`)
}
export function pushChunksToStore(
docId: string,
store: string,
): Promise<{ jobId: string; summary: PushSummary }> {
return apiFetch<{ jobId: string; summary: PushSummary }>(`/api/documents/${docId}/chunks/push`, {
method: 'POST',
body: JSON.stringify({ store }),
})
}

View file

@ -0,0 +1,2 @@
export { useChunksStore } from './store'
export * from './api'

View file

@ -0,0 +1,137 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { setActivePinia, createPinia } from 'pinia'
import { useChunksStore } from './store'
vi.mock('./api', () => ({
fetchChunks: vi.fn(),
updateChunk: vi.fn(),
mergeChunks: vi.fn(),
splitChunk: vi.fn(),
dropChunk: vi.fn(),
addChunk: vi.fn(),
fetchChunkDiff: vi.fn(),
pushChunksToStore: vi.fn(),
}))
import * as api from './api'
const makeChunk = (id: string, text = 'text') => ({
id,
docId: 'd1',
text,
createdAt: '2025-01-01T00:00:00Z',
updatedAt: '2025-01-01T00:00:00Z',
})
describe('useChunksStore', () => {
beforeEach(() => {
setActivePinia(createPinia())
vi.clearAllMocks()
})
it('load — sets chunks and clears loading', async () => {
const chunks = [makeChunk('c1'), makeChunk('c2')]
api.fetchChunks.mockResolvedValue(chunks)
const store = useChunksStore()
const loadPromise = store.load('d1')
expect(store.loading).toBe(true)
await loadPromise
expect(store.loading).toBe(false)
expect(store.chunks).toEqual(chunks)
})
it('load — sets error on failure', async () => {
api.fetchChunks.mockRejectedValue(new Error('boom'))
const store = useChunksStore()
await store.load('d1')
expect(store.error).toBe('boom')
expect(store.chunks).toEqual([])
})
it('updateText — replaces chunk in list', async () => {
const store = useChunksStore()
store.chunks = [makeChunk('c1', 'original')]
const updated = makeChunk('c1', 'changed')
api.updateChunk.mockResolvedValue(updated)
await store.updateText('d1', 'c1', 'changed')
expect(store.chunks[0].text).toBe('changed')
})
it('updateTitle — replaces chunk in list', async () => {
const store = useChunksStore()
store.chunks = [makeChunk('c1')]
const updated = { ...makeChunk('c1'), title: 'New title' }
api.updateChunk.mockResolvedValue(updated)
await store.updateTitle('d1', 'c1', 'New title')
expect(store.chunks[0].title).toBe('New title')
})
it('drop — removes chunk from list', async () => {
const store = useChunksStore()
store.chunks = [makeChunk('c1'), makeChunk('c2')]
api.dropChunk.mockResolvedValue(undefined)
await store.drop('d1', 'c1')
expect(store.chunks).toHaveLength(1)
expect(store.chunks[0].id).toBe('c2')
})
it('add — appends at end when no afterId', async () => {
const store = useChunksStore()
store.chunks = [makeChunk('c1')]
const created = makeChunk('cnew', 'new text')
api.addChunk.mockResolvedValue(created)
await store.add('d1', 'new text')
expect(store.chunks).toHaveLength(2)
expect(store.chunks[1].id).toBe('cnew')
})
it('add — inserts after given chunk', async () => {
const store = useChunksStore()
store.chunks = [makeChunk('c1'), makeChunk('c2')]
const created = makeChunk('cnew', 'new text')
api.addChunk.mockResolvedValue(created)
await store.add('d1', 'new text', 'c1')
expect(store.chunks[1].id).toBe('cnew')
expect(store.chunks[2].id).toBe('c2')
})
it('loadDiff — sets diff', async () => {
const diffs = [{ chunkId: 'c1', status: 'modified' as const, textDiff: 'x' }]
api.fetchChunkDiff.mockResolvedValue(diffs)
const store = useChunksStore()
await store.loadDiff('d1', 'my-store')
expect(store.diff).toEqual(diffs)
})
it('push — returns jobId on success', async () => {
api.pushChunksToStore.mockResolvedValue({ jobId: 'j1', summary: { embeds: 3, tokens: 500 } })
const store = useChunksStore()
const jobId = await store.push('d1', 'my-store')
expect(jobId).toBe('j1')
})
it('push — returns null on failure', async () => {
api.pushChunksToStore.mockRejectedValue(new Error('network'))
const store = useChunksStore()
const jobId = await store.push('d1', 'my-store')
expect(jobId).toBeNull()
expect(store.error).toBe('network')
})
})

View file

@ -0,0 +1,175 @@
import { defineStore } from 'pinia'
import { ref } from 'vue'
import type { DocChunk, ChunkDiff } from '../../shared/types'
import * as api from './api'
export const useChunksStore = defineStore('chunks', () => {
const chunks = ref<DocChunk[]>([])
const loading = ref(false)
const saving = ref(false)
const diffing = ref(false)
const diff = ref<ChunkDiff[]>([])
const error = ref<string | null>(null)
function clearError(): void {
error.value = null
}
async function load(docId: string): Promise<void> {
loading.value = true
error.value = null
try {
chunks.value = await api.fetchChunks(docId)
} catch (e) {
error.value = (e as Error).message || 'Failed to load chunks'
console.error('Failed to load chunks', e)
} finally {
loading.value = false
}
}
async function updateText(docId: string, chunkId: string, text: string): Promise<void> {
saving.value = true
try {
const updated = await api.updateChunk(docId, chunkId, { text })
const idx = chunks.value.findIndex((c) => c.id === chunkId)
if (idx !== -1) chunks.value = chunks.value.with(idx, updated)
} catch (e) {
error.value = (e as Error).message || 'Failed to save chunk'
console.error('Failed to save chunk', e)
} finally {
saving.value = false
}
}
async function updateTitle(docId: string, chunkId: string, title: string): Promise<void> {
saving.value = true
try {
const updated = await api.updateChunk(docId, chunkId, { title })
const idx = chunks.value.findIndex((c) => c.id === chunkId)
if (idx !== -1) chunks.value = chunks.value.with(idx, updated)
} catch (e) {
error.value = (e as Error).message || 'Failed to retitle chunk'
console.error('Failed to retitle chunk', e)
} finally {
saving.value = false
}
}
async function merge(docId: string, ids: string[]): Promise<void> {
saving.value = true
try {
const updated = await api.mergeChunks(docId, ids)
const idSet = new Set(ids)
const kept = chunks.value.filter((c) => !idSet.has(c.id))
const firstIdx = chunks.value.findIndex((c) => idSet.has(c.id))
chunks.value = [...kept.slice(0, firstIdx), ...updated, ...kept.slice(firstIdx)]
} catch (e) {
error.value = (e as Error).message || 'Failed to merge chunks'
console.error('Failed to merge chunks', e)
} finally {
saving.value = false
}
}
async function split(docId: string, chunkId: string, cursorOffset: number): Promise<void> {
saving.value = true
try {
const produced = await api.splitChunk(docId, chunkId, cursorOffset)
const idx = chunks.value.findIndex((c) => c.id === chunkId)
if (idx !== -1) {
chunks.value = [...chunks.value.slice(0, idx), ...produced, ...chunks.value.slice(idx + 1)]
}
} catch (e) {
error.value = (e as Error).message || 'Failed to split chunk'
console.error('Failed to split chunk', e)
} finally {
saving.value = false
}
}
async function drop(docId: string, chunkId: string): Promise<void> {
saving.value = true
try {
await api.dropChunk(docId, chunkId)
chunks.value = chunks.value.filter((c) => c.id !== chunkId)
} catch (e) {
error.value = (e as Error).message || 'Failed to drop chunk'
console.error('Failed to drop chunk', e)
} finally {
saving.value = false
}
}
async function add(docId: string, text: string, afterId?: string): Promise<void> {
saving.value = true
try {
const created = await api.addChunk(docId, text, afterId)
if (afterId) {
const idx = chunks.value.findIndex((c) => c.id === afterId)
if (idx !== -1) {
chunks.value = [
...chunks.value.slice(0, idx + 1),
created,
...chunks.value.slice(idx + 1),
]
return
}
}
chunks.value = [...chunks.value, created]
} catch (e) {
error.value = (e as Error).message || 'Failed to add chunk'
console.error('Failed to add chunk', e)
} finally {
saving.value = false
}
}
async function loadDiff(docId: string, store: string): Promise<void> {
diffing.value = true
error.value = null
try {
diff.value = await api.fetchChunkDiff(docId, store)
} catch (e) {
error.value = (e as Error).message || 'Failed to load diff'
console.error('Failed to load diff', e)
} finally {
diffing.value = false
}
}
function clearDiff(): void {
diff.value = []
}
async function push(docId: string, store: string): Promise<string | null> {
try {
const res = await api.pushChunksToStore(docId, store)
return res.jobId
} catch (e) {
error.value = (e as Error).message || 'Failed to push chunks'
console.error('Failed to push chunks', e)
return null
}
}
return {
chunks,
loading,
saving,
diffing,
diff,
error,
clearError,
load,
updateText,
updateTitle,
merge,
split,
drop,
add,
loadDiff,
clearDiff,
push,
}
})

View file

@ -0,0 +1,382 @@
<template>
<div
class="chunk-item"
:class="[
`chunk-item--${diffStatus}`,
{ 'chunk-item--selected': selected, 'chunk-item--editing': editing },
]"
data-e2e="chunk-item"
@click="$emit('selectChunk', chunk.id)"
>
<!-- Selection checkbox -->
<input
type="checkbox"
class="chunk-select"
:checked="selected"
@change.stop="$emit('toggleSelect', chunk.id)"
@click.stop
/>
<!-- Body -->
<div class="chunk-body">
<!-- Title row -->
<div class="chunk-title-row">
<span
v-if="!editingTitle"
class="chunk-title"
:class="{ 'chunk-title--empty': !chunk.title }"
@dblclick.stop="startEditTitle"
>
{{ chunk.title || t('chunks.noTitle') }}
</span>
<input
v-else
ref="titleInput"
v-model="draftTitle"
class="chunk-title-input"
@blur="commitTitle"
@keydown.enter.prevent="commitTitle"
@keydown.escape.prevent="cancelTitle"
@click.stop
/>
<span class="chunk-meta">
<span v-if="chunk.pageRange">p.{{ chunk.pageRange[0] }}{{ chunk.pageRange[1] }}</span>
<span v-if="chunk.tokenCount">{{ chunk.tokenCount }}t</span>
<span v-if="savingThis" class="saving-indicator">{{ t('chunks.saving') }}</span>
</span>
</div>
<!-- Text -->
<textarea
v-if="editing"
ref="textArea"
v-model="draftText"
class="chunk-text-edit"
:placeholder="t('chunks.editPlaceholder')"
rows="4"
@blur="commitText"
@click.stop
@keydown.escape.prevent="cancelText"
/>
<p
v-else
class="chunk-text"
:class="{ 'chunk-text--removed': diffStatus === 'removed' }"
@dblclick.stop="startEdit"
>
{{ chunk.text }}
</p>
<!-- Diff inline text diff -->
<div v-if="diffStatus === 'modified' && chunk.id && diffEntry?.textDiff" class="chunk-diff">
<code class="diff-text">{{ diffEntry.textDiff }}</code>
</div>
</div>
<!-- Actions menu -->
<div class="chunk-actions" @click.stop>
<button
class="chunk-action-btn"
:title="t('chunks.mergePrev')"
:disabled="isFirst"
@click="$emit('mergePrev', chunk.id)"
>
</button>
<button
class="chunk-action-btn"
:title="t('chunks.mergeNext')"
:disabled="isLast"
@click="$emit('mergeNext', chunk.id)"
>
</button>
<button class="chunk-action-btn" :title="t('chunks.split')" @click="splitAtCurrentCursor">
</button>
<button class="chunk-action-btn" :title="t('chunks.add')" @click="$emit('add', chunk.id)">
+
</button>
<button
class="chunk-action-btn chunk-action-btn--danger"
:title="t('chunks.drop')"
@click="$emit('drop', chunk.id)"
>
</button>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, computed, nextTick } from 'vue'
import type { DocChunk, ChunkDiff, ChunkDiffStatus } from '../../../shared/types'
import { useI18n } from '../../../shared/i18n'
const props = defineProps<{
chunk: DocChunk
selected?: boolean
savingThis?: boolean
isFirst?: boolean
isLast?: boolean
diffEntry?: ChunkDiff | null
}>()
const emit = defineEmits<{
selectChunk: [id: string]
toggleSelect: [id: string]
textChange: [id: string, text: string]
titleChange: [id: string, title: string]
mergePrev: [id: string]
mergeNext: [id: string]
split: [id: string, cursorOffset: number]
drop: [id: string]
add: [afterId: string]
}>()
const { t } = useI18n()
const editing = ref(false)
const editingTitle = ref(false)
const draftText = ref('')
const draftTitle = ref('')
const textArea = ref<HTMLTextAreaElement | null>(null)
const titleInput = ref<HTMLInputElement | null>(null)
const diffStatus = computed<ChunkDiffStatus>(() => props.diffEntry?.status ?? 'unchanged')
function startEdit(): void {
draftText.value = props.chunk.text
editing.value = true
nextTick(() => textArea.value?.focus())
}
function commitText(): void {
if (draftText.value !== props.chunk.text) {
emit('textChange', props.chunk.id, draftText.value)
}
editing.value = false
}
function cancelText(): void {
editing.value = false
}
function startEditTitle(): void {
draftTitle.value = props.chunk.title ?? ''
editingTitle.value = true
nextTick(() => titleInput.value?.focus())
}
function commitTitle(): void {
if (draftTitle.value !== (props.chunk.title ?? '')) {
emit('titleChange', props.chunk.id, draftTitle.value)
}
editingTitle.value = false
}
function cancelTitle(): void {
editingTitle.value = false
}
function splitAtCurrentCursor(): void {
if (!textArea.value) {
emit('split', props.chunk.id, Math.floor(props.chunk.text.length / 2))
return
}
emit('split', props.chunk.id, textArea.value.selectionStart)
}
</script>
<style scoped>
.chunk-item {
display: flex;
gap: 10px;
padding: 12px 14px;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
background: var(--bg-elevated);
transition:
border-color var(--transition),
background var(--transition);
cursor: default;
}
.chunk-item:hover {
border-color: var(--border-hover, var(--border));
}
.chunk-item--selected {
border-color: var(--accent);
background: var(--accent-muted);
}
.chunk-item--added {
border-color: var(--success);
background: color-mix(in srgb, var(--success) 8%, transparent);
}
.chunk-item--modified {
border-color: var(--warning);
background: color-mix(in srgb, var(--warning) 8%, transparent);
}
.chunk-item--removed {
border-color: var(--error);
background: color-mix(in srgb, var(--error) 8%, transparent);
opacity: 0.7;
}
.chunk-select {
margin-top: 2px;
flex-shrink: 0;
cursor: pointer;
accent-color: var(--accent);
}
.chunk-body {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
gap: 6px;
}
.chunk-title-row {
display: flex;
align-items: center;
gap: 8px;
}
.chunk-title {
font-size: 12px;
font-weight: 600;
color: var(--text-secondary);
cursor: text;
}
.chunk-title--empty {
color: var(--text-muted);
font-style: italic;
font-weight: 400;
}
.chunk-title-input {
font-size: 12px;
font-weight: 600;
background: var(--bg-surface);
border: 1px solid var(--accent);
border-radius: var(--radius-sm);
padding: 1px 6px;
color: var(--text);
flex: 1;
outline: none;
}
.chunk-meta {
display: flex;
gap: 6px;
font-size: 11px;
font-family: 'IBM Plex Mono', monospace;
color: var(--text-muted);
flex-shrink: 0;
}
.saving-indicator {
color: var(--accent);
animation: blink 1s step-end infinite;
}
@keyframes blink {
50% {
opacity: 0.4;
}
}
.chunk-text {
font-size: 13px;
color: var(--text);
line-height: 1.6;
white-space: pre-wrap;
word-break: break-word;
cursor: text;
margin: 0;
}
.chunk-text--removed {
text-decoration: line-through;
color: var(--error);
}
.chunk-text-edit {
width: 100%;
font-size: 13px;
font-family: inherit;
color: var(--text);
background: var(--bg-surface);
border: 1px solid var(--accent);
border-radius: var(--radius-sm);
padding: 6px 8px;
resize: vertical;
outline: none;
line-height: 1.6;
}
.chunk-diff {
padding: 4px 8px;
background: var(--bg-surface);
border-radius: var(--radius-sm);
border-left: 3px solid var(--warning);
}
.diff-text {
font-size: 11px;
font-family: 'IBM Plex Mono', monospace;
color: var(--text-secondary);
white-space: pre-wrap;
}
.chunk-actions {
display: flex;
flex-direction: column;
gap: 2px;
flex-shrink: 0;
opacity: 0;
transition: opacity var(--transition);
}
.chunk-item:hover .chunk-actions,
.chunk-item--selected .chunk-actions {
opacity: 1;
}
.chunk-action-btn {
width: 22px;
height: 22px;
display: flex;
align-items: center;
justify-content: center;
font-size: 13px;
background: var(--bg-surface);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
color: var(--text-muted);
transition: all var(--transition);
}
.chunk-action-btn:hover:not(:disabled) {
border-color: var(--accent);
color: var(--accent);
}
.chunk-action-btn:disabled {
opacity: 0.3;
cursor: not-allowed;
}
.chunk-action-btn--danger:hover:not(:disabled) {
border-color: var(--error);
color: var(--error);
}
</style>

View file

@ -0,0 +1,618 @@
<template>
<div class="chunks-editor" data-e2e="chunks-editor">
<!-- Toolbar -->
<div class="chunks-toolbar">
<div class="toolbar-left">
<span class="chunk-count">{{ t('chunks.count', { n: chunksStore.chunks.length }) }}</span>
<span v-if="chunksStore.saving" class="toolbar-saving">{{ t('chunks.saving') }}</span>
</div>
<div class="toolbar-right">
<!-- Diff toggle (#221) -->
<button
class="toolbar-btn"
:class="{ active: showDiff }"
:disabled="!availableStores.length"
@click="toggleDiff"
>
{{ showDiff ? t('chunks.diffClose') : t('chunks.diffToggle') }}
</button>
<!-- Push to store (#222) -->
<button class="toolbar-btn toolbar-btn--primary" @click="openPushModal">
{{ t('chunks.pushTitle') }}
</button>
</div>
</div>
<!-- Diff store selector -->
<div v-if="showDiff" class="diff-bar">
<label class="diff-label">{{ t('chunks.diffRef') }}</label>
<select v-model="diffStore" class="diff-store-select" @change="reloadDiff">
<option v-for="s in availableStores" :key="s" :value="s">{{ s }}</option>
</select>
<span v-if="chunksStore.diffing" class="diff-loading"></span>
<span v-else class="diff-counts">
<span class="diff-added">+{{ diffSummary.added }}</span>
<span class="diff-modified">~{{ diffSummary.modified }}</span>
<span class="diff-removed">-{{ diffSummary.removed }}</span>
</span>
</div>
<!-- Bulk selection bar -->
<div v-if="selectedIds.size > 0" class="bulk-bar" data-e2e="bulk-bar">
<span class="bulk-count">{{ t('chunks.selected', { n: selectedIds.size }) }}</span>
<button class="bulk-btn" @click="bulkDrop">{{ t('chunks.bulkDrop') }}</button>
<button class="bulk-btn" :disabled="selectedIds.size < 2" @click="bulkMerge">
{{ t('chunks.bulkMerge') }}
</button>
<button class="bulk-btn bulk-btn--cancel" @click="selectedIds = new Set()">
{{ t('chunks.bulkCancel') }}
</button>
</div>
<!-- Empty state -->
<div
v-if="!chunksStore.loading && !chunksStore.chunks.length"
class="chunks-empty"
data-e2e="chunks-empty"
>
<p>{{ t('chunks.empty') }}</p>
</div>
<!-- Loading -->
<div v-else-if="chunksStore.loading" class="chunks-loading">
<span class="spinner" />
</div>
<!-- Chunk list -->
<div v-else class="chunk-list" data-e2e="chunk-list">
<ChunkItem
v-for="(chunk, idx) in chunksStore.chunks"
:key="chunk.id"
:chunk="chunk"
:selected="selectedIds.has(chunk.id)"
:saving-this="savingId === chunk.id"
:is-first="idx === 0"
:is-last="idx === chunksStore.chunks.length - 1"
:diff-entry="diffMap.get(chunk.id) ?? null"
@select-chunk="onSelectChunk"
@toggle-select="onToggleSelect"
@text-change="onTextChange"
@title-change="onTitleChange"
@merge-prev="(id) => onMerge(id, 'prev')"
@merge-next="(id) => onMerge(id, 'next')"
@split="onSplit"
@drop="onDrop"
@add="onAdd"
/>
<!-- Add chunk at end -->
<button class="add-chunk-end" @click="onAdd(undefined)">+ {{ t('chunks.add') }}</button>
</div>
<!-- Push to store modal (#222) -->
<div v-if="pushModalOpen" class="modal-backdrop" @click.self="closePushModal">
<div class="modal" role="dialog" :aria-label="t('chunks.pushTitle')" aria-modal="true">
<h2 class="modal-title">{{ t('chunks.pushTitle') }}</h2>
<label class="modal-label">{{ t('chunks.pushStore') }}</label>
<input
ref="pushInput"
v-model="pushStoreName"
class="modal-input"
list="push-store-datalist"
:placeholder="t('chunks.pushPlaceholder')"
@keydown.enter.prevent="confirmPush"
@keydown.escape.prevent="closePushModal"
/>
<datalist id="push-store-datalist">
<option v-for="s in availableStores" :key="s" :value="s" />
</datalist>
<div v-if="pushSummary" class="push-summary">
{{ t('chunks.pushSummary', { embeds: pushSummary.embeds, tokens: pushSummary.tokens }) }}
</div>
<div class="modal-actions">
<button class="modal-btn modal-btn--cancel" @click="closePushModal">
{{ t('chunks.pushCancel') }}
</button>
<button
class="modal-btn modal-btn--primary"
:disabled="!pushStoreName.trim() || pushing"
@click="confirmPush"
>
{{ pushing ? '…' : t('chunks.pushConfirm') }}
</button>
</div>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, computed, nextTick, onMounted } from 'vue'
import { useChunksStore } from '../store'
import { useI18n } from '../../../shared/i18n'
import type { ChunkDiff } from '../../../shared/types'
import ChunkItem from './ChunkItem.vue'
const props = defineProps<{
docId: string
availableStores: string[]
}>()
const emit = defineEmits<{
nodeHighlight: [ref: string | null]
}>()
const { t } = useI18n()
const chunksStore = useChunksStore()
const selectedIds = ref<Set<string>>(new Set())
const savingId = ref<string | null>(null)
// --- Diff state (#221) ---
const showDiff = ref(false)
const diffStore = ref(props.availableStores[0] ?? '')
const diffMap = computed<Map<string, ChunkDiff>>(() => {
const m = new Map<string, ChunkDiff>()
if (!showDiff.value) return m
for (const d of chunksStore.diff) m.set(d.chunkId, d)
return m
})
const diffSummary = computed(() => {
const counts = { added: 0, modified: 0, removed: 0 }
for (const d of chunksStore.diff) {
if (d.status === 'added') counts.added++
else if (d.status === 'modified') counts.modified++
else if (d.status === 'removed') counts.removed++
}
return counts
})
async function toggleDiff(): Promise<void> {
if (showDiff.value) {
showDiff.value = false
chunksStore.clearDiff()
return
}
if (!diffStore.value) diffStore.value = props.availableStores[0] ?? ''
showDiff.value = true
await chunksStore.loadDiff(props.docId, diffStore.value)
}
async function reloadDiff(): Promise<void> {
if (showDiff.value && diffStore.value) {
await chunksStore.loadDiff(props.docId, diffStore.value)
}
}
// --- Push to store (#222) ---
const pushModalOpen = ref(false)
const pushStoreName = ref('')
const pushing = ref(false)
const pushSummary = ref<{ embeds: number; tokens: number } | null>(null)
const pushInput = ref<HTMLInputElement | null>(null)
function openPushModal(): void {
pushStoreName.value = props.availableStores[0] ?? ''
pushSummary.value = null
pushModalOpen.value = true
nextTick(() => pushInput.value?.focus())
}
function closePushModal(): void {
pushModalOpen.value = false
pushing.value = false
}
async function confirmPush(): Promise<void> {
const store = pushStoreName.value.trim()
if (!store || pushing.value) return
pushing.value = true
const jobId = await chunksStore.push(props.docId, store)
pushing.value = false
if (jobId) {
closePushModal()
alert(t('chunks.pushedJob', { jobId }))
}
}
// --- Chunk interactions (#219 / #220) ---
function onSelectChunk(id: string): void {
const chunk = chunksStore.chunks.find((c) => c.id === id)
emit('nodeHighlight', chunk?.sourceNodeRef ?? null)
}
function onToggleSelect(id: string): void {
const next = new Set(selectedIds.value)
if (next.has(id)) {
next.delete(id)
} else {
next.add(id)
}
selectedIds.value = next
}
let saveTimer: ReturnType<typeof setTimeout> | null = null
function onTextChange(id: string, text: string): void {
if (saveTimer) clearTimeout(saveTimer)
savingId.value = id
saveTimer = setTimeout(async () => {
await chunksStore.updateText(props.docId, id, text)
savingId.value = null
saveTimer = null
}, 600)
}
async function onTitleChange(id: string, title: string): Promise<void> {
await chunksStore.updateTitle(props.docId, id, title)
}
async function onMerge(id: string, dir: 'prev' | 'next'): Promise<void> {
const idx = chunksStore.chunks.findIndex((c) => c.id === id)
if (idx === -1) return
const other = dir === 'prev' ? chunksStore.chunks[idx - 1] : chunksStore.chunks[idx + 1]
if (!other) return
const ids = dir === 'prev' ? [other.id, id] : [id, other.id]
await chunksStore.merge(props.docId, ids)
}
async function onSplit(id: string, cursorOffset: number): Promise<void> {
await chunksStore.split(props.docId, id, cursorOffset)
}
async function onDrop(id: string): Promise<void> {
await chunksStore.drop(props.docId, id)
const next = new Set(selectedIds.value)
next.delete(id)
selectedIds.value = next
}
async function onAdd(afterId: string | undefined): Promise<void> {
await chunksStore.add(props.docId, '', afterId)
}
// --- Bulk actions (#220) ---
async function bulkDrop(): Promise<void> {
const ids = [...selectedIds.value]
for (const id of ids) await chunksStore.drop(props.docId, id)
selectedIds.value = new Set()
}
async function bulkMerge(): Promise<void> {
const ids = [...selectedIds.value]
if (ids.length < 2) return
const ordered = chunksStore.chunks.filter((c) => selectedIds.value.has(c.id)).map((c) => c.id)
await chunksStore.merge(props.docId, ordered)
selectedIds.value = new Set()
}
onMounted(() => {
chunksStore.load(props.docId)
})
</script>
<style scoped>
.chunks-editor {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
}
.chunks-toolbar {
display: flex;
align-items: center;
justify-content: space-between;
padding: 8px 16px;
border-bottom: 1px solid var(--border);
background: var(--bg-surface);
flex-shrink: 0;
}
.toolbar-left {
display: flex;
align-items: center;
gap: 12px;
}
.toolbar-right {
display: flex;
align-items: center;
gap: 8px;
}
.chunk-count {
font-size: 12px;
font-family: 'IBM Plex Mono', monospace;
color: var(--text-muted);
}
.toolbar-saving {
font-size: 11px;
color: var(--accent);
}
.toolbar-btn {
padding: 4px 10px;
font-size: 12px;
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
color: var(--text-secondary);
transition: all var(--transition);
}
.toolbar-btn:hover:not(:disabled) {
border-color: var(--accent);
color: var(--accent);
}
.toolbar-btn.active {
background: var(--accent-muted);
border-color: var(--accent);
color: var(--accent);
}
.toolbar-btn:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.toolbar-btn--primary {
background: var(--accent);
border-color: var(--accent);
color: white;
}
.toolbar-btn--primary:hover:not(:disabled) {
background: var(--accent-hover);
border-color: var(--accent-hover);
color: white;
}
/* Diff bar */
.diff-bar {
display: flex;
align-items: center;
gap: 10px;
padding: 6px 16px;
background: color-mix(in srgb, var(--warning) 5%, var(--bg-surface));
border-bottom: 1px solid var(--warning);
font-size: 12px;
flex-shrink: 0;
}
.diff-label {
color: var(--text-muted);
font-size: 11px;
}
.diff-store-select {
font-size: 12px;
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 2px 6px;
color: var(--text);
}
.diff-loading {
color: var(--text-muted);
}
.diff-counts {
display: flex;
gap: 8px;
font-family: 'IBM Plex Mono', monospace;
font-size: 11px;
}
.diff-added {
color: var(--success);
}
.diff-modified {
color: var(--warning);
}
.diff-removed {
color: var(--error);
}
/* Bulk bar */
.bulk-bar {
display: flex;
align-items: center;
gap: 8px;
padding: 8px 16px;
background: var(--accent-muted);
border-bottom: 1px solid var(--accent);
flex-shrink: 0;
}
.bulk-count {
font-size: 12px;
color: var(--accent);
font-weight: 500;
flex: 1;
}
.bulk-btn {
padding: 3px 10px;
font-size: 12px;
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
color: var(--text-secondary);
transition: all var(--transition);
}
.bulk-btn:hover:not(:disabled) {
border-color: var(--error);
color: var(--error);
}
.bulk-btn:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.bulk-btn--cancel:hover {
border-color: var(--border);
color: var(--text-muted);
}
/* States */
.chunks-empty,
.chunks-loading {
display: flex;
align-items: center;
justify-content: center;
flex: 1;
color: var(--text-muted);
font-size: 13px;
}
/* List */
.chunk-list {
flex: 1;
overflow-y: auto;
padding: 12px 16px;
display: flex;
flex-direction: column;
gap: 8px;
}
.add-chunk-end {
padding: 8px;
font-size: 12px;
color: var(--text-muted);
background: none;
border: 1px dashed var(--border-light);
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
}
.add-chunk-end:hover {
border-color: var(--accent);
color: var(--accent);
}
/* Push modal */
.modal-backdrop {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.4);
display: flex;
align-items: center;
justify-content: center;
z-index: 100;
}
.modal {
background: var(--bg-surface);
border: 1px solid var(--border);
border-radius: var(--radius);
padding: 24px;
width: 380px;
max-width: 90vw;
display: flex;
flex-direction: column;
gap: 12px;
}
.modal-title {
font-size: 15px;
font-weight: 600;
color: var(--text);
}
.modal-label {
font-size: 12px;
color: var(--text-secondary);
}
.modal-input {
width: 100%;
font-size: 13px;
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 7px 10px;
color: var(--text);
outline: none;
transition: border-color var(--transition);
box-sizing: border-box;
}
.modal-input:focus {
border-color: var(--accent);
}
.push-summary {
font-size: 12px;
font-family: 'IBM Plex Mono', monospace;
color: var(--text-muted);
background: var(--bg-elevated);
border-radius: var(--radius-sm);
padding: 6px 10px;
}
.modal-actions {
display: flex;
justify-content: flex-end;
gap: 8px;
}
.modal-btn {
padding: 6px 14px;
font-size: 13px;
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
}
.modal-btn--cancel {
background: none;
border: 1px solid var(--border);
color: var(--text-secondary);
}
.modal-btn--cancel:hover {
border-color: var(--text-muted);
}
.modal-btn--primary {
background: var(--accent);
border: 1px solid var(--accent);
color: white;
}
.modal-btn--primary:hover:not(:disabled) {
background: var(--accent-hover);
}
.modal-btn--primary:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.spinner {
width: 24px;
height: 24px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,143 @@
<template>
<div v-if="storeLinks.length > 0" class="stale-strip" data-e2e="stale-strip">
<div class="stale-strip__stores">
<div v-for="link in storeLinks" :key="link.store" class="store-row">
<span class="store-name">{{ link.store }}</span>
<StatusBadge :state="link.state" compact />
<span v-if="link.pushedAt" class="store-date">
{{ t('chunks.stale.pushedAt', { date: formatRelativeTime(link.pushedAt, locale) }) }}
</span>
<button
v-if="link.state === 'Stale'"
class="btn-reingest"
:disabled="reingestingStores.has(link.store)"
@click="reingest(link.store)"
>
{{ t('chunks.stale.reingest') }}
</button>
</div>
</div>
<button
v-if="staleCount > 1"
class="btn-reingest-all"
:disabled="reingestingStores.size > 0"
@click="reingestAll"
>
{{ t('chunks.stale.reingestAll') }}
</button>
</div>
</template>
<script setup lang="ts">
import { ref, computed } from 'vue'
import type { DocStoreLink } from '../../../shared/types'
import { pushChunksToStore } from '../api'
import StatusBadge from '../../document/ui/StatusBadge.vue'
import { useI18n } from '../../../shared/i18n'
import { formatRelativeTime } from '../../../shared/format'
import { appLocale } from '../../../shared/appConfig'
const props = defineProps<{
docId: string
storeLinks: DocStoreLink[]
}>()
const { t } = useI18n()
const locale = appLocale
const reingestingStores = ref<Set<string>>(new Set())
const staleCount = computed(() => props.storeLinks.filter((l) => l.state === 'Stale').length)
async function reingest(store: string): Promise<void> {
const next = new Set(reingestingStores.value)
next.add(store)
reingestingStores.value = next
try {
await pushChunksToStore(props.docId, store)
} catch {
// silent user can retry
} finally {
const after = new Set(reingestingStores.value)
after.delete(store)
reingestingStores.value = after
}
}
async function reingestAll(): Promise<void> {
const stale = props.storeLinks.filter((l) => l.state === 'Stale').map((l) => l.store)
await Promise.all(stale.map(reingest))
}
</script>
<style scoped>
.stale-strip {
display: flex;
align-items: center;
gap: 12px;
padding: 6px 16px;
background: var(--bg-surface);
border-bottom: 1px solid var(--border);
flex-wrap: wrap;
flex-shrink: 0;
}
.stale-strip__stores {
display: flex;
flex-wrap: wrap;
gap: 8px;
flex: 1;
}
.store-row {
display: flex;
align-items: center;
gap: 6px;
padding: 2px 8px;
background: var(--bg);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
font-size: 12px;
}
.store-name {
font-weight: 500;
color: var(--text-secondary);
}
.store-date {
color: var(--text-muted);
font-size: 11px;
}
.btn-reingest,
.btn-reingest-all {
padding: 2px 8px;
font-size: 11px;
font-weight: 500;
background: var(--warning-bg, rgba(234, 179, 8, 0.1));
color: var(--warning, #ca8a04);
border: 1px solid var(--warning, #ca8a04);
border-radius: var(--radius-sm);
cursor: pointer;
transition: opacity var(--transition);
white-space: nowrap;
}
.btn-reingest:disabled,
.btn-reingest-all:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.btn-reingest:hover:not(:disabled),
.btn-reingest-all:hover:not(:disabled) {
opacity: 0.8;
}
.btn-reingest-all {
flex-shrink: 0;
padding: 4px 10px;
font-size: 12px;
}
</style>

View file

@ -1,5 +1,13 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import { fetchDocuments, fetchDocument, uploadDocument, deleteDocument, getPreviewUrl } from './api'
import {
fetchDocuments,
fetchDocument,
uploadDocument,
deleteDocument,
getPreviewUrl,
rechunkDocument,
fetchDocumentTree,
} from './api'
vi.mock('../../shared/api/http', () => ({
apiFetch: vi.fn(),
@ -62,4 +70,27 @@ describe('document API', () => {
it('getPreviewUrl accepts custom page and dpi', () => {
expect(getPreviewUrl('abc', 3, 300)).toBe('/api/documents/abc/preview?page=3&dpi=300')
})
it('rechunkDocument calls POST /api/documents/:id/rechunk and returns chunks', async () => {
const chunks = [{ id: 'c1' }, { id: 'c2' }]
apiFetch.mockResolvedValue(chunks)
const result = await rechunkDocument('42')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/42/rechunk', {
method: 'POST',
body: JSON.stringify({}),
})
expect(result).toEqual(chunks)
})
it('fetchDocumentTree calls GET /api/documents/:id/tree', async () => {
const tree = [{ ref: '#/body', type: 'body', label: 'Body', children: [] }]
apiFetch.mockResolvedValue(tree)
const result = await fetchDocumentTree('42')
expect(apiFetch).toHaveBeenCalledWith('/api/documents/42/tree')
expect(result).toEqual(tree)
})
})

View file

@ -1,4 +1,4 @@
import type { Document } from '../../shared/types'
import type { DocChunk, Document, DocTreeNode } from '../../shared/types'
import { apiFetch } from '../../shared/api/http'
export function fetchDocuments(): Promise<Document[]> {
@ -26,3 +26,16 @@ export function deleteDocument(id: string): Promise<unknown> {
export function getPreviewUrl(id: string, page = 1, dpi = 150): string {
return `/api/documents/${id}/preview?page=${page}&dpi=${dpi}`
}
/** Rechunk the canonical chunkset. Backend runs synchronously and returns
* the new chunks there is no async job to poll. */
export function rechunkDocument(id: string): Promise<DocChunk[]> {
return apiFetch<DocChunk[]>(`/api/documents/${id}/rechunk`, {
method: 'POST',
body: JSON.stringify({}),
})
}
export function fetchDocumentTree(id: string): Promise<DocTreeNode[]> {
return apiFetch<DocTreeNode[]>(`/api/documents/${id}/tree`)
}

View file

@ -6,9 +6,15 @@ vi.mock('./api', () => ({
fetchDocuments: vi.fn(),
uploadDocument: vi.fn(),
deleteDocument: vi.fn(),
rechunkDocument: vi.fn(),
}))
vi.mock('../chunks/api', () => ({
pushChunksToStore: vi.fn(),
}))
import * as api from './api'
import * as chunksApi from '../chunks/api'
describe('useDocumentStore', () => {
beforeEach(() => {
@ -120,4 +126,54 @@ describe('useDocumentStore', () => {
store.select('42')
expect(store.selectedId).toBe('42')
})
it('load() sets loading to false after success', async () => {
api.fetchDocuments.mockResolvedValue([])
const store = useDocumentStore()
await store.load()
expect(store.loading).toBe(false)
})
it('load() sets loading to false after error', async () => {
api.fetchDocuments.mockRejectedValue(new Error('fail'))
vi.spyOn(console, 'error').mockImplementation(() => {})
const store = useDocumentStore()
await store.load()
expect(store.loading).toBe(false)
})
it('rechunk() returns chunk count on success', async () => {
api.rechunkDocument.mockResolvedValue([{ id: 'c1' }, { id: 'c2' }, { id: 'c3' }])
const store = useDocumentStore()
const result = await store.rechunk('42')
expect(api.rechunkDocument).toHaveBeenCalledWith('42')
expect(result).toBe(3)
})
it('rechunk() returns null on error', async () => {
api.rechunkDocument.mockRejectedValue(new Error('fail'))
vi.spyOn(console, 'error').mockImplementation(() => {})
const store = useDocumentStore()
const result = await store.rechunk('42')
expect(result).toBeNull()
})
it('pushToStore() delegates to chunks/api.pushChunksToStore and returns jobId', async () => {
chunksApi.pushChunksToStore.mockResolvedValue({
jobId: 'j2',
summary: { embeds: 5, tokens: 50 },
})
const store = useDocumentStore()
const result = await store.pushToStore('42', 'my-store')
expect(chunksApi.pushChunksToStore).toHaveBeenCalledWith('42', 'my-store')
expect(result).toBe('j2')
})
it('pushToStore() returns null on error', async () => {
chunksApi.pushChunksToStore.mockRejectedValue(new Error('fail'))
vi.spyOn(console, 'error').mockImplementation(() => {})
const store = useDocumentStore()
const result = await store.pushToStore('42', 'my-store')
expect(result).toBeNull()
})
})

View file

@ -2,11 +2,13 @@ import { defineStore } from 'pinia'
import { ref } from 'vue'
import type { Document } from '../../shared/types'
import { appMaxFileSizeMb } from '../../shared/appConfig'
import { pushChunksToStore } from '../chunks/api'
import * as api from './api'
export const useDocumentStore = defineStore('document', () => {
const documents = ref<Document[]>([])
const selectedId = ref<string | null>(null)
const loading = ref(false)
const uploading = ref(false)
const error = ref<string | null>(null)
@ -15,12 +17,15 @@ export const useDocumentStore = defineStore('document', () => {
}
async function load(): Promise<void> {
loading.value = true
try {
error.value = null
documents.value = await api.fetchDocuments()
} catch (e) {
error.value = (e as Error).message || 'Failed to load documents'
console.error('Failed to load documents', e)
} finally {
loading.value = false
}
}
@ -61,5 +66,40 @@ export const useDocumentStore = defineStore('document', () => {
selectedId.value = id
}
return { documents, selectedId, uploading, error, clearError, load, upload, remove, select }
async function rechunk(id: string): Promise<number | null> {
try {
const chunks = await api.rechunkDocument(id)
return chunks.length
} catch (e) {
error.value = (e as Error).message || 'Failed to rechunk'
console.error('Rechunk failed', e)
return null
}
}
async function pushToStore(id: string, store: string): Promise<string | null> {
try {
const res = await pushChunksToStore(id, store)
return res.jobId
} catch (e) {
error.value = (e as Error).message || 'Failed to push to store'
console.error('Push to store failed', e)
return null
}
}
return {
documents,
selectedId,
loading,
uploading,
error,
clearError,
load,
upload,
remove,
select,
rechunk,
pushToStore,
}
})

View file

@ -0,0 +1,139 @@
<template>
<li class="tree-node" role="treeitem" :aria-expanded="hasChildren ? open : undefined">
<button
class="tree-node-row"
:class="{
'tree-node-row--selected': selected === node.ref,
'tree-node-row--highlight': highlight === node.ref,
}"
@click="onRowClick"
>
<span class="tree-node-indent" :style="{ width: `${depth * 12}px` }" />
<span v-if="hasChildren" class="tree-node-toggle" :class="{ open }" @click.stop="open = !open"
></span
>
<span v-else class="tree-node-toggle-placeholder" />
<span class="tree-node-type">{{ node.type }}</span>
<span class="tree-node-label" :title="node.label">{{ node.label }}</span>
</button>
<ul v-if="hasChildren && open" class="tree-list" role="group">
<DocTreeNode
v-for="child in node.children"
:key="child.ref"
:node="child"
:depth="depth + 1"
:selected="selected"
:highlight="highlight"
@select="$emit('select', $event)"
/>
</ul>
</li>
</template>
<script setup lang="ts">
import { ref, computed } from 'vue'
import type { DocTreeNode } from '../../../shared/types'
const props = withDefaults(
defineProps<{
node: DocTreeNode
depth?: number
selected?: string | null
highlight?: string | null
}>(),
{ depth: 0 },
)
const emit = defineEmits<{
select: [ref: string]
}>()
const open = ref(props.depth < 2)
const hasChildren = computed(() => props.node.children.length > 0)
function onRowClick(): void {
emit('select', props.node.ref)
}
</script>
<style scoped>
.tree-node {
list-style: none;
}
.tree-list {
list-style: none;
}
.tree-node-row {
display: flex;
align-items: center;
gap: 4px;
width: 100%;
padding: 3px 8px 3px 4px;
background: none;
border: none;
cursor: pointer;
font-size: 12px;
color: var(--text-secondary);
text-align: left;
border-radius: 0;
transition: background var(--transition);
}
.tree-node-row:hover {
background: var(--bg-elevated);
color: var(--text);
}
.tree-node-row--selected {
background: var(--accent-muted);
color: var(--accent);
}
.tree-node-row--highlight {
background: var(--warning-muted, rgba(234, 179, 8, 0.1));
color: var(--text);
}
.tree-node-toggle {
display: inline-flex;
align-items: center;
justify-content: center;
width: 14px;
font-size: 12px;
color: var(--text-muted);
transform: rotate(0deg);
transition: transform 0.15s;
flex-shrink: 0;
}
.tree-node-toggle.open {
transform: rotate(90deg);
}
.tree-node-toggle-placeholder {
width: 14px;
flex-shrink: 0;
}
.tree-node-type {
font-family: 'IBM Plex Mono', monospace;
font-size: 10px;
color: var(--text-muted);
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: 3px;
padding: 0 4px;
flex-shrink: 0;
white-space: nowrap;
}
.tree-node-label {
flex: 1;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
</style>

View file

@ -0,0 +1,107 @@
<template>
<div class="tree-rail" data-e2e="tree-rail">
<div v-if="loading" class="tree-rail-placeholder">
<span class="spinner" />
</div>
<div v-else-if="error" class="tree-rail-placeholder tree-rail-error">
<span>{{ error }}</span>
<button class="retry-btn" @click="$emit('reload')"></button>
</div>
<div v-else-if="!nodes.length" class="tree-rail-placeholder">
<span class="tree-rail-empty">{{ t('tree.empty') }}</span>
</div>
<ul v-else class="tree-list" role="tree">
<TreeNode
v-for="node in nodes"
:key="node.ref"
:node="node"
:selected="selected"
:highlight="highlight"
@select="(ref) => $emit('select', ref)"
/>
</ul>
</div>
</template>
<script setup lang="ts">
import type { DocTreeNode } from '../../../shared/types'
import { useI18n } from '../../../shared/i18n'
import TreeNode from './DocTreeNode.vue'
defineProps<{
nodes: DocTreeNode[]
loading?: boolean
error?: string | null
selected?: string | null
highlight?: string | null
}>()
defineEmits<{
select: [ref: string]
reload: []
}>()
const { t } = useI18n()
</script>
<style scoped>
.tree-rail {
display: flex;
flex-direction: column;
height: 100%;
overflow-y: auto;
background: var(--bg-surface);
border-right: 1px solid var(--border);
}
.tree-rail-placeholder {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 8px;
height: 100%;
padding: 16px;
color: var(--text-muted);
font-size: 13px;
}
.tree-rail-error {
color: var(--error);
}
.tree-list {
list-style: none;
padding: 8px 0;
}
.retry-btn {
background: none;
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 2px 8px;
cursor: pointer;
font-size: 13px;
color: var(--text-secondary);
}
.spinner {
width: 20px;
height: 20px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
.tree-rail-empty {
font-size: 12px;
color: var(--text-muted);
}
</style>

View file

@ -0,0 +1,123 @@
<template>
<header class="workspace-header" data-e2e="workspace-header">
<div class="workspace-header-main">
<svg class="doc-icon" viewBox="0 0 20 20" fill="currentColor">
<path
fill-rule="evenodd"
d="M4 4a2 2 0 012-2h4.586A2 2 0 0112 2.586L15.414 6A2 2 0 0116 7.414V16a2 2 0 01-2 2H6a2 2 0 01-2-2V4z"
clip-rule="evenodd"
/>
</svg>
<h1 class="workspace-title" :title="doc.filename">{{ doc.filename }}</h1>
<StatusBadge :state="doc.lifecycleState" />
</div>
<div class="workspace-header-meta">
<div v-if="doc.stores?.length" class="workspace-stores">
<RouterLink
v-for="store in doc.stores"
:key="store"
:to="{ name: ROUTES.STORE_DETAIL, params: { store } }"
class="store-chip"
:title="store"
>
{{ store }}
</RouterLink>
</div>
<span v-if="doc.fileSize" class="meta-item">{{ formatSize(doc.fileSize) }}</span>
<span class="meta-item">{{ formatRelativeTime(doc.lifecycleStateAt ?? doc.createdAt) }}</span>
</div>
</header>
</template>
<script setup lang="ts">
import { RouterLink } from 'vue-router'
import type { Document } from '../../../shared/types'
import { formatSize, formatRelativeTime } from '../../../shared/format'
import { ROUTES } from '../../../shared/routing/names'
import StatusBadge from './StatusBadge.vue'
defineProps<{
doc: Document
}>()
</script>
<style scoped>
.workspace-header {
position: sticky;
top: 0;
z-index: 10;
background: var(--bg-surface);
border-bottom: 1px solid var(--border);
padding: 12px 20px;
display: flex;
flex-direction: column;
gap: 6px;
}
.workspace-header-main {
display: flex;
align-items: center;
gap: 10px;
min-width: 0;
}
.doc-icon {
width: 16px;
height: 16px;
color: var(--accent);
flex-shrink: 0;
}
.workspace-title {
font-size: 15px;
font-weight: 600;
color: var(--text);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
flex: 1;
min-width: 0;
}
.workspace-header-meta {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
}
.workspace-stores {
display: flex;
gap: 4px;
flex-wrap: wrap;
}
.store-chip {
display: inline-flex;
align-items: center;
padding: 1px 8px;
font-size: 11px;
font-family: 'IBM Plex Mono', monospace;
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: 100px;
color: var(--text-secondary);
text-decoration: none;
transition: all var(--transition);
white-space: nowrap;
overflow: hidden;
max-width: 120px;
text-overflow: ellipsis;
}
.store-chip:hover {
border-color: var(--accent);
color: var(--accent);
}
.meta-item {
font-size: 11px;
color: var(--text-muted);
font-family: 'IBM Plex Mono', monospace;
}
</style>

View file

@ -0,0 +1,91 @@
<template>
<span
class="status-badge"
:class="[`status-badge--${state}`, { 'status-badge--compact': compact }]"
:title="tooltip"
:aria-label="label"
>
<span aria-hidden="true" class="status-symbol">{{ symbol }}</span>
<span v-if="!compact" class="status-text">{{ label }}</span>
</span>
</template>
<script setup lang="ts">
import { computed } from 'vue'
import { useI18n } from '../../../shared/i18n'
import type { DocumentLifecycleState } from '../../../shared/types'
const props = withDefaults(
defineProps<{
state: DocumentLifecycleState
compact?: boolean
}>(),
{ compact: false },
)
const { t } = useI18n()
const SYMBOLS: Record<DocumentLifecycleState, string> = {
Uploaded: '○',
Parsed: '◐',
Chunked: '◑',
Ingested: '●',
Stale: '⚠',
Failed: '✗',
}
const symbol = computed(() => SYMBOLS[props.state] ?? '?')
const label = computed(() => t(`status.${props.state}`))
const tooltip = computed(() => t(`status.tooltip.${props.state}`))
</script>
<style scoped>
.status-badge {
display: inline-flex;
align-items: center;
gap: 5px;
font-size: 12px;
font-weight: 500;
padding: 2px 8px;
border-radius: 999px;
white-space: nowrap;
cursor: default;
}
.status-badge--Uploaded {
background: rgba(99, 102, 241, 0.12);
color: #818cf8;
}
.status-badge--Parsed {
background: rgba(59, 130, 246, 0.12);
color: var(--info);
}
.status-badge--Chunked {
background: rgba(249, 115, 22, 0.12);
color: var(--accent);
}
.status-badge--Ingested {
background: rgba(34, 197, 94, 0.12);
color: var(--success);
}
.status-badge--Stale {
background: rgba(234, 179, 8, 0.12);
color: var(--warning);
}
.status-badge--Failed {
background: rgba(239, 68, 68, 0.12);
color: var(--error);
}
.status-badge--compact {
padding: 0;
background: transparent;
gap: 0;
}
.status-symbol {
font-size: 1em;
line-height: 1;
}
</style>

View file

@ -155,4 +155,81 @@ describe('useFeatureFlagStore', () => {
expect(store.isEnabled('chunking')).toBe(false)
expect(store.isEnabled('disclaimer')).toBe(false)
})
// 0.6.1 — Surface master flags (#257).
it('exposes studioMode / ragPipeline master flags from /api/health', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
studioModeEnabled: true,
ragPipelineEnabled: false,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.isEnabled('studioMode')).toBe(true)
expect(store.isEnabled('ragPipeline')).toBe(false)
})
it('defaults studio off, rag pipeline on when /api/health omits the fields', async () => {
mockApiFetch.mockResolvedValue({ status: 'ok', engine: 'local' })
const store = useFeatureFlagStore()
await store.load()
expect(store.isEnabled('studioMode')).toBe(false)
expect(store.isEnabled('ragPipeline')).toBe(true)
})
it('sub-flags require ragPipeline enabled', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
ragPipelineEnabled: false,
inspectModeEnabled: true,
linkedModeEnabled: true,
askModeEnabled: true,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.isEnabled('inspectMode')).toBe(false)
expect(store.isEnabled('linkedMode')).toBe(false)
expect(store.isEnabled('askMode')).toBe(false)
})
// 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257).
it('exposes inspectMode / linkedMode / askMode flags from /api/health', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
ragPipelineEnabled: true,
inspectModeEnabled: false,
linkedModeEnabled: true,
askModeEnabled: true,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.isEnabled('inspectMode')).toBe(false)
expect(store.isEnabled('linkedMode')).toBe(true)
expect(store.isEnabled('askMode')).toBe(true)
})
it('falls back to all-modes-enabled when /api/health omits the new fields', async () => {
mockApiFetch.mockResolvedValue({ status: 'ok', engine: 'local' })
const store = useFeatureFlagStore()
await store.load()
expect(store.isEnabled('inspectMode')).toBe(true)
expect(store.isEnabled('linkedMode')).toBe(true)
expect(store.isEnabled('askMode')).toBe(true)
})
it('modeFlags() returns the three flags in a Record<DocMode, boolean>', async () => {
mockApiFetch.mockResolvedValue({
status: 'ok',
engine: 'local',
inspectModeEnabled: true,
linkedModeEnabled: false,
askModeEnabled: true,
})
const store = useFeatureFlagStore()
await store.load()
expect(store.modeFlags()).toEqual({ ask: true, chunks: false, inspect: true })
})
})

View file

@ -15,9 +15,27 @@ interface HealthResponse {
maxFileSizeMb?: number
ingestionAvailable?: boolean
reasoningAvailable?: boolean
// 0.6.1 — Surface master flags (#257). Optional for backward compat:
// studio defaults to false (production target), rag pipeline to true.
studioModeEnabled?: boolean
ragPipelineEnabled?: boolean
// 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257). Optional so an
// older backend image without these fields keeps working: missing → true.
inspectModeEnabled?: boolean
linkedModeEnabled?: boolean
askModeEnabled?: boolean
}
export type FeatureFlag = 'chunking' | 'disclaimer' | 'ingestion' | 'reasoning'
export type FeatureFlag =
| 'chunking'
| 'disclaimer'
| 'ingestion'
| 'reasoning'
| 'studioMode'
| 'ragPipeline'
| 'inspectMode'
| 'linkedMode'
| 'askMode'
interface FeatureFlagDef {
description: string
@ -29,6 +47,11 @@ interface FeatureFlagContext {
deploymentMode: DeploymentMode | null
ingestionAvailable: boolean
reasoningAvailable: boolean
studioModeEnabled: boolean
ragPipelineEnabled: boolean
inspectModeEnabled: boolean
linkedModeEnabled: boolean
askModeEnabled: boolean
}
const featureRegistry: Record<FeatureFlag, FeatureFlagDef> = {
@ -52,6 +75,32 @@ const featureRegistry: Record<FeatureFlag, FeatureFlagDef> = {
description: 'Reasoning trace tunnel (docling-agent ReasoningResult viewer)',
isEnabled: (ctx) => ctx.reasoningAvailable,
},
// 0.6.1 — Surface master flags (#257). Select which UI surface(s) are
// exposed; at least one must be on (validated server-side too).
studioMode: {
description: 'Legacy Studio surface (OCR debug, original 0.5.x UI)',
isEnabled: (ctx) => ctx.studioModeEnabled,
},
ragPipeline: {
description: 'New doc-centric RAG ingestion + visualization pipeline',
isEnabled: (ctx) => ctx.ragPipelineEnabled,
},
// 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257). Each gates a
// mode inside the doc workspace and triggers a router-level redirect
// when a disabled mode is requested via deep link. Effective only when
// ragPipeline is enabled.
inspectMode: {
description: 'Doc workspace Inspect mode (tree + bbox debug view)',
isEnabled: (ctx) => ctx.ragPipelineEnabled && ctx.inspectModeEnabled,
},
linkedMode: {
description: 'Doc workspace Linked mode (preview + aligned chunks panel)',
isEnabled: (ctx) => ctx.ragPipelineEnabled && ctx.linkedModeEnabled,
},
askMode: {
description: 'Doc workspace Ask mode (agentic reasoning over the doc)',
isEnabled: (ctx) => ctx.ragPipelineEnabled && ctx.askModeEnabled,
},
}
export const useFeatureFlagStore = defineStore('feature-flags', () => {
@ -61,6 +110,15 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
const maxFileSizeMb = ref<number>(0)
const ingestionAvailable = ref(false)
const reasoningAvailable = ref(false)
// 0.6.1 — Surface master flags (#257). Defaults match production target:
// legacy Studio off, new RAG pipeline on.
const studioModeEnabled = ref(false)
const ragPipelineEnabled = ref(true)
// 0.6.0 — RAG-pipeline sub-flags (#210, renamed in #257). Default true
// so a backend without these fields behaves like the legacy one.
const inspectModeEnabled = ref(true)
const linkedModeEnabled = ref(true)
const askModeEnabled = ref(true)
const appVersion = ref<string>(__APP_VERSION__)
const loaded = ref(false)
const error = ref<string | null>(null)
@ -70,6 +128,11 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
deploymentMode: deploymentMode.value,
ingestionAvailable: ingestionAvailable.value,
reasoningAvailable: reasoningAvailable.value,
studioModeEnabled: studioModeEnabled.value,
ragPipelineEnabled: ragPipelineEnabled.value,
inspectModeEnabled: inspectModeEnabled.value,
linkedModeEnabled: linkedModeEnabled.value,
askModeEnabled: askModeEnabled.value,
}))
function isEnabled(flag: FeatureFlag): boolean {
@ -87,6 +150,15 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
maxFileSizeMb.value = data.maxFileSizeMb ?? 0
ingestionAvailable.value = data.ingestionAvailable ?? false
reasoningAvailable.value = data.reasoningAvailable ?? false
// 0.6.1 — surface flags. Backward compat: missing studio → false
// (production target), missing rag pipeline → true (legacy behaviour).
studioModeEnabled.value = data.studioModeEnabled ?? false
ragPipelineEnabled.value = data.ragPipelineEnabled ?? true
// Sub-flags: fall back to true so an older backend keeps every mode
// visible.
inspectModeEnabled.value = data.inspectModeEnabled ?? true
linkedModeEnabled.value = data.linkedModeEnabled ?? true
askModeEnabled.value = data.askModeEnabled ?? true
appMaxFileSizeMb.value = maxFileSizeMb.value
appMaxPageCount.value = maxPageCount.value
if (data.version) appVersion.value = data.version
@ -98,6 +170,21 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
}
}
/**
* Convenience accessor for `resolveMode` returns the three doc
* workspace mode flags as a `Record<DocMode, boolean>` so the routing
* guard does not need to know about the FeatureFlag union.
*/
function modeFlags(): { ask: boolean; inspect: boolean; chunks: boolean } {
// Mode key 'chunks' is kept until T3 (Linked view) renames the DocMode
// union and route segment. The flag is already linkedModeEnabled.
return {
ask: askModeEnabled.value,
inspect: inspectModeEnabled.value,
chunks: linkedModeEnabled.value,
}
}
return {
engine,
deploymentMode,
@ -105,10 +192,16 @@ export const useFeatureFlagStore = defineStore('feature-flags', () => {
maxFileSizeMb,
ingestionAvailable,
reasoningAvailable,
studioModeEnabled,
ragPipelineEnabled,
inspectModeEnabled,
linkedModeEnabled,
askModeEnabled,
appVersion,
loaded,
error,
isEnabled,
modeFlags,
load,
}
})

View file

@ -0,0 +1,359 @@
<template>
<div class="ask-runner" data-e2e="ask-runner">
<!-- Form -->
<form class="ask-form" @submit.prevent="run">
<label class="ask-label">{{ t('ask.questionLabel') }}</label>
<textarea
v-model="query"
class="ask-textarea"
:placeholder="t('ask.questionPlaceholder')"
:disabled="running"
rows="3"
/>
<details class="ask-model-details">
<summary class="ask-model-summary">{{ t('ask.modelConfig') }}</summary>
<input
v-model="modelId"
class="ask-model-input"
:placeholder="t('ask.modelPlaceholder')"
:disabled="running"
/>
<p class="ask-model-hint">{{ t('ask.modelHint') }}</p>
</details>
<button class="ask-submit" type="submit" :disabled="!query.trim() || running">
<span v-if="running" class="ask-spinner" />
<span>{{ running ? t('ask.running') : t('ask.run') }}</span>
</button>
</form>
<!-- Error -->
<div v-if="error" class="ask-error" data-e2e="ask-error">{{ error }}</div>
<!-- Result -->
<div v-if="result" class="ask-result" data-e2e="ask-result">
<!-- Answer -->
<div class="ask-answer">
<div class="ask-answer-header">
<span class="ask-answer-label">{{ t('ask.answerLabel') }}</span>
<span class="ask-converged" :class="{ yes: result.converged, no: !result.converged }">
{{ result.converged ? t('reasoning.converged') : t('reasoning.notConverged') }}
</span>
</div>
<!-- eslint-disable-next-line vue/no-v-html -- sanitized by DOMPurify -->
<div class="ask-answer-body" v-html="renderedAnswer" />
</div>
<!-- Iterations -->
<div class="ask-iterations">
<h4 class="ask-section-title">{{ t('reasoning.iterationsTitle') }}</h4>
<IterationCard
v-for="it in resolvedIterations"
:key="it.iteration"
:iteration="it"
:active="activeIteration === it.iteration"
@focus="onIterationFocus"
/>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, computed } from 'vue'
import DOMPurify from 'dompurify'
import { marked } from 'marked'
import { runReasoning } from '../api'
import type { ReasoningResult, ReasoningIteration, ResolvedIteration } from '../types'
import IterationCard from './IterationCard.vue'
import { useI18n } from '../../../shared/i18n'
const props = defineProps<{ docId: string }>()
const emit = defineEmits<{ sectionFocus: [sectionRef: string] }>()
const { t } = useI18n()
const query = ref('')
const modelId = ref('')
const running = ref(false)
const error = ref<string | null>(null)
const result = ref<ReasoningResult | null>(null)
const activeIteration = ref<number | null>(null)
const renderedAnswer = computed(() => {
const raw = result.value?.answer ?? ''
if (!raw.trim()) return ''
return DOMPurify.sanitize(marked.parse(raw, { async: false }) as string)
})
function toResolved(it: ReasoningIteration): ResolvedIteration {
return {
iteration: it.iteration,
sectionRef: it.section_ref,
nodeId: it.section_ref,
present: true,
reason: it.reason,
canAnswer: it.can_answer,
response: it.response,
sectionTextLength: it.section_text_length,
}
}
const resolvedIterations = computed<ResolvedIteration[]>(() =>
(result.value?.iterations ?? []).map(toResolved),
)
async function run(): Promise<void> {
if (!query.value.trim()) return
running.value = true
error.value = null
result.value = null
activeIteration.value = null
try {
result.value = await runReasoning(props.docId, query.value, modelId.value || undefined)
} catch (e) {
error.value = (e as Error).message || t('reasoning.runErrUnknown')
} finally {
running.value = false
}
}
function onIterationFocus(n: number): void {
activeIteration.value = n
const it = resolvedIterations.value.find((r) => r.iteration === n)
if (it?.sectionRef) emit('sectionFocus', it.sectionRef)
}
</script>
<style scoped>
.ask-runner {
display: flex;
flex-direction: column;
gap: 16px;
padding: 16px;
overflow-y: auto;
height: 100%;
}
.ask-form {
display: flex;
flex-direction: column;
gap: 8px;
}
.ask-label {
font-size: 11px;
font-weight: 600;
text-transform: uppercase;
letter-spacing: 0.5px;
color: var(--text-muted);
}
.ask-textarea {
font-family: inherit;
font-size: 13px;
color: var(--text);
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 10px 12px;
resize: vertical;
transition: border-color var(--transition);
}
.ask-textarea:focus {
outline: none;
border-color: var(--accent);
}
.ask-textarea:disabled {
opacity: 0.6;
}
.ask-model-details {
font-size: 12px;
}
.ask-model-summary {
cursor: pointer;
color: var(--text-secondary);
padding: 2px 0;
user-select: none;
}
.ask-model-input {
width: 100%;
margin-top: 6px;
font-family: 'IBM Plex Mono', monospace;
font-size: 12px;
color: var(--text);
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
padding: 6px 10px;
box-sizing: border-box;
}
.ask-model-input:focus {
outline: none;
border-color: var(--accent);
}
.ask-model-hint {
margin: 4px 0 0;
font-size: 11px;
color: var(--text-muted);
}
.ask-submit {
display: flex;
align-items: center;
justify-content: center;
gap: 8px;
padding: 8px 16px;
font-size: 13px;
font-weight: 500;
background: var(--accent);
color: #fff;
border: none;
border-radius: var(--radius-sm);
cursor: pointer;
transition: opacity var(--transition);
}
.ask-submit:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.ask-submit:hover:not(:disabled) {
opacity: 0.9;
}
.ask-spinner {
width: 14px;
height: 14px;
border: 2px solid rgba(255, 255, 255, 0.4);
border-top-color: #fff;
border-radius: 50%;
animation: spin 0.6s linear infinite;
flex-shrink: 0;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
.ask-error {
padding: 10px 12px;
background: rgba(239, 68, 68, 0.1);
border: 1px solid var(--error);
border-radius: var(--radius-sm);
color: var(--error);
font-size: 12px;
}
.ask-result {
display: flex;
flex-direction: column;
gap: 16px;
}
.ask-answer {
background: var(--bg);
border: 1px solid #ea580c;
border-radius: var(--radius);
padding: 14px 16px;
display: flex;
flex-direction: column;
gap: 10px;
box-shadow: 0 1px 3px rgba(234, 88, 12, 0.08);
}
.ask-answer-header {
display: flex;
justify-content: space-between;
align-items: center;
gap: 8px;
}
.ask-answer-label {
font-size: 10px;
font-weight: 700;
letter-spacing: 0.8px;
text-transform: uppercase;
color: #ea580c;
}
.ask-converged {
font-size: 10px;
font-weight: 600;
padding: 2px 8px;
border-radius: 10px;
text-transform: uppercase;
letter-spacing: 0.5px;
}
.ask-converged.yes {
background: rgba(22, 163, 74, 0.15);
color: #15803d;
}
.ask-converged.no {
background: rgba(234, 179, 8, 0.15);
color: #a16207;
}
.ask-answer-body {
font-size: 13.5px;
line-height: 1.6;
color: var(--text);
}
.ask-answer-body :deep(p) {
margin: 0 0 8px;
}
.ask-answer-body :deep(p:last-child) {
margin-bottom: 0;
}
.ask-answer-body :deep(ol),
.ask-answer-body :deep(ul) {
margin: 4px 0 8px;
padding-left: 22px;
}
.ask-answer-body :deep(li) {
margin: 2px 0;
}
.ask-answer-body :deep(strong) {
font-weight: 600;
}
.ask-answer-body :deep(code) {
font-family: 'IBM Plex Mono', monospace;
font-size: 12px;
background: var(--border-light);
padding: 1px 5px;
border-radius: 3px;
}
.ask-section-title {
margin: 0 0 8px;
font-size: 11px;
text-transform: uppercase;
letter-spacing: 0.5px;
color: var(--text-muted);
font-weight: 600;
}
.ask-iterations {
display: flex;
flex-direction: column;
gap: 6px;
}
</style>

View file

@ -108,7 +108,7 @@ const props = defineProps<{
/**
* The live Cytoscape instance from the GraphView. May be `null` while the
* graph is loading or if Maintain hasn't been run for this document.
* Passed down from StudioPage via `graphViewRef.cy`.
* Passed down by the host page via `graphViewRef.cy`.
*/
cy: Core | null
}>()

View file

@ -0,0 +1,129 @@
import { describe, it, expect, vi, beforeEach } from 'vitest'
import {
createStore,
deleteStore,
fetchStore,
fetchStores,
fetchStoreDocuments,
queryStore,
removeDocumentFromStore,
updateStore,
} from './api'
vi.mock('../../shared/api/http', () => ({
apiFetch: vi.fn(),
}))
import { apiFetch } from '../../shared/api/http'
describe('store API', () => {
beforeEach(() => {
vi.clearAllMocks()
})
it('fetchStores calls GET /api/stores', async () => {
const stores = [
{
name: 'my-store',
slug: 'my-store',
type: 'opensearch',
embedder: 'bge-m3',
isDefault: true,
connected: true,
documentCount: 0,
chunkCount: 0,
},
]
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue(stores)
const result = await fetchStores()
expect(apiFetch).toHaveBeenCalledWith('/api/stores')
expect(result).toEqual(stores)
})
it('fetchStore calls GET /api/stores/:slug', async () => {
const store = { id: 's-1', slug: 'rh', name: 'rh', kind: 'opensearch' }
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue(store)
const result = await fetchStore('rh')
expect(apiFetch).toHaveBeenCalledWith('/api/stores/rh')
expect(result).toEqual(store)
})
it('createStore POSTs the payload', async () => {
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue({ slug: 'rh' })
await createStore({
name: 'rh',
slug: 'rh',
kind: 'opensearch',
embedder: 'bge-m3',
config: { indexName: 'rh' },
})
expect(apiFetch).toHaveBeenCalledWith('/api/stores', {
method: 'POST',
body: JSON.stringify({
name: 'rh',
slug: 'rh',
kind: 'opensearch',
embedder: 'bge-m3',
config: { indexName: 'rh' },
}),
})
})
it('updateStore PATCHes the payload', async () => {
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue({ slug: 'rh' })
await updateStore('rh', { embedder: 'bge-large' })
expect(apiFetch).toHaveBeenCalledWith('/api/stores/rh', {
method: 'PATCH',
body: JSON.stringify({ embedder: 'bge-large' }),
})
})
it('deleteStore calls DELETE /api/stores/:slug', async () => {
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue(undefined)
await deleteStore('rh')
expect(apiFetch).toHaveBeenCalledWith('/api/stores/rh', { method: 'DELETE' })
})
it('fetchStoreDocuments calls GET /api/stores/:store/documents', async () => {
const docs = [{ docId: 'doc-1', filename: 'test.pdf', state: 'Ingested', chunkCount: 12 }]
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue(docs)
const result = await fetchStoreDocuments('my-store')
expect(apiFetch).toHaveBeenCalledWith('/api/stores/my-store/documents')
expect(result).toEqual(docs)
})
it('removeDocumentFromStore calls DELETE /api/stores/:store/documents/:docId', async () => {
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue(undefined)
await removeDocumentFromStore('my-store', 'doc-1')
expect(apiFetch).toHaveBeenCalledWith('/api/stores/my-store/documents/doc-1', {
method: 'DELETE',
})
})
it('queryStore calls POST /api/stores/:store/query with body', async () => {
const results = [{ chunkId: 'c1', docId: 'd1', filename: 'a.pdf', text: 'hi', score: 0.9 }]
;(apiFetch as ReturnType<typeof vi.fn>).mockResolvedValue(results)
const result = await queryStore('my-store', 'what is X?', 3)
expect(apiFetch).toHaveBeenCalledWith('/api/stores/my-store/query', {
method: 'POST',
body: JSON.stringify({ query: 'what is X?', top_k: 3 }),
})
expect(result).toEqual(results)
})
})

View file

@ -0,0 +1,104 @@
import { apiFetch } from '../../shared/api/http'
import type { DocumentLifecycleState } from '../../shared/types'
export interface StoreInfo {
name: string
slug: string
type: string
embedder: string
isDefault: boolean
connected: boolean
documentCount: number
chunkCount: number
errorMessage?: string
}
export interface StoreDetail {
id: string
name: string
slug: string
kind: string
embedder: string
isDefault: boolean
config: Record<string, unknown>
createdAt: string
}
export interface StoreCreatePayload {
name: string
slug: string
kind: string
embedder: string
config: Record<string, unknown>
isDefault?: boolean
}
export interface StoreUpdatePayload {
name?: string
slug?: string
kind?: string
embedder?: string
config?: Record<string, unknown>
isDefault?: boolean
}
export interface StoreDocEntry {
docId: string
filename: string
state: DocumentLifecycleState
chunkCount: number
pushedAt: string
}
export interface QueryResult {
chunkId: string
docId: string
filename: string
text: string
score: number
pageRange?: [number, number]
}
export function fetchStores(): Promise<StoreInfo[]> {
return apiFetch<StoreInfo[]>('/api/stores')
}
export function fetchStore(slug: string): Promise<StoreDetail> {
return apiFetch<StoreDetail>(`/api/stores/${encodeURIComponent(slug)}`)
}
export function createStore(payload: StoreCreatePayload): Promise<StoreDetail> {
return apiFetch<StoreDetail>('/api/stores', {
method: 'POST',
body: JSON.stringify(payload),
})
}
export function updateStore(slug: string, payload: StoreUpdatePayload): Promise<StoreDetail> {
return apiFetch<StoreDetail>(`/api/stores/${encodeURIComponent(slug)}`, {
method: 'PATCH',
body: JSON.stringify(payload),
})
}
export function deleteStore(slug: string): Promise<void> {
return apiFetch<void>(`/api/stores/${encodeURIComponent(slug)}`, { method: 'DELETE' })
}
export function fetchStoreDocuments(store: string): Promise<StoreDocEntry[]> {
return apiFetch<StoreDocEntry[]>(`/api/stores/${encodeURIComponent(store)}/documents`)
}
export function removeDocumentFromStore(store: string, docId: string): Promise<void> {
return apiFetch<void>(
`/api/stores/${encodeURIComponent(store)}/documents/${encodeURIComponent(docId)}`,
{ method: 'DELETE' },
)
}
export function queryStore(store: string, query: string, topK = 5): Promise<QueryResult[]> {
return apiFetch<QueryResult[]>(`/api/stores/${encodeURIComponent(store)}/query`, {
method: 'POST',
body: JSON.stringify({ query, top_k: topK }),
})
}

View file

@ -0,0 +1 @@
export * from './api'

View file

@ -0,0 +1,109 @@
<template>
<div class="config-form">
<div class="field">
<label class="field-label" for="store-neo4j-index">{{
t('storeForm.neo4j.indexName')
}}</label>
<input
id="store-neo4j-index"
v-model="indexName"
class="field-input"
type="text"
placeholder="chunks-vec"
:aria-invalid="showError && !indexName.trim()"
@input="emitChange"
/>
<p v-if="showError && !indexName.trim()" class="field-error">
{{ t('storeForm.required') }}
</p>
<p v-else class="field-help">{{ t('storeForm.neo4j.indexNameHelp') }}</p>
</div>
<div class="field">
<label class="field-label" for="store-neo4j-db">{{ t('storeForm.neo4j.database') }}</label>
<input
id="store-neo4j-db"
v-model="database"
class="field-input"
type="text"
placeholder="neo4j"
@input="emitChange"
/>
<p class="field-help">{{ t('storeForm.neo4j.databaseHelp') }}</p>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, watch } from 'vue'
import { useI18n } from '../../../shared/i18n'
const props = defineProps<{
modelValue: Record<string, unknown>
showError?: boolean
}>()
const emit = defineEmits<{
(e: 'update:modelValue', value: Record<string, unknown>): void
(e: 'valid', value: boolean): void
}>()
const { t } = useI18n()
const indexName = ref(String(props.modelValue.indexName ?? props.modelValue.index_name ?? ''))
const database = ref(String(props.modelValue.database ?? ''))
function emitChange() {
emit('update:modelValue', {
indexName: indexName.value,
database: database.value || undefined,
})
emit('valid', indexName.value.trim().length > 0)
}
watch(
() => props.modelValue,
(next) => {
const incomingIndex = String(next.indexName ?? next.index_name ?? '')
if (incomingIndex !== indexName.value) indexName.value = incomingIndex
const incomingDb = String(next.database ?? '')
if (incomingDb !== database.value) database.value = incomingDb
},
)
emit('valid', indexName.value.trim().length > 0)
</script>
<style scoped>
.config-form {
display: flex;
flex-direction: column;
gap: 1rem;
}
.field {
display: flex;
flex-direction: column;
gap: 0.25rem;
}
.field-label {
font-weight: 500;
font-size: 0.875rem;
}
.field-input {
padding: 0.5rem 0.75rem;
border: 1px solid var(--color-border, #d1d5db);
border-radius: 0.375rem;
font-size: 0.875rem;
}
.field-input[aria-invalid='true'] {
border-color: #dc2626;
}
.field-help {
font-size: 0.75rem;
color: var(--color-text-muted, #6b7280);
}
.field-error {
font-size: 0.75rem;
color: #dc2626;
}
</style>

View file

@ -0,0 +1,92 @@
<template>
<div class="config-form">
<div class="field">
<label class="field-label" for="store-os-index">{{
t('storeForm.opensearch.indexName')
}}</label>
<input
id="store-os-index"
v-model="indexName"
class="field-input"
type="text"
:placeholder="'rh-corpus-v3'"
:aria-invalid="showError && !indexName.trim()"
@input="emitChange"
/>
<p v-if="showError && !indexName.trim()" class="field-error">
{{ t('storeForm.required') }}
</p>
<p v-else class="field-help">{{ t('storeForm.opensearch.indexNameHelp') }}</p>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, watch } from 'vue'
import { useI18n } from '../../../shared/i18n'
const props = defineProps<{
modelValue: Record<string, unknown>
showError?: boolean
}>()
const emit = defineEmits<{
(e: 'update:modelValue', value: Record<string, unknown>): void
(e: 'valid', value: boolean): void
}>()
const { t } = useI18n()
const indexName = ref(String(props.modelValue.indexName ?? props.modelValue.index_name ?? ''))
function emitChange() {
emit('update:modelValue', { indexName: indexName.value })
emit('valid', indexName.value.trim().length > 0)
}
watch(
() => props.modelValue,
(next) => {
const incoming = String(next.indexName ?? next.index_name ?? '')
if (incoming !== indexName.value) {
indexName.value = incoming
}
},
)
emit('valid', indexName.value.trim().length > 0)
</script>
<style scoped>
.config-form {
display: flex;
flex-direction: column;
gap: 1rem;
}
.field {
display: flex;
flex-direction: column;
gap: 0.25rem;
}
.field-label {
font-weight: 500;
font-size: 0.875rem;
}
.field-input {
padding: 0.5rem 0.75rem;
border: 1px solid var(--color-border, #d1d5db);
border-radius: 0.375rem;
font-size: 0.875rem;
}
.field-input[aria-invalid='true'] {
border-color: #dc2626;
}
.field-help {
font-size: 0.75rem;
color: var(--color-text-muted, #6b7280);
}
.field-error {
font-size: 0.75rem;
color: #dc2626;
}
</style>

View file

@ -0,0 +1,47 @@
<template>
<component
:is="component"
v-model="config"
:show-error="showError"
@valid="(v: boolean) => emit('valid', v)"
/>
</template>
<script setup lang="ts">
import { computed, ref, watch } from 'vue'
import OpenSearchConfigForm from './OpenSearchConfigForm.vue'
import Neo4jConfigForm from './Neo4jConfigForm.vue'
const props = defineProps<{
kind: string
modelValue: Record<string, unknown>
showError?: boolean
}>()
const emit = defineEmits<{
(e: 'update:modelValue', value: Record<string, unknown>): void
(e: 'valid', value: boolean): void
}>()
const config = ref({ ...props.modelValue })
watch(config, (next) => emit('update:modelValue', next), { deep: true })
watch(
() => props.modelValue,
(next) => {
config.value = { ...next }
},
{ deep: true },
)
const component = computed(() => {
switch (props.kind) {
case 'opensearch':
return OpenSearchConfigForm
case 'neo4j':
return Neo4jConfigForm
default:
return OpenSearchConfigForm
}
})
</script>

View file

@ -0,0 +1,264 @@
<template>
<form class="store-form" @submit.prevent="onSubmit">
<div class="field">
<label class="field-label" for="store-name">{{ t('storeForm.fieldName') }}</label>
<input
id="store-name"
v-model="form.name"
class="field-input"
type="text"
autocomplete="off"
:aria-invalid="showError && !form.name.trim()"
/>
<p v-if="showError && !form.name.trim()" class="field-error">
{{ t('storeForm.required') }}
</p>
<p v-else class="field-help">{{ t('storeForm.fieldNameHelp') }}</p>
</div>
<div class="field">
<label class="field-label" for="store-slug">{{ t('storeForm.fieldSlug') }}</label>
<input
id="store-slug"
v-model="form.slug"
class="field-input"
type="text"
autocomplete="off"
:disabled="lockSlug"
:aria-invalid="showError && !slugIsValid"
/>
<p v-if="showError && !form.slug.trim()" class="field-error">
{{ t('storeForm.required') }}
</p>
<p v-else-if="showError && !slugIsValid" class="field-error">
{{ t('storeForm.invalidSlug') }}
</p>
<p v-else class="field-help">{{ t('storeForm.fieldSlugHelp') }}</p>
</div>
<div class="field">
<label class="field-label" for="store-kind">{{ t('storeForm.fieldKind') }}</label>
<select id="store-kind" v-model="form.kind" class="field-input">
<option v-for="opt in kindOptions" :key="opt.value" :value="opt.value">
{{ opt.label }}
</option>
</select>
</div>
<div class="field">
<label class="field-label" for="store-embedder">{{ t('storeForm.fieldEmbedder') }}</label>
<input
id="store-embedder"
v-model="form.embedder"
class="field-input"
type="text"
autocomplete="off"
:placeholder="'bge-m3'"
:aria-invalid="showError && !form.embedder.trim()"
/>
<p v-if="showError && !form.embedder.trim()" class="field-error">
{{ t('storeForm.required') }}
</p>
<p v-else class="field-help">{{ t('storeForm.fieldEmbedderHelp') }}</p>
</div>
<div class="field field--checkbox">
<label class="checkbox-label">
<input v-model="form.isDefault" type="checkbox" />
<span>{{ t('storeForm.fieldIsDefault') }}</span>
</label>
<p class="field-help">{{ t('storeForm.fieldIsDefaultHelp') }}</p>
</div>
<fieldset class="config-section">
<legend>{{ t('storeForm.sectionConfig') }}</legend>
<StoreConfigForm
v-model="form.config"
:kind="form.kind"
:show-error="showError"
@valid="onConfigValid"
/>
</fieldset>
<p v-if="errorMessage" class="form-error" role="alert">{{ errorMessage }}</p>
<div class="actions">
<button type="button" class="btn-secondary" :disabled="submitting" @click="emit('cancel')">
{{ t('storeForm.cancel') }}
</button>
<button type="submit" class="btn-primary" :disabled="submitting">
{{ submitLabel }}
</button>
</div>
</form>
</template>
<script setup lang="ts">
import { computed, reactive, ref, watch } from 'vue'
import type { StoreCreatePayload, StoreDetail } from '../api'
import { useI18n } from '../../../shared/i18n'
import StoreConfigForm from './StoreConfigForm.vue'
const SLUG_PATTERN = /^[a-z0-9]([a-z0-9-]*[a-z0-9])?$/
const props = defineProps<{
initialValue?: Partial<StoreDetail>
mode: 'create' | 'edit'
submitting?: boolean
errorMessage?: string | null
/** When true, the slug input is disabled (typically in edit mode for the seeded default). */
lockSlug?: boolean
}>()
const emit = defineEmits<{
(e: 'submit', payload: StoreCreatePayload): void
(e: 'cancel'): void
}>()
const { t } = useI18n()
const kindOptions = [
{ value: 'opensearch', label: 'OpenSearch' },
{ value: 'neo4j', label: 'Neo4j' },
]
const form = reactive<StoreCreatePayload>({
name: props.initialValue?.name ?? '',
slug: props.initialValue?.slug ?? '',
kind: props.initialValue?.kind ?? 'opensearch',
embedder: props.initialValue?.embedder ?? '',
config: { ...(props.initialValue?.config ?? {}) },
isDefault: props.initialValue?.isDefault ?? false,
})
const showError = ref(false)
const configValid = ref(false)
watch(
() => props.initialValue,
(next) => {
if (!next) return
form.name = next.name ?? form.name
form.slug = next.slug ?? form.slug
form.kind = next.kind ?? form.kind
form.embedder = next.embedder ?? form.embedder
form.config = { ...(next.config ?? {}) }
form.isDefault = next.isDefault ?? form.isDefault
},
{ deep: true },
)
const slugIsValid = computed(() => SLUG_PATTERN.test(form.slug.trim()))
const submitLabel = computed(() =>
props.mode === 'create' ? t('storeForm.create') : t('storeForm.save'),
)
function onConfigValid(valid: boolean) {
configValid.value = valid
}
function onSubmit() {
showError.value = true
if (!form.name.trim() || !form.embedder.trim() || !slugIsValid.value || !configValid.value) {
return
}
emit('submit', { ...form, slug: form.slug.trim().toLowerCase() })
}
</script>
<style scoped>
.store-form {
display: flex;
flex-direction: column;
gap: 1.25rem;
max-width: 640px;
}
.field {
display: flex;
flex-direction: column;
gap: 0.25rem;
}
.field--checkbox {
flex-direction: column;
}
.field-label {
font-weight: 500;
font-size: 0.875rem;
}
.field-input {
padding: 0.5rem 0.75rem;
border: 1px solid var(--color-border, #d1d5db);
border-radius: 0.375rem;
font-size: 0.875rem;
}
.field-input[aria-invalid='true'] {
border-color: #dc2626;
}
.field-input:disabled {
background: var(--color-bg-muted, #f3f4f6);
}
.field-help {
font-size: 0.75rem;
color: var(--color-text-muted, #6b7280);
}
.field-error {
font-size: 0.75rem;
color: #dc2626;
}
.checkbox-label {
display: inline-flex;
align-items: center;
gap: 0.5rem;
font-size: 0.875rem;
}
.config-section {
border: 1px solid var(--color-border, #d1d5db);
border-radius: 0.5rem;
padding: 1rem;
}
.config-section legend {
font-size: 0.875rem;
font-weight: 500;
padding: 0 0.5rem;
}
.form-error {
background: #fee2e2;
color: #991b1b;
padding: 0.5rem 0.75rem;
border-radius: 0.375rem;
font-size: 0.875rem;
}
.actions {
display: flex;
justify-content: flex-end;
gap: 0.75rem;
}
.btn-primary {
padding: 0.5rem 1rem;
border-radius: 0.375rem;
background: var(--color-primary, #2563eb);
color: white;
border: none;
font-size: 0.875rem;
font-weight: 500;
cursor: pointer;
}
.btn-primary:disabled {
opacity: 0.6;
cursor: not-allowed;
}
.btn-secondary {
padding: 0.5rem 1rem;
border-radius: 0.375rem;
background: white;
color: var(--color-text, #111827);
border: 1px solid var(--color-border, #d1d5db);
font-size: 0.875rem;
cursor: pointer;
}
.btn-secondary:disabled {
opacity: 0.6;
cursor: not-allowed;
}
</style>

View file

@ -0,0 +1,199 @@
<template>
<div class="ask-tab" data-e2e="ask-tab">
<!-- Loading analysis -->
<div v-if="loading" class="ask-state">
<span class="spinner" />
</div>
<!-- Error -->
<div v-else-if="error" class="ask-state ask-state--error">
<p>{{ error }}</p>
<button class="retry-btn" @click="loadAnalysis">{{ t('inspect.retry') }}</button>
</div>
<!-- No analysis -->
<div v-else-if="pages.length === 0" class="ask-state">
<p class="ask-empty-title">{{ t('ask.noAnalysis') }}</p>
<p class="ask-empty-sub">{{ t('ask.noAnalysisSub') }}</p>
<RouterLink :to="{ name: ROUTES.STUDIO }" class="ask-cta">
{{ t('inspect.goToStudio') }}
</RouterLink>
</div>
<!-- Split layout: document left + ask panel right -->
<template v-else>
<div class="ask-doc-pane">
<StructureViewer
ref="structureViewerRef"
:pages="pages"
:document-id="docId"
:visited-by-self-ref="visitedBySelfRef"
:focused-self-ref="focusedSelfRef"
selectable
/>
</div>
<div class="ask-panel-pane">
<AskRunner :doc-id="docId" @section-focus="onSectionFocus" />
</div>
</template>
</div>
</template>
<script setup lang="ts">
import { ref, computed, watch } from 'vue'
import { RouterLink } from 'vue-router'
import type { Analysis, Page } from '../shared/types'
import { fetchDocumentAnalyses } from '../features/analysis/api'
import StructureViewer from '../features/analysis/ui/StructureViewer.vue'
import AskRunner from '../features/reasoning/ui/AskRunner.vue'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
const props = defineProps<{ docId: string }>()
const { t } = useI18n()
const loading = ref(false)
const error = ref<string | null>(null)
const analysis = ref<Analysis | null>(null)
const focusedSelfRef = ref<string | null>(null)
const structureViewerRef = ref<InstanceType<typeof StructureViewer> | null>(null)
const pages = computed<Page[]>(() => {
if (!analysis.value?.pagesJson) return []
try {
return JSON.parse(analysis.value.pagesJson) as Page[]
} catch {
return []
}
})
const visitedBySelfRef = computed<Map<string, number>>(() => new Map())
async function loadAnalysis(): Promise<void> {
loading.value = true
error.value = null
analysis.value = null
focusedSelfRef.value = null
const requestedId = props.docId
try {
const analyses = await fetchDocumentAnalyses(requestedId)
if (requestedId !== props.docId) return
// Defensive: filter by documentId because the backend currently ignores
// the ?documentId= query param and returns all analyses.
analysis.value =
analyses.find((a) => a.documentId === requestedId && a.status === 'COMPLETED') ?? null
} catch (e) {
if (requestedId !== props.docId) return
error.value = (e as Error).message || 'Failed to load analysis'
} finally {
if (requestedId === props.docId) loading.value = false
}
}
function onSectionFocus(sectionRef: string): void {
if (focusedSelfRef.value === sectionRef) {
structureViewerRef.value?.scrollToFocused(sectionRef)
} else {
focusedSelfRef.value = sectionRef
}
}
watch(() => props.docId, loadAnalysis, { immediate: true })
</script>
<style scoped>
.ask-tab {
display: flex;
height: 100%;
overflow: hidden;
}
.ask-state {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
width: 100%;
gap: 12px;
color: var(--text-muted);
font-size: 13px;
}
.ask-state--error {
color: var(--error);
}
.ask-empty-title {
font-size: 14px;
font-weight: 500;
color: var(--text-secondary);
margin: 0;
}
.ask-empty-sub {
font-size: 12px;
color: var(--text-muted);
margin: 0;
}
.ask-cta {
font-size: 13px;
color: var(--accent);
text-decoration: none;
border: 1px solid var(--accent);
padding: 6px 14px;
border-radius: var(--radius-sm);
transition: all var(--transition);
}
.ask-cta:hover {
background: var(--accent-muted);
}
.retry-btn {
font-size: 13px;
color: var(--text-secondary);
background: none;
border: 1px solid var(--border);
padding: 6px 14px;
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
}
.retry-btn:hover {
border-color: var(--text-secondary);
color: var(--text);
}
.ask-doc-pane {
flex: 1;
min-width: 0;
overflow-y: auto;
border-right: 1px solid var(--border);
padding: 16px 20px;
}
.ask-panel-pane {
width: 360px;
flex-shrink: 0;
overflow-y: auto;
background: var(--bg-surface);
}
.spinner {
width: 28px;
height: 28px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,139 @@
<template>
<div class="chunks-tab" data-e2e="chunks-tab">
<!-- Tree rail -->
<div class="tree-pane" :class="{ 'tree-pane--collapsed': treeCollapsed }">
<button
class="tree-toggle"
:title="treeCollapsed ? t('tree.expand') : t('tree.collapse')"
@click="treeCollapsed = !treeCollapsed"
>
<span class="tree-toggle-icon" :class="{ 'tree-toggle-icon--collapsed': treeCollapsed }"
></span
>
</button>
<DocTreeRail
v-if="!treeCollapsed"
:nodes="treeNodes"
:loading="treeLoading"
:error="treeError"
:selected="selectedNodeRef"
:highlight="highlightRef"
@select="onTreeSelect"
@reload="loadTree"
/>
</div>
<!-- Chunks editor -->
<div class="editor-pane">
<StaleStoresStrip
v-if="storeLinks && storeLinks.length > 0"
:doc-id="docId"
:store-links="storeLinks"
/>
<ChunksEditor
:doc-id="docId"
:available-stores="availableStores"
@node-highlight="highlightRef = $event"
/>
</div>
</div>
</template>
<script setup lang="ts">
import { ref, onMounted } from 'vue'
import type { DocTreeNode, DocStoreLink } from '../shared/types'
import { fetchDocumentTree } from '../features/document/api'
import DocTreeRail from '../features/document/ui/DocTreeRail.vue'
import ChunksEditor from '../features/chunks/ui/ChunksEditor.vue'
import StaleStoresStrip from '../features/chunks/ui/StaleStoresStrip.vue'
import { useI18n } from '../shared/i18n'
const props = defineProps<{
docId: string
availableStores: string[]
storeLinks?: DocStoreLink[]
}>()
const { t } = useI18n()
const treeNodes = ref<DocTreeNode[]>([])
const treeLoading = ref(false)
const treeError = ref<string | null>(null)
const treeCollapsed = ref(false)
const selectedNodeRef = ref<string | null>(null)
const highlightRef = ref<string | null>(null)
async function loadTree(): Promise<void> {
treeLoading.value = true
treeError.value = null
try {
treeNodes.value = await fetchDocumentTree(props.docId)
} catch (e) {
treeError.value = (e as Error).message || 'Failed to load tree'
} finally {
treeLoading.value = false
}
}
function onTreeSelect(ref: string): void {
selectedNodeRef.value = ref
highlightRef.value = ref
}
onMounted(loadTree)
</script>
<style scoped>
.chunks-tab {
display: flex;
height: 100%;
overflow: hidden;
}
.tree-pane {
width: 240px;
flex-shrink: 0;
display: flex;
position: relative;
transition: width 0.2s;
}
.tree-pane--collapsed {
width: 24px;
}
.tree-toggle {
position: absolute;
top: 12px;
right: -1px;
z-index: 1;
width: 18px;
height: 32px;
display: flex;
align-items: center;
justify-content: center;
background: var(--bg-surface);
border: 1px solid var(--border);
border-left: none;
border-radius: 0 var(--radius-sm) var(--radius-sm) 0;
cursor: pointer;
color: var(--text-muted);
font-size: 14px;
}
.tree-toggle-icon {
transition: transform 0.2s;
}
.tree-toggle-icon--collapsed {
transform: rotate(180deg);
}
.editor-pane {
flex: 1;
min-width: 0;
display: flex;
flex-direction: column;
overflow: hidden;
}
</style>

View file

@ -0,0 +1,148 @@
<template>
<div class="inspect-tab" data-e2e="inspect-tab">
<!-- Loading -->
<div v-if="loading" class="inspect-state">
<span class="spinner" />
</div>
<!-- Error -->
<div v-else-if="error" class="inspect-state inspect-state--error">
<p>{{ error }}</p>
<button class="retry-btn" @click="load">{{ t('inspect.retry') }}</button>
</div>
<!-- No analysis yet -->
<div v-else-if="!analysis" class="inspect-state">
<p class="inspect-empty-title">{{ t('inspect.noAnalysis') }}</p>
<p class="inspect-empty-sub">{{ t('inspect.noAnalysisSub') }}</p>
<RouterLink :to="{ name: ROUTES.STUDIO }" class="inspect-cta">
{{ t('inspect.goToStudio') }}
</RouterLink>
</div>
<!-- Result -->
<InspectResultTabs v-else :analysis="analysis" :doc-id="docId" />
</div>
</template>
<script setup lang="ts">
import { ref, watch } from 'vue'
import { RouterLink } from 'vue-router'
import type { Analysis } from '../shared/types'
import { fetchDocumentAnalyses } from '../features/analysis/api'
import InspectResultTabs from '../features/analysis/ui/InspectResultTabs.vue'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
const props = defineProps<{ docId: string }>()
const { t } = useI18n()
const loading = ref(false)
const error = ref<string | null>(null)
const analysis = ref<Analysis | null>(null)
async function load(): Promise<void> {
loading.value = true
error.value = null
analysis.value = null
const requestedId = props.docId
try {
const analyses = await fetchDocumentAnalyses(requestedId)
if (requestedId !== props.docId) return
// Defensive: filter by documentId because the backend currently ignores
// the ?documentId= query param and returns all analyses.
analysis.value =
analyses.find((a) => a.documentId === requestedId && a.status === 'COMPLETED') ?? null
} catch (e) {
if (requestedId !== props.docId) return
error.value = (e as Error).message || 'Failed to load analysis'
} finally {
if (requestedId === props.docId) loading.value = false
}
}
watch(() => props.docId, load, { immediate: true })
</script>
<style scoped>
.inspect-tab {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
}
.inspect-state {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
height: 100%;
gap: 12px;
color: var(--text-muted);
font-size: 13px;
}
.inspect-state--error {
color: var(--error);
}
.inspect-empty-title {
font-size: 14px;
font-weight: 500;
color: var(--text-secondary);
margin: 0;
}
.inspect-empty-sub {
font-size: 12px;
color: var(--text-muted);
margin: 0;
}
.inspect-cta {
font-size: 13px;
color: var(--accent);
text-decoration: none;
border: 1px solid var(--accent);
padding: 6px 14px;
border-radius: var(--radius-sm);
transition: all var(--transition);
}
.inspect-cta:hover {
background: var(--accent-muted);
}
.retry-btn {
font-size: 13px;
color: var(--text-secondary);
background: none;
border: 1px solid var(--border);
padding: 6px 14px;
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
}
.retry-btn:hover {
border-color: var(--text-secondary);
color: var(--text);
}
.spinner {
width: 28px;
height: 28px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,250 @@
<template>
<div class="workspace-page">
<!-- Loading doc -->
<div v-if="loadingDoc" class="workspace-loading">
<span class="spinner" />
</div>
<!-- Doc error -->
<div v-else-if="docError" class="workspace-error">
<p>{{ docError }}</p>
<RouterLink :to="{ name: ROUTES.DOCS_LIBRARY }" class="back-link">
{{ t('workspace.backToLibrary') }}
</RouterLink>
</div>
<template v-else-if="doc">
<!-- Sticky header (#218) -->
<DocWorkspaceHeader :doc="doc" />
<!-- Tab strip (#216) -->
<div class="tab-strip" role="tablist" data-e2e="tab-strip">
<button
v-for="m in ALL_MODES"
:key="m"
class="tab-btn"
:class="{ active: activeMode === m, disabled: !modeEnabled(m) }"
role="tab"
:aria-selected="activeMode === m"
:disabled="!modeEnabled(m)"
:title="!modeEnabled(m) ? t('workspace.modeDisabled') : undefined"
:data-e2e="`tab-${m}`"
@click="switchMode(m)"
>
{{ t(`workspace.tabs.${m}`) }}
</button>
</div>
<!-- Tab content lazy loaded (#216) -->
<!-- :key on docId forces a clean remount when navigating to a different doc,
preventing stale state (bbox, selectedPage, etc.) from leaking. -->
<div class="tab-content" role="tabpanel" data-e2e="tab-content">
<Suspense>
<DocChunksTab
v-if="activeMode === 'chunks'"
:key="id"
:doc-id="id"
:available-stores="doc.stores ?? []"
:store-links="doc.storeLinks"
/>
<DocInspectTab v-else-if="activeMode === 'inspect'" :key="id" :doc-id="id" />
<DocAskTab v-else-if="activeMode === 'ask'" :key="id" :doc-id="id" />
</Suspense>
</div>
</template>
</div>
</template>
<script setup lang="ts">
import { ref, computed, onMounted, watch } from 'vue'
import { RouterLink, useRouter, useRoute } from 'vue-router'
import type { Document } from '../shared/types'
import { fetchDocument } from '../features/document/api'
import { useFeatureFlagStore } from '../features/feature-flags/store'
import { ALL_MODES, type DocMode } from '../shared/routing/modes'
import { resolveMode } from '../shared/routing/resolveMode'
import { useCrumbs } from '../shared/breadcrumb/store'
import { truncate } from '../shared/breadcrumb/text'
import type { Crumb } from '../shared/breadcrumb/types'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
import DocWorkspaceHeader from '../features/document/ui/DocWorkspaceHeader.vue'
import DocChunksTab from './DocChunksTab.vue'
import DocInspectTab from './DocInspectTab.vue'
import DocAskTab from './DocAskTab.vue'
const props = defineProps<{ id: string; mode: DocMode }>()
const router = useRouter()
const route = useRoute()
const { t } = useI18n()
const flagStore = useFeatureFlagStore()
const doc = ref<Document | null>(null)
const loadingDoc = ref(true)
const docError = ref<string | null>(null)
const activeMode = ref<DocMode>(props.mode)
const crumbs = computed<Crumb[]>(() => [
{ kind: 'link', label: t('breadcrumb.studio'), to: { name: ROUTES.HOME } },
{
kind: 'link',
label: doc.value ? truncate(doc.value.filename, 40) : truncate(props.id, 40),
to: { name: ROUTES.DOC_WORKSPACE, params: { id: props.id } },
},
{ kind: 'leaf', label: t(`breadcrumb.mode.${activeMode.value}`) },
])
useCrumbs(crumbs)
function modeEnabled(m: DocMode): boolean {
return flagStore.modeFlags()[m]
}
function switchMode(m: DocMode): void {
if (!modeEnabled(m)) return
activeMode.value = m
router.replace({ query: { ...route.query, mode: m } })
}
async function loadDoc(): Promise<void> {
loadingDoc.value = true
docError.value = null
doc.value = null
const requestedId = props.id
try {
const fetched = await fetchDocument(requestedId)
if (requestedId !== props.id) return
doc.value = fetched
} catch (e) {
if (requestedId !== props.id) return
docError.value = (e as Error).message || 'Failed to load document'
} finally {
if (requestedId === props.id) loadingDoc.value = false
}
}
onMounted(async () => {
await flagStore.load()
const flags = flagStore.modeFlags()
const resolved = resolveMode(props.mode, flags)
if (!resolved) {
router.replace({ name: ROUTES.DOCS_LIBRARY, query: { reason: 'no-mode-enabled' } })
return
}
if (resolved !== props.mode) {
router.replace({ query: { ...route.query, mode: resolved } })
}
activeMode.value = resolved
await loadDoc()
})
watch(
() => props.mode,
(m) => {
const flags = flagStore.modeFlags()
const resolved = resolveMode(m, flags)
if (resolved) activeMode.value = resolved
},
)
watch(
() => props.id,
(newId, oldId) => {
if (newId !== oldId) loadDoc()
},
)
</script>
<style scoped>
.workspace-page {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
}
.workspace-loading,
.workspace-error {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
height: 100%;
gap: 12px;
color: var(--text-muted);
font-size: 13px;
}
.workspace-error {
color: var(--error);
}
.back-link {
font-size: 13px;
color: var(--text-secondary);
text-decoration: none;
}
.back-link:hover {
color: var(--text);
}
.tab-strip {
display: flex;
gap: 0;
border-bottom: 1px solid var(--border);
background: var(--bg-surface);
padding: 0 20px;
flex-shrink: 0;
}
.tab-btn {
padding: 8px 16px;
font-size: 13px;
font-weight: 500;
color: var(--text-muted);
background: none;
border: none;
border-bottom: 2px solid transparent;
cursor: pointer;
transition: all var(--transition);
margin-bottom: -1px;
}
.tab-btn:hover:not(.disabled) {
color: var(--text);
}
.tab-btn.active {
color: var(--accent);
border-bottom-color: var(--accent);
}
.tab-btn.disabled {
opacity: 0.35;
cursor: not-allowed;
}
.tab-content {
flex: 1;
overflow: hidden;
}
.spinner {
width: 28px;
height: 28px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,849 @@
<template>
<div class="library-page">
<!-- Flash: redirected here because all workspace modes are disabled (#210) -->
<div v-if="showFlashAllModesDisabled" class="flash flash--warning" role="alert">
{{ t('flags.allModesDisabled') }}
</div>
<!-- Page header -->
<div class="library-header">
<h1 class="library-title">{{ t('docs.title') }}</h1>
<RouterLink :to="{ name: ROUTES.DOCS_NEW }" class="btn-primary">
{{ t('docs.import') }}
</RouterLink>
</div>
<!-- Filter bar (#212) -->
<div v-if="docStore.documents.length" class="filter-bar">
<!-- Status pills -->
<div class="filter-group">
<button
v-for="state in ALL_STATES"
:key="state"
class="filter-pill"
:class="{ active: statusFilter.has(state) }"
@click="toggleStatus(state)"
>
<StatusBadge :state="state" compact />
{{ t(`status.${state}`) }}
</button>
</div>
<!-- Store pills (shown only when at least one doc has stores) -->
<div v-if="allStores.length" class="filter-group">
<button
v-for="store in allStores"
:key="store"
class="filter-pill filter-pill--store"
:class="{ active: storeFilter.has(store) }"
@click="toggleStore(store)"
>
{{ store }}
</button>
</div>
<!-- Search -->
<input
v-model="searchInput"
type="search"
class="filter-search"
:placeholder="t('docs.filterSearch')"
/>
<!-- Clear -->
<button v-if="hasActiveFilters" class="filter-clear" @click="clearFilters">
{{ t('docs.filterClear') }}
</button>
</div>
<!-- Loading skeleton -->
<div v-if="docStore.loading" class="loading-state">
<div class="spinner" />
</div>
<!-- Table (#211) -->
<div v-else-if="docStore.documents.length" class="table-wrapper">
<table class="doc-table" data-e2e="docs-table">
<thead>
<tr>
<th class="col-check">
<input
type="checkbox"
:checked="allSelected"
:indeterminate="someSelected"
:aria-label="t('docs.selected', { n: filteredDocs.length })"
@change="toggleAll"
/>
</th>
<th>{{ t('docs.colName') }}</th>
<th>{{ t('docs.colStatus') }}</th>
<th>{{ t('docs.colStores') }}</th>
<th class="col-updated">{{ t('docs.colUpdated') }}</th>
</tr>
</thead>
<tbody>
<tr
v-for="doc in filteredDocs"
:key="doc.id"
class="doc-row"
data-e2e="doc-row"
@click="openDoc(doc.id)"
>
<td class="col-check" @click.stop>
<input
type="checkbox"
:checked="selectedIds.has(doc.id)"
@change="toggleDoc(doc.id)"
/>
</td>
<td class="col-name">
<svg class="doc-icon" viewBox="0 0 20 20" fill="currentColor">
<path
fill-rule="evenodd"
d="M4 4a2 2 0 012-2h4.586A2 2 0 0112 2.586L15.414 6A2 2 0 0116 7.414V16a2 2 0 01-2 2H6a2 2 0 01-2-2V4z"
clip-rule="evenodd"
/>
</svg>
<span class="doc-filename" :title="doc.filename">{{ doc.filename }}</span>
</td>
<td class="col-status">
<StatusBadge :state="doc.lifecycleState" />
</td>
<td class="col-stores">
<span v-if="doc.stores?.length" class="store-chips">
<span v-for="s in doc.stores" :key="s" class="store-chip">{{ s }}</span>
</span>
<span v-else class="no-value"></span>
</td>
<td class="col-updated">
<span class="updated-time">{{ formatUpdated(doc) }}</span>
</td>
</tr>
</tbody>
</table>
<!-- Filtered empty state -->
<div v-if="!filteredDocs.length" class="empty-state empty-state--filtered">
<p class="empty-title">{{ t('docs.emptyFiltered') }}</p>
<button class="btn-secondary" @click="clearFilters">{{ t('docs.filterClear') }}</button>
</div>
</div>
<!-- Empty corpus state -->
<div v-else-if="!docStore.loading" class="empty-state" data-e2e="docs-empty">
<svg
class="empty-icon"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="1"
>
<path
d="M7 21h10a2 2 0 002-2V9.414a1 1 0 00-.293-.707l-5.414-5.414A1 1 0 0012.586 3H7a2 2 0 00-2 2v14a2 2 0 002 2z"
/>
</svg>
<p class="empty-title">{{ t('docs.emptyTitle') }}</p>
<p class="empty-subtitle">{{ t('docs.emptySubtitle') }}</p>
<RouterLink :to="{ name: ROUTES.DOCS_NEW }" class="btn-primary">
{{ t('docs.emptyAction') }}
</RouterLink>
</div>
<!-- Sticky bulk action bar (#213) -->
<div v-if="selectedIds.size > 0" class="bulk-bar" data-e2e="bulk-bar">
<span class="bulk-count">{{ t('docs.selected', { n: selectedIds.size }) }}</span>
<div class="bulk-actions">
<button class="btn-sm" @click="bulkRechunk">{{ t('docs.bulkRechunk') }}</button>
<button class="btn-sm" @click="openPushModal">{{ t('docs.bulkPush') }}</button>
<button class="btn-sm btn-sm--danger" @click="bulkDelete">
{{ t('docs.bulkDelete') }}
</button>
<button class="btn-sm btn-sm--ghost" @click="clearSelection">
{{ t('docs.bulkCancel') }}
</button>
</div>
</div>
<!-- Push to store modal -->
<div
v-if="showPushModal"
class="modal-overlay"
role="dialog"
:aria-label="t('docs.pushTitle')"
@click.self="showPushModal = false"
>
<div class="modal">
<h3 class="modal-title">{{ t('docs.pushTitle') }}</h3>
<label class="modal-label" for="push-store-input">{{ t('docs.pushLabel') }}</label>
<input
id="push-store-input"
v-model="pushStoreName"
class="modal-input"
list="store-datalist"
:placeholder="t('docs.pushPlaceholder')"
@keyup.enter="confirmPush"
@keyup.escape="showPushModal = false"
/>
<datalist id="store-datalist">
<option v-for="s in availableStores" :key="s" :value="s" />
</datalist>
<div class="modal-footer">
<button class="btn-primary" :disabled="!pushStoreName.trim()" @click="confirmPush">
{{ t('docs.pushSubmit') }}
</button>
<button class="btn-secondary" @click="showPushModal = false">
{{ t('docs.pushCancel') }}
</button>
</div>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { computed, onMounted, ref, watch } from 'vue'
import { RouterLink, useRoute, useRouter } from 'vue-router'
import { useDocumentStore } from '../features/document/store'
import StatusBadge from '../features/document/ui/StatusBadge.vue'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
import type { Document, DocumentLifecycleState } from '../shared/types'
import { formatRelativeTime } from '../shared/format'
import { appLocale } from '../shared/appConfig'
const ALL_STATES: DocumentLifecycleState[] = [
'Uploaded',
'Parsed',
'Chunked',
'Ingested',
'Stale',
'Failed',
]
const docStore = useDocumentStore()
const route = useRoute()
const router = useRouter()
const { t } = useI18n()
// ---------------------------------------------------------------------------
// Flash for "no-mode-enabled" redirect (#210)
// ---------------------------------------------------------------------------
const showFlashAllModesDisabled = computed(() => route.query.reason === 'no-mode-enabled')
// ---------------------------------------------------------------------------
// Filters init from URL query params (#212)
// ---------------------------------------------------------------------------
function parseSetParam(raw: unknown): Set<string> {
const str = typeof raw === 'string' ? raw : ''
return new Set(str.split(',').filter(Boolean))
}
const statusFilter = ref<Set<DocumentLifecycleState>>(
parseSetParam(route.query.status) as Set<DocumentLifecycleState>,
)
const storeFilter = ref<Set<string>>(parseSetParam(route.query.store))
const searchInput = ref<string>(typeof route.query.q === 'string' ? route.query.q : '')
const debouncedSearch = ref<string>(searchInput.value)
let searchTimer: ReturnType<typeof setTimeout> | null = null
watch(searchInput, (val) => {
if (searchTimer) clearTimeout(searchTimer)
searchTimer = setTimeout(() => {
debouncedSearch.value = val
syncUrl()
}, 300)
})
function syncUrl(): void {
const query: Record<string, string> = {}
if (statusFilter.value.size) query.status = [...statusFilter.value].join(',')
if (storeFilter.value.size) query.store = [...storeFilter.value].join(',')
if (debouncedSearch.value) query.q = debouncedSearch.value
router.replace({ query })
}
function toggleStatus(state: DocumentLifecycleState): void {
const next = new Set(statusFilter.value)
if (next.has(state)) next.delete(state)
else next.add(state)
statusFilter.value = next
syncUrl()
}
function toggleStore(store: string): void {
const next = new Set(storeFilter.value)
if (next.has(store)) next.delete(store)
else next.add(store)
storeFilter.value = next
syncUrl()
}
const hasActiveFilters = computed(
() => statusFilter.value.size > 0 || storeFilter.value.size > 0 || !!debouncedSearch.value,
)
function clearFilters(): void {
statusFilter.value = new Set()
storeFilter.value = new Set()
searchInput.value = ''
debouncedSearch.value = ''
router.replace({ query: {} })
}
// ---------------------------------------------------------------------------
// Derived store list for store-filter chips and push modal datalist
// ---------------------------------------------------------------------------
const allStores = computed(() => {
const acc = new Set<string>()
docStore.documents.forEach((d) => d.stores?.forEach((s) => acc.add(s)))
return [...acc].sort()
})
const availableStores = allStores
// ---------------------------------------------------------------------------
// Filtered documents
// ---------------------------------------------------------------------------
const filteredDocs = computed(() => {
const q = debouncedSearch.value.toLowerCase()
return docStore.documents.filter((doc) => {
if (statusFilter.value.size && !statusFilter.value.has(doc.lifecycleState)) return false
if (storeFilter.value.size && !doc.stores?.some((s) => storeFilter.value.has(s))) return false
if (q && !doc.filename.toLowerCase().includes(q)) return false
return true
})
})
// ---------------------------------------------------------------------------
// Table helpers
// ---------------------------------------------------------------------------
function formatUpdated(doc: Document): string {
return formatRelativeTime(doc.lifecycleStateAt ?? doc.createdAt, appLocale.value)
}
function openDoc(id: string): void {
router.push({ name: ROUTES.DOC_WORKSPACE, params: { id } })
}
// ---------------------------------------------------------------------------
// Multi-select (#213)
// ---------------------------------------------------------------------------
const selectedIds = ref<Set<string>>(new Set())
const allSelected = computed(
() =>
filteredDocs.value.length > 0 && filteredDocs.value.every((d) => selectedIds.value.has(d.id)),
)
const someSelected = computed(
() => filteredDocs.value.some((d) => selectedIds.value.has(d.id)) && !allSelected.value,
)
function toggleDoc(id: string): void {
const next = new Set(selectedIds.value)
if (next.has(id)) next.delete(id)
else next.add(id)
selectedIds.value = next
}
function toggleAll(): void {
if (allSelected.value) {
const next = new Set(selectedIds.value)
filteredDocs.value.forEach((d) => next.delete(d.id))
selectedIds.value = next
} else {
const next = new Set(selectedIds.value)
filteredDocs.value.forEach((d) => next.add(d.id))
selectedIds.value = next
}
}
function clearSelection(): void {
selectedIds.value = new Set()
}
// ---------------------------------------------------------------------------
// Bulk actions
// ---------------------------------------------------------------------------
async function bulkRechunk(): Promise<void> {
const ids = [...selectedIds.value]
clearSelection()
const counts = await Promise.all(ids.map((id) => docStore.rechunk(id)))
const succeeded = counts.filter((n): n is number => n !== null).length
if (succeeded) {
window.alert(t('docs.rechunkDone', { n: succeeded }))
}
}
const showPushModal = ref(false)
const pushStoreName = ref('')
function openPushModal(): void {
pushStoreName.value = availableStores.value[0] ?? ''
showPushModal.value = true
}
async function confirmPush(): Promise<void> {
const target = pushStoreName.value.trim()
if (!target) return
showPushModal.value = false
const ids = [...selectedIds.value]
clearSelection()
const jobs = await Promise.all(ids.map((id) => docStore.pushToStore(id, target)))
const dispatched = jobs.filter(Boolean)
if (dispatched.length) {
window.alert(t('docs.jobDispatched', { jobId: dispatched.join(', ') }))
}
}
async function bulkDelete(): Promise<void> {
const n = selectedIds.value.size
if (!window.confirm(t('docs.deleteConfirm', { n }))) return
const ids = [...selectedIds.value]
clearSelection()
await Promise.all(ids.map((id) => docStore.remove(id)))
}
// ---------------------------------------------------------------------------
// Lifecycle
// ---------------------------------------------------------------------------
onMounted(() => {
docStore.load()
})
</script>
<style scoped>
.library-page {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
position: relative;
}
/* Header */
.library-header {
display: flex;
align-items: center;
justify-content: space-between;
padding: 20px 24px 16px;
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.library-title {
font-size: 20px;
font-weight: 600;
color: var(--text);
}
/* Flash */
.flash {
margin: 12px 24px 0;
padding: 10px 14px;
border-radius: var(--radius-sm);
font-size: 13px;
flex-shrink: 0;
}
.flash--warning {
color: #92400e;
background: #fef3c7;
border: 1px solid #fde68a;
}
/* Filter bar */
.filter-bar {
display: flex;
align-items: center;
flex-wrap: wrap;
gap: 8px;
padding: 12px 24px;
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.filter-group {
display: flex;
align-items: center;
gap: 4px;
flex-wrap: wrap;
}
.filter-pill {
display: inline-flex;
align-items: center;
gap: 5px;
padding: 4px 10px;
border-radius: 999px;
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
background: var(--bg-elevated);
border: 1px solid var(--border);
cursor: pointer;
transition: all var(--transition);
white-space: nowrap;
}
.filter-pill:hover {
border-color: var(--accent);
color: var(--text);
}
.filter-pill.active {
background: var(--accent-muted);
border-color: var(--accent);
color: var(--accent);
}
.filter-search {
flex: 1;
min-width: 180px;
max-width: 300px;
padding: 6px 10px;
border-radius: var(--radius-sm);
border: 1px solid var(--border);
background: var(--bg-elevated);
color: var(--text);
font-size: 13px;
outline: none;
transition: border-color var(--transition);
}
.filter-search:focus {
border-color: var(--accent);
}
.filter-clear {
padding: 4px 10px;
font-size: 12px;
color: var(--text-muted);
background: none;
border: none;
cursor: pointer;
transition: color var(--transition);
}
.filter-clear:hover {
color: var(--text);
}
/* Loading */
.loading-state {
display: flex;
justify-content: center;
padding: 60px;
}
.spinner {
width: 28px;
height: 28px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
/* Table */
.table-wrapper {
flex: 1;
overflow-y: auto;
padding: 0 24px 80px; /* 80px bottom pad for bulk bar */
}
.doc-table {
width: 100%;
border-collapse: collapse;
font-size: 13px;
}
.doc-table thead {
position: sticky;
top: 0;
background: var(--bg);
z-index: 1;
}
.doc-table th {
padding: 10px 12px;
text-align: left;
font-size: 11px;
font-weight: 600;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.05em;
border-bottom: 1px solid var(--border);
}
.col-check {
width: 40px;
}
.col-updated {
width: 130px;
white-space: nowrap;
}
.doc-row {
cursor: pointer;
transition: background var(--transition);
border-bottom: 1px solid var(--border);
}
.doc-row:hover {
background: var(--bg-hover);
}
.doc-table td {
padding: 12px 12px;
vertical-align: middle;
}
.col-name {
display: flex;
align-items: center;
gap: 8px;
min-width: 0;
}
.doc-icon {
width: 14px;
height: 14px;
color: var(--accent);
flex-shrink: 0;
}
.doc-filename {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-weight: 500;
color: var(--text);
}
.store-chips {
display: flex;
flex-wrap: wrap;
gap: 4px;
}
.store-chip {
padding: 2px 8px;
border-radius: 999px;
font-size: 11px;
font-family: 'IBM Plex Mono', monospace;
background: var(--bg-elevated);
border: 1px solid var(--border-light);
color: var(--text-secondary);
white-space: nowrap;
}
.no-value {
color: var(--text-muted);
}
.updated-time {
color: var(--text-muted);
font-size: 12px;
font-family: 'IBM Plex Mono', monospace;
}
/* Empty states */
.empty-state {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 12px;
padding: 80px 24px;
text-align: center;
}
.empty-icon {
width: 48px;
height: 48px;
color: var(--text-muted);
}
.empty-title {
font-size: 16px;
font-weight: 600;
color: var(--text);
}
.empty-subtitle {
font-size: 13px;
color: var(--text-secondary);
}
.empty-state--filtered {
padding: 40px 24px;
}
/* Bulk action bar */
.bulk-bar {
position: absolute;
bottom: 0;
left: 0;
right: 0;
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 24px;
background: var(--bg-elevated);
border-top: 1px solid var(--border);
gap: 12px;
}
.bulk-count {
font-size: 13px;
font-weight: 600;
color: var(--text);
white-space: nowrap;
}
.bulk-actions {
display: flex;
gap: 8px;
}
/* Buttons */
.btn-primary {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 7px 14px;
font-size: 13px;
font-weight: 500;
color: white;
background: var(--accent);
border: none;
border-radius: var(--radius-sm);
cursor: pointer;
text-decoration: none;
transition: background var(--transition);
}
.btn-primary:hover {
background: var(--accent-hover);
}
.btn-primary:disabled {
opacity: 0.4;
cursor: not-allowed;
}
.btn-secondary {
padding: 7px 14px;
font-size: 13px;
font-weight: 500;
color: var(--text-secondary);
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
}
.btn-secondary:hover {
background: var(--bg-hover);
color: var(--text);
}
.btn-sm {
padding: 5px 12px;
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
background: var(--bg-surface);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
white-space: nowrap;
}
.btn-sm:hover {
background: var(--bg-hover);
color: var(--text);
}
.btn-sm--danger {
color: var(--error);
border-color: rgba(239, 68, 68, 0.3);
}
.btn-sm--danger:hover {
background: rgba(239, 68, 68, 0.1);
}
.btn-sm--ghost {
border-color: transparent;
background: transparent;
}
/* Push modal */
.modal-overlay {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.6);
display: flex;
align-items: center;
justify-content: center;
z-index: 100;
}
.modal {
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-lg);
padding: 24px;
width: 360px;
display: flex;
flex-direction: column;
gap: 12px;
}
.modal-title {
font-size: 15px;
font-weight: 600;
color: var(--text);
}
.modal-label {
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
}
.modal-input {
padding: 8px 10px;
border-radius: var(--radius-sm);
border: 1px solid var(--border);
background: var(--bg-surface);
color: var(--text);
font-size: 13px;
outline: none;
width: 100%;
transition: border-color var(--transition);
}
.modal-input:focus {
border-color: var(--accent);
}
.modal-footer {
display: flex;
gap: 8px;
justify-content: flex-end;
margin-top: 4px;
}
</style>

View file

@ -0,0 +1,391 @@
<template>
<div class="docs-new-page">
<!-- Header -->
<div class="page-header">
<RouterLink :to="{ name: ROUTES.DOCS_LIBRARY }" class="back-link">
<svg viewBox="0 0 20 20" fill="currentColor" class="back-icon">
<path
fill-rule="evenodd"
d="M9.707 16.707a1 1 0 01-1.414 0l-6-6a1 1 0 010-1.414l6-6a1 1 0 011.414 1.414L5.414 9H17a1 1 0 110 2H5.414l4.293 4.293a1 1 0 010 1.414z"
clip-rule="evenodd"
/>
</svg>
{{ t('docsNew.backToLibrary') }}
</RouterLink>
<h1 class="page-title">{{ t('docsNew.title') }}</h1>
</div>
<!-- Drop zone -->
<div
class="drop-zone"
:class="{ dragging, disabled: isUploading }"
data-e2e="drop-zone"
@dragover.prevent="onDragOver"
@dragleave.prevent="dragging = false"
@drop.prevent="onDrop"
@click="openPicker"
>
<input ref="fileInput" type="file" multiple accept=".pdf" hidden @change="onFileSelect" />
<svg
class="drop-icon"
viewBox="0 0 24 24"
fill="none"
stroke="currentColor"
stroke-width="1.5"
>
<path d="M12 16V4m0 0L8 8m4-4l4 4M4 17v2a1 1 0 001 1h14a1 1 0 001-1v-2" />
</svg>
<p class="drop-text">{{ t('docsNew.drop') }}</p>
<p class="drop-hint">{{ t('docsNew.dropHint') }}</p>
</div>
<!-- Per-file upload list -->
<ul v-if="uploads.length" class="upload-list" data-e2e="upload-list">
<li
v-for="(item, idx) in uploads"
:key="idx"
class="upload-item"
:class="`upload-item--${item.status}`"
data-e2e="upload-item"
>
<svg class="upload-item-icon" viewBox="0 0 20 20" fill="currentColor">
<path
fill-rule="evenodd"
d="M4 4a2 2 0 012-2h4.586A2 2 0 0112 2.586L15.414 6A2 2 0 0116 7.414V16a2 2 0 01-2 2H6a2 2 0 01-2-2V4z"
clip-rule="evenodd"
/>
</svg>
<span class="upload-item-name" :title="item.file.name">{{ item.file.name }}</span>
<span class="upload-item-status">{{ statusLabel(item.status) }}</span>
<span v-if="item.status === 'uploading'" class="upload-spinner" />
<RouterLink
v-if="item.status === 'done' && item.docId"
:to="{ name: ROUTES.DOC_WORKSPACE, params: { id: item.docId } }"
class="upload-item-link"
>
{{ t('docsNew.viewDoc') }}
</RouterLink>
<span v-if="item.status === 'failed'" class="upload-item-error">{{ item.error }}</span>
</li>
</ul>
<!-- All done state -->
<div v-if="allDone && uploads.length" class="done-state" data-e2e="done-state">
<p class="done-msg">{{ t('docsNew.allDone') }}</p>
<RouterLink :to="{ name: ROUTES.DOCS_LIBRARY }" class="btn-primary">
{{ t('docsNew.viewLibrary') }}
</RouterLink>
</div>
</div>
</template>
<script setup lang="ts">
import { computed, ref } from 'vue'
import { RouterLink } from 'vue-router'
import { uploadDocument } from '../features/document/api'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
import { appMaxFileSizeMb } from '../shared/appConfig'
type UploadStatus = 'queued' | 'uploading' | 'done' | 'failed'
interface UploadItem {
file: File
status: UploadStatus
docId?: string
error?: string
}
const { t } = useI18n()
const fileInput = ref<HTMLInputElement | null>(null)
const dragging = ref(false)
const uploads = ref<UploadItem[]>([])
const isUploading = computed(() => uploads.value.some((u) => u.status === 'uploading'))
const allDone = computed(
() =>
uploads.value.length > 0 &&
uploads.value.every((u) => u.status === 'done' || u.status === 'failed'),
)
function statusLabel(status: UploadStatus): string {
return t(`docsNew.${status}`)
}
function isPdf(file: File): boolean {
return file.type === 'application/pdf' || file.name.toLowerCase().endsWith('.pdf')
}
function openPicker(): void {
if (isUploading.value) return
fileInput.value?.click()
}
function onDragOver(): void {
if (!isUploading.value) dragging.value = true
}
function onFileSelect(e: Event): void {
dragging.value = false
const target = e.target as HTMLInputElement
const files = target.files ? [...target.files] : []
target.value = ''
enqueue(files)
}
function onDrop(e: DragEvent): void {
dragging.value = false
const files = e.dataTransfer?.files ? [...e.dataTransfer.files] : []
enqueue(files)
}
function enqueue(files: File[]): void {
const pdfs = files.filter(isPdf)
if (!pdfs.length) return
const newItems: UploadItem[] = pdfs.map((f) => ({ file: f, status: 'queued' }))
uploads.value = [...uploads.value, ...newItems]
// Upload sequentially to avoid overloading the backend
processQueue()
}
async function processQueue(): Promise<void> {
for (const item of uploads.value) {
if (item.status !== 'queued') continue
await uploadOne(item)
}
}
async function uploadOne(item: UploadItem): Promise<void> {
const maxMb = appMaxFileSizeMb.value
if (maxMb > 0 && item.file.size > maxMb * 1024 * 1024) {
item.status = 'failed'
item.error = t('upload.tooLarge').replace('{n}', String(maxMb))
return
}
item.status = 'uploading'
try {
const doc = await uploadDocument(item.file)
item.status = 'done'
item.docId = doc.id
} catch (e) {
item.status = 'failed'
item.error = (e as Error).message || 'Upload failed'
}
}
</script>
<style scoped>
.docs-new-page {
display: flex;
flex-direction: column;
gap: 24px;
padding: 24px;
overflow-y: auto;
height: 100%;
}
/* Header */
.page-header {
display: flex;
flex-direction: column;
gap: 8px;
}
.back-link {
display: inline-flex;
align-items: center;
gap: 6px;
font-size: 13px;
color: var(--text-secondary);
text-decoration: none;
transition: color var(--transition);
}
.back-link:hover {
color: var(--text);
}
.back-icon {
width: 14px;
height: 14px;
}
.page-title {
font-size: 20px;
font-weight: 600;
color: var(--text);
}
/* Drop zone */
.drop-zone {
border: 2px dashed var(--border-light);
border-radius: var(--radius);
padding: 48px 24px;
display: flex;
flex-direction: column;
align-items: center;
gap: 10px;
cursor: pointer;
transition: all var(--transition);
text-align: center;
}
.drop-zone:hover,
.drop-zone.dragging {
border-color: var(--accent);
background: var(--accent-muted);
}
.drop-zone.disabled {
pointer-events: none;
opacity: 0.5;
}
.drop-icon {
width: 40px;
height: 40px;
color: var(--text-muted);
}
.drop-text {
font-size: 15px;
font-weight: 500;
color: var(--text-secondary);
}
.drop-hint {
font-size: 12px;
color: var(--text-muted);
}
/* Upload list */
.upload-list {
list-style: none;
display: flex;
flex-direction: column;
gap: 4px;
}
.upload-item {
display: flex;
align-items: center;
gap: 10px;
padding: 10px 14px;
border-radius: var(--radius-sm);
background: var(--bg-elevated);
border: 1px solid var(--border);
font-size: 13px;
}
.upload-item-icon {
width: 14px;
height: 14px;
color: var(--accent);
flex-shrink: 0;
}
.upload-item-name {
flex: 1;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-weight: 500;
color: var(--text);
}
.upload-item-status {
font-size: 12px;
color: var(--text-muted);
font-family: 'IBM Plex Mono', monospace;
flex-shrink: 0;
}
.upload-item--done .upload-item-status {
color: var(--success);
}
.upload-item--failed .upload-item-status {
color: var(--error);
}
.upload-spinner {
width: 14px;
height: 14px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
flex-shrink: 0;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
.upload-item-link {
font-size: 12px;
color: var(--accent);
text-decoration: none;
flex-shrink: 0;
}
.upload-item-link:hover {
text-decoration: underline;
}
.upload-item-error {
font-size: 11px;
color: var(--error);
flex-shrink: 0;
max-width: 200px;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
/* Done state */
.done-state {
display: flex;
flex-direction: column;
align-items: center;
gap: 12px;
padding: 24px;
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius);
text-align: center;
}
.done-msg {
font-size: 14px;
color: var(--success);
font-weight: 500;
}
/* Shared button */
.btn-primary {
display: inline-flex;
align-items: center;
padding: 7px 16px;
font-size: 13px;
font-weight: 500;
color: white;
background: var(--accent);
border: none;
border-radius: var(--radius-sm);
cursor: pointer;
text-decoration: none;
transition: background var(--transition);
}
.btn-primary:hover {
background: var(--accent-hover);
}
</style>

View file

@ -0,0 +1,20 @@
<template>
<ComingSoonShell
:title="t('comingSoon.title')"
:subtitle="t('comingSoon.subtitle.runDetail')"
:hint="hint"
/>
</template>
<script setup lang="ts">
import { computed } from 'vue'
import { useI18n } from '../shared/i18n'
import ComingSoonShell from '../shared/ui/ComingSoonShell.vue'
const props = defineProps<{ id: string }>()
const { t } = useI18n()
const hint = computed(() => t('comingSoon.hint.runDetail', { id: props.id }))
</script>

View file

@ -0,0 +1,11 @@
<template>
<ComingSoonShell :title="t('comingSoon.title')" :subtitle="t('comingSoon.subtitle.runs')" />
</template>
<script setup lang="ts">
import { useI18n } from '../shared/i18n'
import ComingSoonShell from '../shared/ui/ComingSoonShell.vue'
const { t } = useI18n()
</script>

View file

@ -0,0 +1,59 @@
<template>
<div class="store-edit-page">
<header class="header">
<h1 class="title">{{ t('storeForm.titleCreate') }}</h1>
</header>
<StoreForm
mode="create"
:submitting="submitting"
:error-message="errorMessage"
@submit="onSubmit"
@cancel="onCancel"
/>
</div>
</template>
<script setup lang="ts">
import { ref } from 'vue'
import { useRouter } from 'vue-router'
import StoreForm from '../features/store/ui/StoreForm.vue'
import { createStore, type StoreCreatePayload } from '../features/store/api'
import { ROUTES } from '../shared/routing/names'
import { useI18n } from '../shared/i18n'
const { t } = useI18n()
const router = useRouter()
const submitting = ref(false)
const errorMessage = ref<string | null>(null)
async function onSubmit(payload: StoreCreatePayload) {
submitting.value = true
errorMessage.value = null
try {
await createStore(payload)
router.push({ name: ROUTES.STORES_LIST })
} catch (err) {
errorMessage.value = err instanceof Error ? err.message : String(err)
} finally {
submitting.value = false
}
}
function onCancel() {
router.push({ name: ROUTES.STORES_LIST })
}
</script>
<style scoped>
.store-edit-page {
padding: 1.5rem 2rem;
}
.header {
margin-bottom: 1.5rem;
}
.title {
font-size: 1.5rem;
font-weight: 600;
margin: 0;
}
</style>

View file

@ -0,0 +1,512 @@
<template>
<div class="detail-page">
<div class="detail-header">
<RouterLink :to="{ name: ROUTES.STORES_LIST }" class="back-link">
{{ t('storeDetail.back') }}
</RouterLink>
<h1 class="detail-title">{{ store }}</h1>
<div class="header-actions">
<RouterLink :to="{ name: ROUTES.STORE_EDIT, params: { store } }" class="btn-secondary">
{{ t('stores.edit') }}
</RouterLink>
<button
class="btn-secondary btn-danger"
:disabled="store === 'default'"
:title="store === 'default' ? t('stores.deleteDefaultBlocked') : ''"
@click="onDeleteStore"
>
{{ t('stores.delete') }}
</button>
<RouterLink :to="{ name: ROUTES.STORE_QUERY, params: { store } }" class="btn-primary">
{{ t('storeDetail.query') }}
</RouterLink>
</div>
</div>
<div v-if="loading" class="loading-state">
<div class="spinner" />
</div>
<div v-else-if="error" class="error-state">
<p class="error-text">{{ error }}</p>
<button class="btn-secondary" @click="load">Retry</button>
</div>
<div v-else-if="docs.length" class="table-wrapper">
<table class="detail-table">
<thead>
<tr>
<th class="col-check">
<input
type="checkbox"
:checked="allSelected"
:indeterminate="someSelected"
@change="toggleAll"
/>
</th>
<th>{{ t('storeDetail.colDoc') }}</th>
<th>{{ t('storeDetail.colState') }}</th>
<th class="col-num">{{ t('storeDetail.colChunks') }}</th>
<th>{{ t('storeDetail.colIngested') }}</th>
<th class="col-action" />
</tr>
</thead>
<tbody>
<tr v-for="doc in docs" :key="doc.docId" class="doc-row" @click="openDoc(doc.docId)">
<td class="col-check" @click.stop>
<input
type="checkbox"
:checked="selectedIds.has(doc.docId)"
@change="toggleDoc(doc.docId)"
/>
</td>
<td class="col-name">
<svg class="doc-icon" viewBox="0 0 20 20" fill="currentColor">
<path
fill-rule="evenodd"
d="M4 4a2 2 0 012-2h4.586A2 2 0 0112 2.586L15.414 6A2 2 0 0116 7.414V16a2 2 0 01-2 2H6a2 2 0 01-2-2V4z"
clip-rule="evenodd"
/>
</svg>
<span class="doc-filename" :title="doc.filename">{{ doc.filename }}</span>
</td>
<td>
<StatusBadge :state="doc.state" />
</td>
<td class="col-num">{{ doc.chunkCount }}</td>
<td class="col-date">{{ doc.pushedAt ? formatDate(doc.pushedAt) : '—' }}</td>
<td class="col-action" @click.stop>
<button class="btn-sm btn-sm--danger" @click="removeDoc(doc)">
{{ t('storeDetail.remove') }}
</button>
</td>
</tr>
</tbody>
</table>
</div>
<div v-else class="empty-state">
<p class="empty-title">{{ t('storeDetail.empty') }}</p>
</div>
<!-- Bulk action bar -->
<div v-if="selectedIds.size > 0" class="bulk-bar">
<span class="bulk-count">{{ t('storeDetail.selected', { n: selectedIds.size }) }}</span>
<div class="bulk-actions">
<button class="btn-sm btn-sm--danger" @click="bulkRemove">
{{ t('storeDetail.bulkRemove') }}
</button>
<button class="btn-sm btn-sm--ghost" @click="clearSelection">
{{ t('storeDetail.bulkCancel') }}
</button>
</div>
</div>
</div>
</template>
<script setup lang="ts">
import { computed, onMounted, ref } from 'vue'
import { RouterLink, useRouter } from 'vue-router'
import {
deleteStore,
fetchStoreDocuments,
removeDocumentFromStore,
type StoreDocEntry,
} from '../features/store/api'
import StatusBadge from '../features/document/ui/StatusBadge.vue'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
import { appLocale } from '../shared/appConfig'
const props = defineProps<{ store: string }>()
const router = useRouter()
const { t } = useI18n()
const docs = ref<StoreDocEntry[]>([])
const loading = ref(false)
const error = ref<string | null>(null)
const selectedIds = ref<Set<string>>(new Set())
async function load(): Promise<void> {
loading.value = true
error.value = null
try {
docs.value = await fetchStoreDocuments(props.store)
} catch (e) {
error.value = e instanceof Error ? e.message : String(e)
} finally {
loading.value = false
}
}
function formatDate(iso: string): string {
return new Date(iso).toLocaleDateString(appLocale.value === 'fr' ? 'fr-FR' : 'en-US', {
year: 'numeric',
month: 'short',
day: 'numeric',
})
}
function openDoc(docId: string): void {
router.push({ name: ROUTES.DOC_WORKSPACE, params: { id: docId } })
}
async function removeDoc(doc: StoreDocEntry): Promise<void> {
if (!window.confirm(t('storeDetail.removeConfirm', { doc: doc.filename }))) return
await removeDocumentFromStore(props.store, doc.docId)
docs.value = docs.value.filter((d) => d.docId !== doc.docId)
const next = new Set(selectedIds.value)
next.delete(doc.docId)
selectedIds.value = next
}
async function onDeleteStore(): Promise<void> {
const message = t('stores.deleteConfirm').replace('{name}', props.store)
if (!window.confirm(message)) return
try {
await deleteStore(props.store)
router.push({ name: ROUTES.STORES_LIST })
} catch (e) {
error.value = e instanceof Error ? e.message : String(e)
}
}
const allSelected = computed(
() => docs.value.length > 0 && docs.value.every((d) => selectedIds.value.has(d.docId)),
)
const someSelected = computed(
() => docs.value.some((d) => selectedIds.value.has(d.docId)) && !allSelected.value,
)
function toggleDoc(id: string): void {
const next = new Set(selectedIds.value)
if (next.has(id)) next.delete(id)
else next.add(id)
selectedIds.value = next
}
function toggleAll(): void {
if (allSelected.value) {
selectedIds.value = new Set()
} else {
selectedIds.value = new Set(docs.value.map((d) => d.docId))
}
}
function clearSelection(): void {
selectedIds.value = new Set()
}
async function bulkRemove(): Promise<void> {
const n = selectedIds.value.size
if (!window.confirm(t('storeDetail.bulkConfirm', { n }))) return
const ids = [...selectedIds.value]
clearSelection()
await Promise.all(ids.map((id) => removeDocumentFromStore(props.store, id)))
docs.value = docs.value.filter((d) => !ids.includes(d.docId))
}
onMounted(load)
</script>
<style scoped>
.detail-page {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
position: relative;
}
.detail-header {
display: flex;
align-items: center;
gap: 12px;
padding: 20px 24px 16px;
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.back-link {
font-size: 13px;
color: var(--text-muted);
text-decoration: none;
transition: color var(--transition);
white-space: nowrap;
}
.back-link:hover {
color: var(--text);
}
.detail-title {
flex: 1;
font-size: 20px;
font-weight: 600;
font-family: 'IBM Plex Mono', monospace;
color: var(--text);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.loading-state {
display: flex;
justify-content: center;
padding: 60px;
}
.spinner {
width: 28px;
height: 28px;
border: 2px solid var(--border-light);
border-top-color: var(--accent);
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
.error-state {
display: flex;
flex-direction: column;
align-items: center;
gap: 12px;
padding: 60px 24px;
}
.error-text {
font-size: 13px;
color: var(--error);
}
.table-wrapper {
flex: 1;
overflow-y: auto;
padding: 0 24px 80px;
}
.detail-table {
width: 100%;
border-collapse: collapse;
font-size: 13px;
}
.detail-table thead {
position: sticky;
top: 0;
background: var(--bg);
z-index: 1;
}
.detail-table th {
padding: 10px 12px;
text-align: left;
font-size: 11px;
font-weight: 600;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.05em;
border-bottom: 1px solid var(--border);
}
.col-check {
width: 40px;
}
.col-num {
width: 80px;
text-align: right;
}
.detail-table td.col-num {
text-align: right;
}
.col-action {
width: 80px;
text-align: right;
}
.doc-row {
cursor: pointer;
border-bottom: 1px solid var(--border);
transition: background var(--transition);
}
.doc-row:hover {
background: var(--bg-hover);
}
.detail-table td {
padding: 12px;
vertical-align: middle;
}
.col-name {
display: flex;
align-items: center;
gap: 8px;
min-width: 0;
}
.doc-icon {
width: 14px;
height: 14px;
color: var(--accent);
flex-shrink: 0;
}
.doc-filename {
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
font-weight: 500;
color: var(--text);
}
.col-date {
font-size: 12px;
font-family: 'IBM Plex Mono', monospace;
color: var(--text-muted);
white-space: nowrap;
}
.empty-state {
display: flex;
flex-direction: column;
align-items: center;
justify-content: center;
gap: 12px;
padding: 80px 24px;
text-align: center;
}
.empty-title {
font-size: 16px;
font-weight: 600;
color: var(--text);
}
.bulk-bar {
position: absolute;
bottom: 0;
left: 0;
right: 0;
display: flex;
align-items: center;
justify-content: space-between;
padding: 12px 24px;
background: var(--bg-elevated);
border-top: 1px solid var(--border);
gap: 12px;
}
.bulk-count {
font-size: 13px;
font-weight: 600;
color: var(--text);
white-space: nowrap;
}
.bulk-actions {
display: flex;
gap: 8px;
}
.btn-primary {
display: inline-flex;
align-items: center;
gap: 6px;
padding: 7px 14px;
font-size: 13px;
font-weight: 500;
color: white;
background: var(--accent);
border: none;
border-radius: var(--radius-sm);
cursor: pointer;
text-decoration: none;
transition: background var(--transition);
white-space: nowrap;
}
.btn-primary:hover {
background: var(--accent-hover);
}
.btn-secondary {
padding: 7px 14px;
font-size: 13px;
font-weight: 500;
color: var(--text-secondary);
background: var(--bg-elevated);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
}
.btn-secondary:hover {
background: var(--bg-hover);
color: var(--text);
}
.header-actions {
display: flex;
gap: 8px;
}
.btn-danger {
color: #b91c1c;
border-color: rgba(220, 38, 38, 0.4);
}
.btn-danger:hover:not(:disabled) {
background: rgba(220, 38, 38, 0.08);
}
.btn-danger:disabled {
opacity: 0.5;
cursor: not-allowed;
}
.btn-sm {
padding: 5px 10px;
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
background: var(--bg-surface);
border: 1px solid var(--border);
border-radius: var(--radius-sm);
cursor: pointer;
transition: all var(--transition);
white-space: nowrap;
}
.btn-sm:hover {
background: var(--bg-hover);
color: var(--text);
}
.btn-sm--danger {
color: var(--error);
border-color: rgba(239, 68, 68, 0.3);
}
.btn-sm--danger:hover {
background: rgba(239, 68, 68, 0.1);
}
.btn-sm--ghost {
border-color: transparent;
background: transparent;
}
</style>

View file

@ -0,0 +1,118 @@
<template>
<div class="store-edit-page">
<header class="header">
<h1 class="title">{{ t('storeForm.titleEdit') }}</h1>
</header>
<div v-if="loading" class="loading-state">
<div class="spinner" />
</div>
<div v-else-if="loadError" class="error-state">
<p class="error-text">{{ loadError }}</p>
</div>
<StoreForm
v-else-if="initial"
mode="edit"
:initial-value="initial"
:submitting="submitting"
:error-message="errorMessage"
:lock-slug="initial.slug === 'default'"
@submit="onSubmit"
@cancel="onCancel"
/>
</div>
</template>
<script setup lang="ts">
import { onMounted, ref } from 'vue'
import { useRouter } from 'vue-router'
import StoreForm from '../features/store/ui/StoreForm.vue'
import {
fetchStore,
updateStore,
type StoreCreatePayload,
type StoreDetail,
} from '../features/store/api'
import { ROUTES } from '../shared/routing/names'
import { useI18n } from '../shared/i18n'
const props = defineProps<{ store: string }>()
const { t } = useI18n()
const router = useRouter()
const initial = ref<StoreDetail | null>(null)
const loading = ref(true)
const loadError = ref<string | null>(null)
const submitting = ref(false)
const errorMessage = ref<string | null>(null)
onMounted(async () => {
try {
initial.value = await fetchStore(props.store)
} catch (err) {
loadError.value = err instanceof Error ? err.message : String(err)
} finally {
loading.value = false
}
})
async function onSubmit(payload: StoreCreatePayload) {
submitting.value = true
errorMessage.value = null
try {
const updated = await updateStore(props.store, payload)
if (updated.slug !== props.store) {
router.push({ name: ROUTES.STORE_DETAIL, params: { store: updated.slug } })
} else {
router.push({ name: ROUTES.STORE_DETAIL, params: { store: updated.slug } })
}
} catch (err) {
errorMessage.value = err instanceof Error ? err.message : String(err)
} finally {
submitting.value = false
}
}
function onCancel() {
router.push({ name: ROUTES.STORES_LIST })
}
</script>
<style scoped>
.store-edit-page {
padding: 1.5rem 2rem;
}
.header {
margin-bottom: 1.5rem;
}
.title {
font-size: 1.5rem;
font-weight: 600;
margin: 0;
}
.loading-state,
.error-state {
padding: 2rem;
display: flex;
justify-content: center;
}
.error-text {
color: #dc2626;
}
.spinner {
width: 2rem;
height: 2rem;
border: 3px solid #e5e7eb;
border-top-color: #2563eb;
border-radius: 50%;
animation: spin 0.6s linear infinite;
}
@keyframes spin {
to {
transform: rotate(360deg);
}
}
</style>

View file

@ -0,0 +1,389 @@
<template>
<div class="query-page">
<div class="query-header">
<RouterLink :to="{ name: ROUTES.STORE_DETAIL, params: { store } }" class="back-link">
{{ t('storeQuery.back') }}
</RouterLink>
<h1 class="query-title">{{ store }}</h1>
</div>
<div class="query-form">
<div class="form-row">
<label class="form-label" for="query-input">{{ t('storeQuery.queryLabel') }}</label>
<div class="query-input-group">
<textarea
id="query-input"
v-model="queryText"
class="query-textarea"
:placeholder="t('storeQuery.queryPlaceholder')"
rows="3"
@keydown.ctrl.enter="run"
@keydown.meta.enter="run"
/>
<button class="btn-primary" :disabled="running || !queryText.trim()" @click="run">
{{ running ? t('storeQuery.running') : t('storeQuery.run') }}
</button>
</div>
</div>
<div class="form-row form-row--inline">
<label class="form-label" for="topk-input">{{ t('storeQuery.topKLabel') }}</label>
<input
id="topk-input"
v-model.number="topK"
type="number"
class="topk-input"
min="1"
max="50"
/>
</div>
</div>
<div v-if="error" class="error-banner">{{ error }}</div>
<div v-if="results !== null" class="results-section">
<div v-if="results.length === 0" class="empty-results">
{{ t('storeQuery.empty') }}
</div>
<table v-else class="results-table">
<thead>
<tr>
<th class="col-score">{{ t('storeQuery.colScore') }}</th>
<th>{{ t('storeQuery.colDoc') }}</th>
<th>{{ t('storeQuery.colText') }}</th>
<th class="col-page">{{ t('storeQuery.colPage') }}</th>
<th class="col-view" />
</tr>
</thead>
<tbody>
<tr v-for="(r, idx) in results" :key="idx" class="result-row">
<td class="col-score">
<span class="score-badge">{{ r.score.toFixed(3) }}</span>
</td>
<td class="col-doc">
<span class="doc-name" :title="r.filename">{{ r.filename }}</span>
</td>
<td class="col-text">
<span class="excerpt">{{ r.text }}</span>
</td>
<td class="col-page">
<span v-if="r.pageRange" class="page-range">
{{ r.pageRange[0] }}{{ r.pageRange[1] }}
</span>
<span v-else class="no-value"></span>
</td>
<td class="col-view">
<RouterLink
:to="{ name: ROUTES.DOC_WORKSPACE, params: { id: r.docId } }"
class="view-link"
>
{{ t('storeQuery.viewDoc') }}
</RouterLink>
</td>
</tr>
</tbody>
</table>
</div>
</div>
</template>
<script setup lang="ts">
import { ref } from 'vue'
import { RouterLink } from 'vue-router'
import { queryStore, type QueryResult } from '../features/store/api'
import { useI18n } from '../shared/i18n'
import { ROUTES } from '../shared/routing/names'
const props = defineProps<{ store: string }>()
const { t } = useI18n()
const queryText = ref('')
const topK = ref(5)
const running = ref(false)
const error = ref<string | null>(null)
const results = ref<QueryResult[] | null>(null)
async function run(): Promise<void> {
const q = queryText.value.trim()
if (!q || running.value) return
running.value = true
error.value = null
results.value = null
try {
results.value = await queryStore(props.store, q, topK.value)
} catch (e) {
error.value = e instanceof Error ? e.message : String(e)
} finally {
running.value = false
}
}
</script>
<style scoped>
.query-page {
display: flex;
flex-direction: column;
height: 100%;
overflow: hidden;
}
.query-header {
display: flex;
align-items: center;
gap: 12px;
padding: 20px 24px 16px;
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.back-link {
font-size: 13px;
color: var(--text-muted);
text-decoration: none;
transition: color var(--transition);
white-space: nowrap;
}
.back-link:hover {
color: var(--text);
}
.query-title {
font-size: 20px;
font-weight: 600;
font-family: 'IBM Plex Mono', monospace;
color: var(--text);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.query-form {
display: flex;
flex-direction: column;
gap: 12px;
padding: 16px 24px;
border-bottom: 1px solid var(--border);
flex-shrink: 0;
}
.form-row {
display: flex;
flex-direction: column;
gap: 6px;
}
.form-row--inline {
flex-direction: row;
align-items: center;
gap: 12px;
}
.form-label {
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
}
.query-input-group {
display: flex;
gap: 8px;
align-items: flex-start;
}
.query-textarea {
flex: 1;
padding: 8px 10px;
border-radius: var(--radius-sm);
border: 1px solid var(--border);
background: var(--bg-elevated);
color: var(--text);
font-size: 13px;
font-family: inherit;
outline: none;
resize: vertical;
transition: border-color var(--transition);
}
.query-textarea:focus {
border-color: var(--accent);
}
.topk-input {
width: 80px;
padding: 6px 8px;
border-radius: var(--radius-sm);
border: 1px solid var(--border);
background: var(--bg-elevated);
color: var(--text);
font-size: 13px;
outline: none;
transition: border-color var(--transition);
}
.topk-input:focus {
border-color: var(--accent);
}
.error-banner {
margin: 12px 24px 0;
padding: 10px 14px;
border-radius: var(--radius-sm);
font-size: 13px;
color: var(--error);
background: rgba(239, 68, 68, 0.08);
border: 1px solid rgba(239, 68, 68, 0.2);
flex-shrink: 0;
}
.results-section {
flex: 1;
overflow-y: auto;
padding: 0 24px 24px;
}
.empty-results {
padding: 40px;
text-align: center;
font-size: 14px;
color: var(--text-muted);
}
.results-table {
width: 100%;
border-collapse: collapse;
font-size: 13px;
}
.results-table thead {
position: sticky;
top: 0;
background: var(--bg);
z-index: 1;
}
.results-table th {
padding: 10px 12px;
text-align: left;
font-size: 11px;
font-weight: 600;
color: var(--text-muted);
text-transform: uppercase;
letter-spacing: 0.05em;
border-bottom: 1px solid var(--border);
}
.col-score {
width: 80px;
}
.col-page {
width: 80px;
text-align: right;
}
.results-table td.col-page {
text-align: right;
}
.col-view {
width: 60px;
text-align: right;
}
.result-row {
border-bottom: 1px solid var(--border);
}
.results-table td {
padding: 12px;
vertical-align: top;
}
.score-badge {
display: inline-block;
padding: 2px 7px;
border-radius: var(--radius-sm);
font-size: 11px;
font-family: 'IBM Plex Mono', monospace;
font-weight: 600;
background: var(--accent-muted);
color: var(--accent);
}
.col-doc {
max-width: 200px;
}
.doc-name {
font-size: 12px;
font-weight: 500;
color: var(--text-secondary);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
display: block;
}
.col-text {
min-width: 0;
}
.excerpt {
display: -webkit-box;
-webkit-line-clamp: 3;
-webkit-box-orient: vertical;
overflow: hidden;
font-size: 12px;
line-height: 1.5;
color: var(--text-secondary);
}
.page-range {
font-size: 12px;
font-family: 'IBM Plex Mono', monospace;
color: var(--text-muted);
}
.no-value {
color: var(--text-muted);
}
.view-link {
font-size: 12px;
color: var(--accent);
text-decoration: none;
font-weight: 500;
transition: color var(--transition);
}
.view-link:hover {
color: var(--accent-hover);
}
.btn-primary {
display: inline-flex;
align-items: center;
padding: 8px 16px;
font-size: 13px;
font-weight: 500;
color: white;
background: var(--accent);
border: none;
border-radius: var(--radius-sm);
cursor: pointer;
text-decoration: none;
transition: background var(--transition);
white-space: nowrap;
}
.btn-primary:hover:not(:disabled) {
background: var(--accent-hover);
}
.btn-primary:disabled {
opacity: 0.4;
cursor: not-allowed;
}
</style>

Some files were not shown because too many files have changed in this diff Show more