docling-studio/document-parser
Pier-Jean Malandrino 4b59ef2330 feat(#204): deterministic chunkset hash for auto-stale detection
Adds the pure-domain hash function that summarises a chunkset for
stale-detection purposes. Recorded on each DocumentStoreLink at push
time (#203 ships the column slot); compared against the recomputed
current hash to flip a link to Stale when the source has drifted.

domain/hashing.py
- chunkset_hash(chunks: Iterable[ChunkResult]) -> str
- SHA-256 over (text, source_page, headings) per chunk
- Excludes bboxes / doc_items / token_count by design
- 0x1F separator between chunks defends against the join-attack
  (split A+B vs concat AB)

Tests
- 9 tests: determinism, sensitivity per included field, exclusion of
  rendering-only fields, join-attack resistance, order sensitivity,
  empty-input invariant
- Locked fixture: a hand-built 3-chunk input has a fixed expected hash;
  CI fails loud if anyone changes the canonical inputs without updating
  the fixture deliberately (and the release notes)

Service integration (recompute on chunk write, set on push) lands with
#205 once chunks are first-class — direct integration on the legacy
chunks_json path is deliberately deferred to keep #204 focused.

Refs #204
2026-05-05 09:38:39 +02:00
..
api feat(#202): introduce Document lifecycle state machine 2026-05-05 09:38:39 +02:00
domain feat(#204): deterministic chunkset hash for auto-stale detection 2026-05-05 09:38:39 +02:00
infra refactor(audit): remediate 0.5.0 audit findings — clean architecture, security, DRY, SOLID, perf 2026-04-29 14:00:00 +02:00
persistence feat(#203): per (document, store) ingestion state 2026-05-05 09:38:39 +02:00
services feat(#202): introduce Document lifecycle state machine 2026-05-05 09:38:39 +02:00
tests feat(#204): deterministic chunkset hash for auto-stale detection 2026-05-05 09:38:39 +02:00
.dockerignore Work on full Docker integration 2026-03-17 13:33:36 +01:00
conftest.py Format backend codebase with ruff 2026-04-02 12:33:07 +02:00
Dockerfile Fix RapidOCR model download permission in local image 2026-04-03 10:44:20 +02:00
main.py refactor(audit): remediate 0.5.0 audit findings — clean architecture, security, DRY, SOLID, perf 2026-04-29 14:00:00 +02:00
package-lock.json Add slide on side bar in Studio mode 2026-03-20 16:24:38 +01:00
pyproject.toml Add qualityt check and contributing doc 2026-03-21 15:34:54 +01:00
pytest.ini Add a stronger testing strategy 2026-03-18 14:46:37 +01:00
requirements-local.txt Move bbox.py from domain/ to infra/ and unify coordinate logic 2026-04-03 13:17:26 +02:00
requirements-test.txt feat: add hexagonal architecture tests with pytestarch (#177) 2026-04-29 14:00:00 +02:00
requirements.txt refactor(audit): remediate 0.5.0 audit findings — clean architecture, security, DRY, SOLID, perf 2026-04-29 14:00:00 +02:00