danvics/haiku.rag
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
haiku.rag/tests/ingester
History
Chris McDonough 8deac2fee8
Fix constant re-ingestion when server provides no revision header 
HTTP, S3, and WebDAV sources all check `revision is not None and
snapshot.get(uri) == revision` to decide UPSERT vs UNCHANGED.  When
a server returns no ETag or Last-Modified, revision is None and the
condition always fails — every sweep emits UPSERT even though the
content hasn't changed.

Now emit UNCHANGED when revision is None and the URI is already
known (has been ingested before).  A first-time discovery with no
revision still correctly emits UPSERT.
2026-06-01 18:14:27 +03:00
..
__init__.py ingester: add source-adapter scaffolding + FS source 2026-05-26 11:41:53 +03:00
test_api.py Probe docling-serve only when converter or chunker uses it 2026-05-27 15:18:25 +03:00
test_circuit_breaker.py additional config, pollers, serve 2026-05-26 11:41:54 +03:00
test_cli.py Strengthen config-load assertions 2026-06-01 18:03:09 +03:00
test_config.py Collapse worker_count and max_concurrent into worker_count 2026-05-27 12:48:14 +03:00
test_filter.py Improve test coverage for cli, filter, registry, and migrations 2026-06-01 18:00:47 +03:00
test_fs_source.py Improve discover stat race test to actually exercise the try/except 2026-06-01 09:28:26 -04:00
test_http_source.py Fix constant re-ingestion when server provides no revision header 2026-06-01 18:14:27 +03:00
test_pipeline.py Classify IsADirectoryError and NotADirectoryError as PermanentError 2026-06-01 17:59:23 +03:00
test_pollers.py Merge pull request #396 from mcdonc/perf/stagger-periodic-polls 2026-06-01 17:23:22 +03:00
test_queue.py Auto-prune dead jobs when a sibling DELETE succeeds 2026-05-27 17:17:01 +03:00
test_resolve_fetcher.py Improve test coverage for cli, filter, registry, and migrations 2026-06-01 18:00:47 +03:00
test_revision_round_trip.py record cassettes against new fixtures, fix stale assertions 2026-05-26 11:44:46 +03:00
test_run_batch.py Merge pull request #405 from mcdonc/fix/run-batch-hang-on-dead-workers 2026-06-01 17:51:49 +03:00
test_s3_source.py Fix constant re-ingestion when server provides no revision header 2026-06-01 18:14:27 +03:00
test_serve_integration.py Collapse worker_count and max_concurrent into worker_count 2026-05-27 12:48:14 +03:00
test_sources_base.py Fix shutdown-order bug: close source clients after workers stop 2026-06-01 09:55:39 -04:00
test_webdav_source.py Fix constant re-ingestion when server provides no revision header 2026-06-01 18:14:27 +03:00
test_workers.py Merge pull request #409 from mcdonc/fix/sync-state-write-crash 2026-06-01 17:46:46 +03:00