Commit graph

5 commits

Author SHA1 Message Date
Chris McDonough
8deac2fee8
Fix constant re-ingestion when server provides no revision header
HTTP, S3, and WebDAV sources all check `revision is not None and
snapshot.get(uri) == revision` to decide UPSERT vs UNCHANGED.  When
a server returns no ETag or Last-Modified, revision is None and the
condition always fails — every sweep emits UPSERT even though the
content hasn't changed.

Now emit UNCHANGED when revision is None and the URI is already
known (has been ingested before).  A first-time discovery with no
revision still correctly emits UPSERT.
2026-06-01 18:14:27 +03:00
Yiorgis Gozadinos
b0d0ac588d
Split snapshot APIs and resolve_fetcher by intent 2026-05-27 14:39:54 +03:00
Yiorgis Gozadinos
13cbadeb6f
canonical metadata keys: source_revision + content_type, bump to 0.50.0 2026-05-26 11:43:30 +03:00
Yiorgis Gozadinos
717fc3320b
route create_document_from_source through source adapters 2026-05-26 11:41:54 +03:00
Yiorgis Gozadinos
b9637cd625
add HTTP and S3 source adapters + resolve_fetcher 2026-05-26 11:41:53 +03:00