Commit graph

1 commit

Author SHA1 Message Date
BoulderBadgeDad
88da265ef4 Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five
Measured during a live album download: ~4m15s per track in post-processing
(normal is ~20s), with the time vanishing silently inside embed_source_ids —
up to 5 MusicBrainz calls per track crawling against a degraded musicbrainz.org
while the MB enrichment worker kept eating the same ~1 req/s per-IP budget.
Only Spotify/Last.fm/Genius were in the yield set; MusicBrainz, Deezer, iTunes,
Discogs etc. kept grinding through downloads.

Policy (new core/enrichment/yield_policy, tested):
- downloads active  -> ALL enrichment workers yield (post-processing touches
  every metadata source). listening-stats (local-only) and repair
  (user-scheduled) intentionally keep running.
- discovery active  -> the API-contention five yield (spotify/itunes/deezer/
  discogs/hydrabase) — discovery never paused anything before, despite the
  pause helper literally defaulting to label='discovery'.
- user overrides and user-paused bookkeeping keep their existing semantics;
  the dashboard yield_reason label now says WHICH foreground work caused it.

Observability (the 4-minute silence can never come back):
- every source lookup is timed; >2s logs a warning NAMING the source and
  duration (core/metadata/source.py _call_source_lookup)
- the pipeline always logs "Metadata enhancement took X.Xs" per track

7 policy tests (incl. the motivating case: MB yields to downloads, keeps
running during discovery); 277 pipeline/enrichment tests pass.
2026-06-06 19:05:56 -07:00