soulsync/core/imports
BoulderBadgeDad 88da265ef4 Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five
Measured during a live album download: ~4m15s per track in post-processing
(normal is ~20s), with the time vanishing silently inside embed_source_ids —
up to 5 MusicBrainz calls per track crawling against a degraded musicbrainz.org
while the MB enrichment worker kept eating the same ~1 req/s per-IP budget.
Only Spotify/Last.fm/Genius were in the yield set; MusicBrainz, Deezer, iTunes,
Discogs etc. kept grinding through downloads.

Policy (new core/enrichment/yield_policy, tested):
- downloads active  -> ALL enrichment workers yield (post-processing touches
  every metadata source). listening-stats (local-only) and repair
  (user-scheduled) intentionally keep running.
- discovery active  -> the API-contention five yield (spotify/itunes/deezer/
  discogs/hydrabase) — discovery never paused anything before, despite the
  pause helper literally defaulting to label='discovery'.
- user overrides and user-paused bookkeeping keep their existing semantics;
  the dashboard yield_reason label now says WHICH foreground work caused it.

Observability (the 4-minute silence can never come back):
- every source lookup is timed; >2s logs a warning NAMING the source and
  duration (core/metadata/source.py _call_source_lookup)
- the pipeline always logs "Metadata enhancement took X.Xs" per track

7 policy tests (incl. the motivating case: MB yields to downloads, keeps
running during discovery); 277 pipeline/enrichment tests pass.
2026-06-06 19:05:56 -07:00
..
__init__.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
album.py Show MusicBrainz release variants in import 2026-05-24 09:33:19 -07:00
album_matching.py Auto-import matching: fix Deezer source classification + bump tolerance 2026-05-09 15:53:17 -07:00
album_naming.py Rehome import helpers into core/imports 2026-04-27 19:54:44 +03:00
context.py Wishlist payloads: preserve real track_number + release_date end-to-end 2026-05-27 14:25:03 -07:00
file_integrity.py Integrity check: don't quarantine valid streamed FLAC as 'zero-length' (#756) 2026-05-31 11:28:40 -07:00
file_ops.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
filename.py Fix album-bundle downloads landing every track as track 1 2026-05-26 21:04:27 -07:00
guards.py Quarantine management — list, approve, delete, recover 2026-05-14 08:06:19 -07:00
paths.py Fix #767: Library Organizer dry run no longer creates folders 2026-06-02 10:32:06 -07:00
pipeline.py Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five 2026-06-06 19:05:56 -07:00
quarantine.py Quarantine: manage a quarantined file from the download modal (Listen / Accept / Search) 2026-05-31 15:41:04 -07:00
resolution.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
routes.py Fix #764: manual import reported quarantined files as a successful "Done" 2026-06-02 08:40:26 -07:00
side_effects.py Remove the old Retag Tool (superseded by Library Re-tag job + Write Tags) 2026-06-04 09:33:03 -07:00
staging.py Fix album-bundle downloads landing every track as track 1 2026-05-26 21:04:27 -07:00
track_number.py Wishlist: fix three regressions causing all imports to land as track 01 with no year 2026-05-27 15:39:22 -07:00