danvics/soulsync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1
soulsync/core
History
BoulderBadgeDad 88da265ef4 Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five 
Measured during a live album download: ~4m15s per track in post-processing
(normal is ~20s), with the time vanishing silently inside embed_source_ids —
up to 5 MusicBrainz calls per track crawling against a degraded musicbrainz.org
while the MB enrichment worker kept eating the same ~1 req/s per-IP budget.
Only Spotify/Last.fm/Genius were in the yield set; MusicBrainz, Deezer, iTunes,
Discogs etc. kept grinding through downloads.

Policy (new core/enrichment/yield_policy, tested):
- downloads active  -> ALL enrichment workers yield (post-processing touches
  every metadata source). listening-stats (local-only) and repair
  (user-scheduled) intentionally keep running.
- discovery active  -> the API-contention five yield (spotify/itunes/deezer/
  discogs/hydrabase) — discovery never paused anything before, despite the
  pause helper literally defaulting to label='discovery'.
- user overrides and user-paused bookkeeping keep their existing semantics;
  the dashboard yield_reason label now says WHICH foreground work caused it.

Observability (the 4-minute silence can never come back):
- every source lookup is timed; >2s logs a warning NAMING the source and
  duration (core/metadata/source.py _call_source_lookup)
- the pipeline always logs "Metadata enhancement took X.Xs" per track

7 policy tests (incl. the motivating case: MB yields to downloads, keeps
running during discovery); 277 pipeline/enrichment tests pass.
2026-06-06 19:05:56 -07:00
..
artists Complete MusicBrainz watchlist source parity 2026-05-18 19:19:25 -07:00
automation Merge pull request #780 from kekkokk/feature/organize-by-playlist-library 2026-06-03 20:33:18 -07:00
diagnostics #802: on-demand memory-growth diagnostic (tracemalloc, browser-drivable) 2026-06-06 18:31:14 -07:00
discovery #799: stop mirrored discovery from reverting a manual fix to Wing It 2026-06-05 11:07:32 -07:00
download_engine Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
download_plugins Fix #796: Soulseek album bundle left completed files in slskd download folder 2026-06-04 21:56:07 -07:00
downloads #797: stop AcoustID quarantining correct non-English-artist downloads 2026-06-05 16:02:01 -07:00
enrichment Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five 2026-06-06 19:05:56 -07:00
imports Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five 2026-06-06 19:05:56 -07:00
library Auto-reconcile embedded IDs for new tracks on library scans 2026-06-05 18:31:11 -07:00
maintenance Add one-off repair for source ids shared across multiple artists 2026-06-05 08:38:55 -07:00
matching #797: stop AcoustID quarantining correct non-English-artist downloads 2026-06-05 16:02:01 -07:00
media_server Server playlist sync: append mode (preserve user-added tracks) 2026-05-10 22:52:11 -07:00
metadata Import speed: downloads pause ALL enrichment workers, discovery pauses the contention five 2026-06-06 19:05:56 -07:00
personalized Personalized pipeline: auto-refresh stale snapshots after watchlist scan 2026-05-15 20:53:03 -07:00
playback Player: log SoulSync web-player plays (recently-played + smart-radio recency) 2026-05-30 15:11:46 -07:00
playlists PR #780 follow-ups: snapshot-based stale check + submit guard + dead code 2026-06-03 20:45:17 -07:00
radio Player revamp Phase 2: smart radio ranking (play-count + popularity) 2026-05-30 08:47:18 -07:00
repair_jobs Library Re-tag: cover-mode scans stop producing unappliable "(0 track(s))" findings 2026-06-06 13:38:13 -07:00
search #798: Spotify Free metadata mode — backend (opt-in, shared spotify tables) 2026-06-05 13:32:37 -07:00
stats Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
streaming Stream button: the player never learns its stream is ready (2.6.5 regression) 2026-06-06 12:43:20 -07:00
sync Fix #792: reconcile sync mode was clamped back to 'replace' in the backend 2026-06-04 15:57:40 -07:00
text Fix #768: playlist sync editor refusing to match certain tracks 2026-06-02 10:16:21 -07:00
torrent_clients Fix #790: torrent client URL without http:// scheme fails to connect 2026-06-04 11:57:53 -07:00
usenet_clients Fix: Usenet bundle stuck at 99%/100% — SAB reports post-processing in History as non-terminal (#721) 2026-05-29 00:49:02 -04:00
watchlist Complete MusicBrainz watchlist source parity 2026-05-18 19:19:25 -07:00
webui Extract WebUI asset helpers 2026-05-13 22:26:25 +03:00
wishlist Fix organize-by-playlist downloads: library entries, wishlist, and stale Spotify cache 2026-06-03 10:26:32 +02:00
workers Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
acoustid_client.py AcoustID: report errors honestly instead of masking them as 'Skipped' 2026-05-31 20:10:31 -07:00
acoustid_verification.py #797: stop AcoustID quarantining correct non-English-artist downloads 2026-06-05 16:02:01 -07:00
album_consistency.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
amazon_client.py Fix #759: Amazon enrichment floods when its public proxy is down 2026-06-01 13:10:51 -07:00
amazon_download_client.py Amazon download client: write final size==transferred before returning file path 2026-05-16 11:09:04 -07:00
amazon_outage.py Fix #759: Amazon enrichment floods when its public proxy is down 2026-06-01 13:10:51 -07:00
amazon_worker.py Enrichment manager: 'process this group first' + refined hero header 2026-06-02 19:45:04 -07:00
api_call_tracker.py Wire Amazon Music as a first-class download source 2026-05-16 09:40:50 -07:00
archive_pipeline.py fix(downloads): harden album bundle staging 2026-05-20 21:39:06 -07:00
artist_source_detail.py Fix MusicBrainz artist detail showing MBID as name 2026-05-19 12:34:35 -07:00
artist_source_lookup.py Fix artist-detail showing wrong artist when a source id is duplicated 2026-06-05 07:24:51 -07:00
audiodb_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
audiodb_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
auto_import_worker.py Auto-import: fall through to other metadata sources when primary returns no match 2026-05-12 12:32:18 -07:00
automation_engine.py Wire automation engine through next_run_at + register monthly_time (PR 2/4) 2026-05-27 12:03:41 -07:00
connection_detect.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
connection_test.py Add torrent and usenet release staging support 2026-05-21 14:22:21 -07:00
database_update_worker.py Run auto-reconcile as a scan phase inside the running window 2026-06-05 19:10:31 -07:00
db_integrity.py DB backups: verify integrity + never evict the last good backup 2026-05-30 21:13:04 -07:00
debug_info.py Fix Copy Debug Info music_source + surface missing services 2026-05-12 16:47:55 -07:00
deezer_client.py Fix Deezer contributors tagging silently dropping for cache-polluted tracks 2026-05-14 11:10:51 -07:00
deezer_download_client.py Surface silent exceptions in metadata clients — 37 sites 2026-05-07 09:33:38 -07:00
deezer_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
discogs_client.py Discogs: strip artist disambiguation suffixes at every name surface (#634) 2026-05-27 09:05:47 -07:00
discogs_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
download_orchestrator.py Fix Usenet settings reload without restart 2026-05-22 08:28:56 -07:00
genius_client.py Add API Rate Monitor dashboard with real-time speedometer gauges 2026-04-02 10:46:43 -07:00
genius_worker.py Enrichment manager: 'process this group first' + refined hero header 2026-06-02 19:45:04 -07:00
genre_filter.py Expand default genre whitelist from 223 to 272 genres 2026-04-18 20:32:49 -07:00
hifi_client.py Log exception when inferring HiFi manifest ext 2026-05-21 18:24:56 -07:00
hydrabase_client.py feat: add explicit badges to discography modal and artist-detail cards 2026-05-12 23:35:08 -04:00
hydrabase_worker.py Improve graceful shutdown and rollback safety 2026-04-12 15:17:18 +03:00
image_cache.py Image cache: reject truncated downloads instead of caching broken covers (#750) 2026-05-30 17:45:01 -07:00
itunes_client.py feat: add explicit badges to discography modal and artist-detail cards 2026-05-12 23:35:08 -04:00
itunes_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
jellyfin_client.py Fix #792: 'reconcile' playlist sync mode (edit in place, keep image/description) 2026-06-04 15:15:49 -07:00
lastfm_client.py Add Last.fm Track Radio to Discover page 2026-04-14 23:41:36 -07:00
lastfm_worker.py Enrichment manager: 'process this group first' + refined hero header 2026-06-02 19:45:04 -07:00
library_reorganize.py #767-2: reorganize finds the right album edition instead of mislabeling singles as deluxe 2026-06-06 10:53:13 -07:00
lidarr_download_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
listenbrainz_client.py Remove emojis from all Python log and print statements 2026-04-11 21:11:02 -07:00
listenbrainz_manager.py Auto-Sync: fix LB pipelines stuck on "Refreshing:" for 5+ minutes 2026-05-27 18:04:55 -07:00
listening_stats_worker.py Final silent-exception sweep + ruff S110 lint guardrail — ~45 sites 2026-05-07 11:16:06 -07:00
lyrics_client.py Retag now re-embeds LYRICS tag instead of leaving it empty 2026-05-14 15:52:05 -07:00
matching_engine.py Fix: duplicate tracks in albums with Japanese / CJK titles (#722) 2026-05-28 08:53:43 -07:00
metadata_service.py Add MusicBrainz as a metadata source 2026-05-18 18:47:13 -07:00
musicbrainz_client.py MusicBrainz manual search: field-scope the artist in non-strict mode (#754) 2026-05-31 10:05:24 -07:00
musicbrainz_search.py Fix: MusicBrainz artist discography capped at 25 releases 2026-05-28 12:00:05 -07:00
musicbrainz_service.py Cross-script artist aliases: include canonical name + non-strict fallback 2026-05-14 13:07:15 -07:00
musicbrainz_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
navidrome_client.py Fix #792: 'reconcile' playlist sync mode (edit in place, keep image/description) 2026-06-04 15:15:49 -07:00
personalized_playlists.py Add MusicBrainz as a metadata source 2026-05-18 18:47:13 -07:00
plex_client.py Fix #792: 'reconcile' playlist sync mode (edit in place, keep image/description) 2026-06-04 15:15:49 -07:00
prowlarr_client.py feat(settings): add Prowlarr integration as indexer aggregator 2026-05-20 14:41:54 -07:00
qobuz_client.py Fetch all Qobuz favorite tracks for discovery 2026-05-24 01:17:38 -07:00
qobuz_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
reorganize_queue.py Reorganize: optional embedded-tag mode (closes #592) 2026-05-15 07:56:18 -07:00
reorganize_runner.py Reorganize: optional embedded-tag mode (closes #592) 2026-05-15 07:56:18 -07:00
repair_worker.py Library Re-tag: add light/full depth setting, default source to active, fix dropdown CSS 2026-06-04 10:21:30 -07:00
replaygain.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
runtime_state.py Add module logger + surface silent exceptions in 7 logger-less files — 12 sites 2026-05-07 10:27:04 -07:00
seasonal_discovery.py Fix source artist detail navigation from discover modals 2026-05-18 13:50:10 -07:00
similar_artists_worker.py Lint: fix ruff B905 (zip strict=) + S110 (try/except/pass) in similar-artists worker + perf endpoint 2026-06-03 16:35:19 -07:00
socketio_cors.py Socket.IO CORS: handle self-review nits 2026-04-26 19:24:43 -07:00
soulid_worker.py Surface silent exceptions in workers + repair jobs — ~30 sites 2026-05-07 10:27:24 -07:00
soulseek_client.py Fix #796: Soulseek album bundle left completed files in slskd download folder 2026-06-04 21:56:07 -07:00
soulsync_client.py Surface silent exceptions across remaining modules — ~70 sites 2026-05-07 10:28:58 -07:00
soundcloud_client.py Surface engine-not-wired errors + exclude soulseek from monitor aggregation 2026-05-05 12:20:51 -07:00
source_ids.py Source IDs: add canonical registry, adopt at the highest-value sites 2026-05-29 12:19:59 -07:00
spotify_client.py Mirrored "Liked Songs" stops 400ing on every auto-refresh (wolf39us) 2026-06-06 13:50:49 -07:00
spotify_free_metadata.py Bridge the Spotify worker to Free when the daily budget is spent (don't pause) 2026-06-06 08:51:30 -07:00
spotify_public_api.py SpotipyFree integration confirmed working (236 tracks live); deps + meta tweak 2026-06-02 22:50:04 -07:00
spotify_public_scraper.py Diagnostics: route public-fetch logs to soulsync namespace + log HTTP status 2026-06-02 22:09:51 -07:00
spotify_worker.py Bridge the Spotify worker to Free when the daily budget is spent (don't pause) 2026-06-06 08:51:30 -07:00
tag_writer.py #800: Write Tags must not overwrite a real file value with a placeholder 2026-06-05 20:07:03 -07:00
tidal_client.py Your Albums: selectable wishlist modal + Tidal album resolution 2026-05-11 12:36:16 -07:00
tidal_download_client.py Handle non-JSON Tidal auth responses 2026-05-20 14:04:45 -07:00
tidal_worker.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
watchlist_scanner.py #798: Spotify Free metadata mode — backend (opt-in, shared spotify tables) 2026-06-05 13:32:37 -07:00
web_scan_manager.py MS Cin-5: Drop per-server globals — engine owns the clients 2026-05-05 18:05:45 -07:00
wishlist_service.py Extract wishlist logic into dedicated package 2026-04-28 21:17:24 +03:00
worker_utils.py Tighten artist matching: 0.85 gate + shared uniqueness guard 2026-06-05 10:01:17 -07:00
youtube_client.py YouTube: ship the JS runtime + nightly yt-dlp so streams/music videos work out of the box 2026-06-06 16:50:24 -07:00