danvics/soulsync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

PR5d: lift _run_deezer_discovery_worker to core/discovery/deezer.py

Browse source 
Fourth lift in the PR5 discovery-workers series. Pulls the 270-line
Deezer discovery worker out of `web_server.py` into its own focused
module under `core/discovery/`. Pure 1:1 lift — wrapper keeps the
original entry-point name so the existing call sites continue to work
without changes.

What the Deezer discovery worker does:

1. Pause enrichment workers (release shared resources).
2. For each Deezer track:
   - Cancellation gate (state['cancelled']).
   - Discovery cache lookup; cache hit short-circuits the search and
     populates display fields from the cached match (artist string,
     album name).
   - SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
     (shared search helper, returns tuple for Spotify or dict for iTunes).
   - On Spotify match: build `match_data` preserving track_number /
     disc_number from raw API data, image extracted from album images
     or track object fallback, release_date filled from track.release_date
     when album dict is missing it.
   - On iTunes match: dict result populated as `match_data`, source set
     to discovery_source, image extracted from album images.
   - Save matched result to discovery cache.
   - On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry, sync
   discovery results back to mirrored playlist via
   `_sync_discovery_results_to_mirrored`.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.

Dependencies injected via `DeezerDiscoveryDeps` (13 fields) —
deezer_discovery_states dict, spotify_client, plus 11 callable helpers
(pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item,
sync_discovery_results_to_mirrored).

Diff vs original after `deps.X` → global X normalization is **zero
differences** — 270 lines orig = 270 lines lifted, byte-identical body
(including all whitespace, comments, log strings).

Tests: 10 new under tests/discovery/test_discovery_deezer.py covering
cache hit short-circuit, Spotify tuple match (track/disc number
preservation), iTunes dict match path, Wing It fallback, cancellation,
completion phase update, activity feed entry, mirrored sync invocation,
top-level error handler, per-track error handling.

Full suite: 1108 passing (was 1098). Ruff clean.
This commit is contained in:
Broque Thomas 2026-04-28 12:45:18 -07:00
parent 2bf07b0336
commit 2bc665e487
3 changed files with 674 additions and 268 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

327
core/discovery/deezer.py Normal file
View file

@ -0,0 +1,327 @@
"""Background worker for Deezer playlist discovery.
`run_deezer_discovery_worker(playlist_id, deps)` is the function the
Deezer discovery start-endpoint submits to the deezer discovery executor
to match each Deezer playlist track against Spotify (preferred) or iTunes
(fallback):
1. Pause enrichment workers (release shared resources).
2. For each Deezer track:
- Cancellation gate (state['cancelled']).
- Discovery cache lookup; cache hit short-circuits the search.
- SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
(shared search helper, returns tuple for Spotify or dict for iTunes).
- On match: build `match_data` (Spotify path preserves track_number /
disc_number from raw API data, image extracted from album images).
- Save to discovery cache.
- On miss: Wing It stub created from raw Deezer track data.
3. After all tracks: phase='discovered', activity feed entry.
4. Sync discovery results back to mirrored playlist via
`_sync_discovery_results_to_mirrored`.
5. On error: state['phase']='error' + status with error string.
6. Finally: resume enrichment workers.
Lifted verbatim from web_server.py. Wide dependency surface (Spotify and
iTunes clients, multiple metadata helpers, state dict, mirrored sync,
shared tidal search helper) all injected via `DeezerDiscoveryDeps`.
"""
from __future__ import annotations
import logging
import time
import types
from dataclasses import dataclass
from typing import Any, Callable
logger = logging.getLogger(__name__)
@dataclass
class DeezerDiscoveryDeps:
"""Bundle of cross-cutting deps the Deezer discovery worker needs."""
deezer_discovery_states: dict
spotify_client: Any
pause_enrichment_workers: Callable[[str], dict]
resume_enrichment_workers: Callable[[dict, str], None]
get_active_discovery_source: Callable[[], str]
get_metadata_fallback_client: Callable[[], Any]
get_discovery_cache_key: Callable
get_database: Callable[[], Any]
validate_discovery_cache_artist: Callable
search_spotify_for_tidal_track: Callable
build_discovery_wing_it_stub: Callable
add_activity_item: Callable
sync_discovery_results_to_mirrored: Callable
def run_deezer_discovery_worker(playlist_id, deps: DeezerDiscoveryDeps):
"""Background worker for Deezer discovery process (Spotify preferred, iTunes fallback)"""
_ew_state = {}
try:
_ew_state = deps.pause_enrichment_workers('Deezer discovery')
state = deps.deezer_discovery_states[playlist_id]
playlist = state['playlist']
# Determine which provider to use
discovery_source = deps.get_active_discovery_source()
use_spotify = (discovery_source == 'spotify') and deps.spotify_client and deps.spotify_client.is_spotify_authenticated()
# Initialize fallback client if needed
itunes_client_instance = None
if not use_spotify:
itunes_client_instance = deps.get_metadata_fallback_client()
logger.info(f"Starting Deezer discovery for: {playlist['name']} (using {discovery_source.upper()})")
# Store discovery source in state for frontend
state['discovery_source'] = discovery_source
successful_discoveries = 0
tracks = playlist['tracks']
for i, deezer_track in enumerate(tracks):
if state.get('cancelled', False):
break
try:
track_name = deezer_track['name']
track_artists = deezer_track['artists']
track_id = deezer_track['id']
track_album = deezer_track.get('album', '')
track_duration_ms = deezer_track.get('duration_ms', 0)
logger.info(f"[{i+1}/{len(tracks)}] Searching {discovery_source.upper()}: {track_name} by {', '.join(track_artists)}")
# Check discovery cache first
cache_key = deps.get_discovery_cache_key(track_name, track_artists[0] if track_artists else '')
try:
cache_db = deps.get_database()
cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
if cached_match and deps.validate_discovery_cache_artist(track_artists[0] if track_artists else '', cached_match):
logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {track_name} by {', '.join(track_artists)}")
# Extract display-friendly artist string from cached match
cached_artists = cached_match.get('artists', [])
if cached_artists:
cached_artist_str = ', '.join(
a if isinstance(a, str) else a.get('name', '') for a in cached_artists
)
else:
cached_artist_str = ''
cached_album = cached_match.get('album', '')
if isinstance(cached_album, dict):
cached_album = cached_album.get('name', '')
result = {
'deezer_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album,
'duration_ms': track_duration_ms,
},
'spotify_data': cached_match,
'match_data': cached_match,
'status': 'Found',
'status_class': 'found',
'spotify_track': cached_match.get('name', ''),
'spotify_artist': cached_artist_str,
'spotify_album': cached_album,
'spotify_id': cached_match.get('id', ''),
'discovery_source': discovery_source,
'index': i
}
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
continue
except Exception as cache_err:
logger.error(f"Cache lookup error: {cache_err}")
# Create a SimpleNamespace duck-type object for _search_spotify_for_tidal_track
track_ns = types.SimpleNamespace(
id=track_id,
name=track_name,
artists=track_artists,
album=track_album,
duration_ms=track_duration_ms
)
# Use the search function with appropriate provider
track_result = deps.search_spotify_for_tidal_track(
track_ns,
use_spotify=use_spotify,
itunes_client=itunes_client_instance
)
# Create result entry
result = {
'deezer_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album,
'duration_ms': track_duration_ms,
},
'spotify_data': None,
'match_data': None,
'status': 'Not Found',
'status_class': 'not-found',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'discovery_source': discovery_source
}
match_confidence = 0.0
if use_spotify and isinstance(track_result, tuple):
# Spotify: Function returns (Track, raw_data, confidence)
track_obj, raw_track_data, match_confidence = track_result
album_obj = raw_track_data.get('album', {}) if raw_track_data else {}
# Ensure album has a name — fall back to track_obj.album if raw_data was missing
if isinstance(album_obj, dict) and not album_obj.get('name') and track_obj.album:
album_obj['name'] = track_obj.album
elif not album_obj and track_obj.album:
album_obj = {'name': track_obj.album}
# Ensure release_date is present (raw Spotify data has it, but fallback may not)
if isinstance(album_obj, dict) and not album_obj.get('release_date'):
album_obj['release_date'] = getattr(track_obj, 'release_date', '') or ''
# Extract image URL from album data or track object
_album_images = album_obj.get('images', []) if isinstance(album_obj, dict) else []
_image_url = _album_images[0].get('url', '') if _album_images else (getattr(track_obj, 'image_url', '') or '')
match_data = {
'id': track_obj.id,
'name': track_obj.name,
'artists': track_obj.artists,
'album': album_obj,
'duration_ms': track_obj.duration_ms,
'external_urls': track_obj.external_urls,
'image_url': _image_url,
'source': 'spotify'
}
# Preserve track_number/disc_number from raw Spotify API data
if raw_track_data and raw_track_data.get('track_number'):
match_data['track_number'] = raw_track_data['track_number']
if raw_track_data and raw_track_data.get('disc_number'):
match_data['disc_number'] = raw_track_data['disc_number']
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = track_obj.name
result['spotify_artist'] = ', '.join(track_obj.artists) if isinstance(track_obj.artists, list) else str(track_obj.artists)
result['spotify_album'] = album_obj.get('name', '') if isinstance(album_obj, dict) else str(album_obj)
result['spotify_id'] = track_obj.id
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
elif not use_spotify and track_result and isinstance(track_result, dict):
# Fallback: Function returns a dict with track data (includes 'confidence' key)
match_confidence = track_result.pop('confidence', 0.80)
match_data = track_result
match_data['source'] = discovery_source
# Extract image URL from album images
_fb_album = match_data.get('album', {})
_fb_images = _fb_album.get('images', []) if isinstance(_fb_album, dict) else []
if _fb_images and 'image_url' not in match_data:
match_data['image_url'] = _fb_images[0].get('url', '')
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = match_data.get('name', '')
itunes_artists = match_data.get('artists', [])
result['spotify_artist'] = ', '.join(a if isinstance(a, str) else a.get('name', '') for a in itunes_artists) if itunes_artists else ''
result['spotify_album'] = match_data.get('album', {}).get('name', '') if isinstance(match_data.get('album'), dict) else match_data.get('album', '')
result['spotify_id'] = match_data.get('id', '')
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
# Save to discovery cache if match found
if result['status_class'] == 'found' and result.get('match_data'):
try:
cache_db = deps.get_database()
cache_db.save_discovery_cache_match(
cache_key[0], cache_key[1], discovery_source, match_confidence,
result['match_data'], track_name,
track_artists[0] if track_artists else ''
)
logger.info(f"CACHE SAVED: {track_name} (confidence: {match_confidence:.3f})")
except Exception as cache_err:
logger.error(f"Cache save error: {cache_err}")
# Auto Wing It fallback for unmatched tracks
if result['status_class'] == 'not-found':
deezer_t = result.get('deezer_track', {})
stub = deps.build_discovery_wing_it_stub(
deezer_t.get('name', ''),
', '.join(deezer_t.get('artists', [])),
deezer_t.get('duration_ms', 0)
)
result['status'] = 'Wing It'
result['status_class'] = 'wing-it'
result['spotify_data'] = stub
result['match_data'] = stub
result['spotify_track'] = deezer_t.get('name', '')
result['spotify_artist'] = ', '.join(deezer_t.get('artists', []))
result['wing_it_fallback'] = True
result['confidence'] = 0
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['wing_it_count'] = state.get('wing_it_count', 0) + 1
result['index'] = i
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Add delay between requests
time.sleep(0.1)
except Exception as e:
logger.error(f"Error processing track {i+1}: {e}")
# Add error result
result = {
'deezer_track': {
'name': deezer_track.get('name', 'Unknown'),
'artists': deezer_track.get('artists', []),
},
'spotify_data': None,
'match_data': None,
'status': 'Error',
'status_class': 'error',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'error': str(e),
'discovery_source': discovery_source,
'index': i
}
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Mark as complete
state['phase'] = 'discovered'
state['status'] = 'discovered'
state['discovery_progress'] = 100
# Add activity for discovery completion
source_label = discovery_source.upper()
deps.add_activity_item("", f"Deezer Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
logger.info(f"Deezer discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
# Sync discovery results back to mirrored playlist
deps.sync_discovery_results_to_mirrored('deezer', playlist_id, state.get('discovery_results', []), discovery_source, profile_id=state.get('_profile_id', 1))
except Exception as e:
logger.error(f"Error in Deezer discovery worker: {e}")
if playlist_id in deps.deezer_discovery_states:
deps.deezer_discovery_states[playlist_id]['phase'] = 'error'
deps.deezer_discovery_states[playlist_id]['status'] = f'error: {str(e)}'
finally:
deps.resume_enrichment_workers(_ew_state, 'Deezer discovery')

323
tests/discovery/test_discovery_deezer.py Normal file
View file

@ -0,0 +1,323 @@
"""Tests for core/discovery/deezer.py — Deezer discovery worker."""
from __future__ import annotations
from dataclasses import dataclass
import pytest
from core.discovery import deezer as dd
# ---------------------------------------------------------------------------
# Fakes
# ---------------------------------------------------------------------------
@dataclass
class _FakeTrackObj:
id: str = 'sp-1'
name: str = 'Found'
artists: list = None
album: str = 'Album X'
duration_ms: int = 200000
image_url: str = ''
external_urls: dict = None
release_date: str = '2024-01-01'
def __post_init__(self):
if self.artists is None:
self.artists = ['Found Artist']
if self.external_urls is None:
self.external_urls = {}
class _FakeSpotifyClient:
def __init__(self, authenticated=True):
self._authenticated = authenticated
def is_spotify_authenticated(self):
return self._authenticated
class _FakeDB:
def __init__(self, cache_match=None):
self._cache_match = cache_match
self.cache_saves = []
def get_discovery_cache_match(self, t, a, src):
return self._cache_match
def save_discovery_cache_match(self, t, a, src, conf, data, raw_t, raw_a):
self.cache_saves.append((t, a, src, conf))
def _build_deps(
*,
states=None,
spotify_auth=True,
discovery_source='spotify',
cache_match=None,
search_result=None,
sync_calls=None,
activity_log=None,
):
sync_calls = sync_calls if sync_calls is not None else []
activity_log = activity_log if activity_log is not None else []
db = _FakeDB(cache_match=cache_match)
spotify = _FakeSpotifyClient(authenticated=spotify_auth)
itunes = object() # placeholder
deps = dd.DeezerDiscoveryDeps(
deezer_discovery_states=states if states is not None else {},
spotify_client=spotify,
pause_enrichment_workers=lambda label: {'paused': True},
resume_enrichment_workers=lambda state, label: None,
get_active_discovery_source=lambda: discovery_source,
get_metadata_fallback_client=lambda: itunes,
get_discovery_cache_key=lambda title, artist: (title.lower(), artist.lower()),
get_database=lambda: db,
validate_discovery_cache_artist=lambda artist, m: True,
search_spotify_for_tidal_track=lambda track, use_spotify, itunes_client: search_result,
build_discovery_wing_it_stub=lambda title, artist, dur: {
'name': title, 'artists': [artist], 'duration_ms': dur, 'wing_it': True
},
add_activity_item=lambda *a, **kw: activity_log.append((a, kw)),
sync_discovery_results_to_mirrored=lambda *a, **kw: sync_calls.append((a, kw)),
)
deps._db = db
deps._spotify = spotify
deps._sync_calls = sync_calls
deps._activity_log = activity_log
return deps
def _seed_state(playlist_id, states, *, tracks=None, cancelled=False):
states[playlist_id] = {
'cancelled': cancelled,
'playlist': {'name': 'My Deezer Playlist', 'tracks': tracks or []},
'spotify_matches': 0,
'discovery_results': [],
'discovery_progress': 0,
}
def _track(track_id=1, name='Track', artists=None, album='Album', duration_ms=180000):
return {
'id': track_id,
'name': name,
'artists': artists or ['Artist'],
'album': album,
'duration_ms': duration_ms,
}
# ---------------------------------------------------------------------------
# Cache hit
# ---------------------------------------------------------------------------
def test_cache_hit_short_circuits():
"""Discovery cache hit appends Found result, no live search."""
states = {}
cached = {
'id': 'spt-cached',
'name': 'Cached Track',
'artists': ['Cached Artist'],
'album': {'name': 'Cached Album'},
}
_seed_state('p1', states, tracks=[_track()])
deps = _build_deps(states=states, cache_match=cached)
dd.run_deezer_discovery_worker('p1', deps)
state = states['p1']
assert state['spotify_matches'] == 1
result = state['discovery_results'][0]
assert result['status'] == 'Found'
assert result['spotify_track'] == 'Cached Track'
# ---------------------------------------------------------------------------
# Spotify path (tuple result)
# ---------------------------------------------------------------------------
def test_spotify_match_extracts_full_data():
"""Spotify result tuple → builds match_data with track/disc numbers preserved."""
states = {}
raw = {
'album': {'name': 'Spt Album', 'release_date': '2024-05-05', 'images': [{'url': 'http://i'}]},
'track_number': 4,
'disc_number': 2,
}
track_obj = _FakeTrackObj(name='Found Track', artists=['Found Artist'])
_seed_state('p2', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=(track_obj, raw, 0.95))
dd.run_deezer_discovery_worker('p2', deps)
state = states['p2']
result = state['discovery_results'][0]
assert result['status'] == 'Found'
assert result['confidence'] == 0.95
md = result['match_data']
assert md['track_number'] == 4
assert md['disc_number'] == 2
assert md['album']['release_date'] == '2024-05-05'
assert md['image_url'] == 'http://i'
# ---------------------------------------------------------------------------
# iTunes path (dict result)
# ---------------------------------------------------------------------------
def test_itunes_fallback_dict_result():
"""When use_spotify=False, dict result populates match_data."""
states = {}
track_data = {
'id': 'it-1',
'name': 'iTunes Match',
'artists': ['iA'],
'album': {'name': 'iAlb', 'images': [{'url': 'http://it'}]},
'duration_ms': 210000,
'confidence': 0.85,
}
_seed_state('p3', states, tracks=[_track()])
deps = _build_deps(states=states, spotify_auth=False, discovery_source='itunes',
search_result=track_data)
dd.run_deezer_discovery_worker('p3', deps)
result = states['p3']['discovery_results'][0]
assert result['status'] == 'Found'
assert result['confidence'] == 0.85
assert result['match_data']['source'] == 'itunes'
assert result['match_data']['image_url'] == 'http://it'
# ---------------------------------------------------------------------------
# Wing It fallback
# ---------------------------------------------------------------------------
def test_no_match_falls_back_to_wing_it():
"""No match (None) → Wing It stub stored, status='Wing It'."""
states = {}
_seed_state('p4', states, tracks=[_track(name='Untitled', artists=['A'])])
deps = _build_deps(states=states, search_result=None)
dd.run_deezer_discovery_worker('p4', deps)
state = states['p4']
assert state.get('wing_it_count') == 1
result = state['discovery_results'][0]
assert result['status'] == 'Wing It'
assert result['wing_it_fallback'] is True
# ---------------------------------------------------------------------------
# Cancellation
# ---------------------------------------------------------------------------
def test_cancellation_breaks_loop():
"""state['cancelled']=True bails immediately."""
states = {}
_seed_state('p5', states, tracks=[_track(), _track(track_id=2)], cancelled=True)
deps = _build_deps(states=states)
dd.run_deezer_discovery_worker('p5', deps)
assert states['p5']['discovery_results'] == []
# ---------------------------------------------------------------------------
# Completion
# ---------------------------------------------------------------------------
def test_completion_marks_phase_discovered():
"""Completion → phase='discovered', status='discovered', progress=100."""
states = {}
_seed_state('p6', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=None)
dd.run_deezer_discovery_worker('p6', deps)
assert states['p6']['phase'] == 'discovered'
assert states['p6']['status'] == 'discovered'
assert states['p6']['discovery_progress'] == 100
def test_activity_feed_logged():
"""Completion emits an activity feed entry."""
states = {}
_seed_state('p7', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=None)
dd.run_deezer_discovery_worker('p7', deps)
assert len(deps._activity_log) == 1
# ---------------------------------------------------------------------------
# Mirrored sync
# ---------------------------------------------------------------------------
def test_sync_discovery_to_mirrored_called():
"""Completion calls sync_discovery_results_to_mirrored with playlist_id and source."""
states = {}
_seed_state('p8', states, tracks=[_track()])
states['p8']['_profile_id'] = 7
deps = _build_deps(states=states, search_result=None)
dd.run_deezer_discovery_worker('p8', deps)
assert len(deps._sync_calls) == 1
args, kwargs = deps._sync_calls[0]
assert args[0] == 'deezer'
assert args[1] == 'p8'
assert kwargs.get('profile_id') == 7
# ---------------------------------------------------------------------------
# Error path
# ---------------------------------------------------------------------------
def test_error_during_setup_marks_state_error():
"""Exception in main try (state lookup ok) → state phase='error'."""
states = {}
_seed_state('perr', states, tracks=[_track()])
deps = _build_deps(states=states)
# Force search helper to raise unhandled
deps.get_active_discovery_source = lambda: (_ for _ in ()).throw(RuntimeError("boom"))
dd.run_deezer_discovery_worker('perr', deps)
assert states['perr']['phase'] == 'error'
assert 'boom' in states['perr']['status']
# ---------------------------------------------------------------------------
# Per-track error
# ---------------------------------------------------------------------------
def test_per_track_error_appends_error_result():
"""Track-level exception → 'Error' result entry, loop continues."""
states = {}
tracks = [_track(track_id=1), _track(track_id=2)]
_seed_state('p9', states, tracks=tracks)
deps = _build_deps(states=states)
# First call raises, second returns None (Wing It path)
call_count = [0]
def search_side_effect(track, use_spotify, itunes_client):
call_count[0] += 1
if call_count[0] == 1:
raise RuntimeError("track-level boom")
return None
deps.search_spotify_for_tidal_track = search_side_effect
dd.run_deezer_discovery_worker('p9', deps)
state = states['p9']
assert len(state['discovery_results']) == 2
assert state['discovery_results'][0]['status'] == 'Error'
assert state['discovery_results'][1]['status'] == 'Wing It'

292
web_server.py
View file

@ -27104,276 +27104,32 @@ def update_deezer_playlist_phase(playlist_id):
return jsonify({"error": str(e)}), 500
# Deezer discovery worker logic lives in core/discovery/deezer.py.
from core.discovery import deezer as _discovery_deezer
def _build_deezer_discovery_deps():
"""Build the DeezerDiscoveryDeps bundle from web_server.py globals on each call."""
return _discovery_deezer.DeezerDiscoveryDeps(
deezer_discovery_states=deezer_discovery_states,
spotify_client=spotify_client,
pause_enrichment_workers=_pause_enrichment_workers,
resume_enrichment_workers=_resume_enrichment_workers,
get_active_discovery_source=_get_active_discovery_source,
get_metadata_fallback_client=_get_metadata_fallback_client,
get_discovery_cache_key=_get_discovery_cache_key,
get_database=get_database,
validate_discovery_cache_artist=_validate_discovery_cache_artist,
search_spotify_for_tidal_track=_search_spotify_for_tidal_track,
build_discovery_wing_it_stub=_build_discovery_wing_it_stub,
add_activity_item=add_activity_item,
sync_discovery_results_to_mirrored=_sync_discovery_results_to_mirrored,
)
def _run_deezer_discovery_worker(playlist_id):
"""Background worker for Deezer discovery process (Spotify preferred, iTunes fallback)"""
_ew_state = {}
try:
_ew_state = _pause_enrichment_workers('Deezer discovery')
state = deezer_discovery_states[playlist_id]
playlist = state['playlist']
return _discovery_deezer.run_deezer_discovery_worker(playlist_id, _build_deezer_discovery_deps())
# Determine which provider to use
discovery_source = _get_active_discovery_source()
use_spotify = (discovery_source == 'spotify') and spotify_client and spotify_client.is_spotify_authenticated()
# Initialize fallback client if needed
itunes_client_instance = None
if not use_spotify:
itunes_client_instance = _get_metadata_fallback_client()
logger.info(f"Starting Deezer discovery for: {playlist['name']} (using {discovery_source.upper()})")
# Store discovery source in state for frontend
state['discovery_source'] = discovery_source
successful_discoveries = 0
tracks = playlist['tracks']
for i, deezer_track in enumerate(tracks):
if state.get('cancelled', False):
break
try:
track_name = deezer_track['name']
track_artists = deezer_track['artists']
track_id = deezer_track['id']
track_album = deezer_track.get('album', '')
track_duration_ms = deezer_track.get('duration_ms', 0)
logger.info(f"[{i+1}/{len(tracks)}] Searching {discovery_source.upper()}: {track_name} by {', '.join(track_artists)}")
# Check discovery cache first
cache_key = _get_discovery_cache_key(track_name, track_artists[0] if track_artists else '')
try:
cache_db = get_database()
cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
if cached_match and _validate_discovery_cache_artist(track_artists[0] if track_artists else '', cached_match):
logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {track_name} by {', '.join(track_artists)}")
# Extract display-friendly artist string from cached match
cached_artists = cached_match.get('artists', [])
if cached_artists:
cached_artist_str = ', '.join(
a if isinstance(a, str) else a.get('name', '') for a in cached_artists
)
else:
cached_artist_str = ''
cached_album = cached_match.get('album', '')
if isinstance(cached_album, dict):
cached_album = cached_album.get('name', '')
result = {
'deezer_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album,
'duration_ms': track_duration_ms,
},
'spotify_data': cached_match,
'match_data': cached_match,
'status': 'Found',
'status_class': 'found',
'spotify_track': cached_match.get('name', ''),
'spotify_artist': cached_artist_str,
'spotify_album': cached_album,
'spotify_id': cached_match.get('id', ''),
'discovery_source': discovery_source,
'index': i
}
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
continue
except Exception as cache_err:
logger.error(f"Cache lookup error: {cache_err}")
# Create a SimpleNamespace duck-type object for _search_spotify_for_tidal_track
track_ns = types.SimpleNamespace(
id=track_id,
name=track_name,
artists=track_artists,
album=track_album,
duration_ms=track_duration_ms
)
# Use the search function with appropriate provider
track_result = _search_spotify_for_tidal_track(
track_ns,
use_spotify=use_spotify,
itunes_client=itunes_client_instance
)
# Create result entry
result = {
'deezer_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album,
'duration_ms': track_duration_ms,
},
'spotify_data': None,
'match_data': None,
'status': 'Not Found',
'status_class': 'not-found',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'discovery_source': discovery_source
}
match_confidence = 0.0
if use_spotify and isinstance(track_result, tuple):
# Spotify: Function returns (Track, raw_data, confidence)
track_obj, raw_track_data, match_confidence = track_result
album_obj = raw_track_data.get('album', {}) if raw_track_data else {}
# Ensure album has a name — fall back to track_obj.album if raw_data was missing
if isinstance(album_obj, dict) and not album_obj.get('name') and track_obj.album:
album_obj['name'] = track_obj.album
elif not album_obj and track_obj.album:
album_obj = {'name': track_obj.album}
# Ensure release_date is present (raw Spotify data has it, but fallback may not)
if isinstance(album_obj, dict) and not album_obj.get('release_date'):
album_obj['release_date'] = getattr(track_obj, 'release_date', '') or ''
# Extract image URL from album data or track object
_album_images = album_obj.get('images', []) if isinstance(album_obj, dict) else []
_image_url = _album_images[0].get('url', '') if _album_images else (getattr(track_obj, 'image_url', '') or '')
match_data = {
'id': track_obj.id,
'name': track_obj.name,
'artists': track_obj.artists,
'album': album_obj,
'duration_ms': track_obj.duration_ms,
'external_urls': track_obj.external_urls,
'image_url': _image_url,
'source': 'spotify'
}
# Preserve track_number/disc_number from raw Spotify API data
if raw_track_data and raw_track_data.get('track_number'):
match_data['track_number'] = raw_track_data['track_number']
if raw_track_data and raw_track_data.get('disc_number'):
match_data['disc_number'] = raw_track_data['disc_number']
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = track_obj.name
result['spotify_artist'] = ', '.join(track_obj.artists) if isinstance(track_obj.artists, list) else str(track_obj.artists)
result['spotify_album'] = album_obj.get('name', '') if isinstance(album_obj, dict) else str(album_obj)
result['spotify_id'] = track_obj.id
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
elif not use_spotify and track_result and isinstance(track_result, dict):
# Fallback: Function returns a dict with track data (includes 'confidence' key)
match_confidence = track_result.pop('confidence', 0.80)
match_data = track_result
match_data['source'] = discovery_source
# Extract image URL from album images
_fb_album = match_data.get('album', {})
_fb_images = _fb_album.get('images', []) if isinstance(_fb_album, dict) else []
if _fb_images and 'image_url' not in match_data:
match_data['image_url'] = _fb_images[0].get('url', '')
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = match_data.get('name', '')
itunes_artists = match_data.get('artists', [])
result['spotify_artist'] = ', '.join(a if isinstance(a, str) else a.get('name', '') for a in itunes_artists) if itunes_artists else ''
result['spotify_album'] = match_data.get('album', {}).get('name', '') if isinstance(match_data.get('album'), dict) else match_data.get('album', '')
result['spotify_id'] = match_data.get('id', '')
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
# Save to discovery cache if match found
if result['status_class'] == 'found' and result.get('match_data'):
try:
cache_db = get_database()
cache_db.save_discovery_cache_match(
cache_key[0], cache_key[1], discovery_source, match_confidence,
result['match_data'], track_name,
track_artists[0] if track_artists else ''
)
logger.info(f"CACHE SAVED: {track_name} (confidence: {match_confidence:.3f})")
except Exception as cache_err:
logger.error(f"Cache save error: {cache_err}")
# Auto Wing It fallback for unmatched tracks
if result['status_class'] == 'not-found':
deezer_t = result.get('deezer_track', {})
stub = _build_discovery_wing_it_stub(
deezer_t.get('name', ''),
', '.join(deezer_t.get('artists', [])),
deezer_t.get('duration_ms', 0)
)
result['status'] = 'Wing It'
result['status_class'] = 'wing-it'
result['spotify_data'] = stub
result['match_data'] = stub
result['spotify_track'] = deezer_t.get('name', '')
result['spotify_artist'] = ', '.join(deezer_t.get('artists', []))
result['wing_it_fallback'] = True
result['confidence'] = 0
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['wing_it_count'] = state.get('wing_it_count', 0) + 1
result['index'] = i
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Add delay between requests
time.sleep(0.1)
except Exception as e:
logger.error(f"Error processing track {i+1}: {e}")
# Add error result
result = {
'deezer_track': {
'name': deezer_track.get('name', 'Unknown'),
'artists': deezer_track.get('artists', []),
},
'spotify_data': None,
'match_data': None,
'status': 'Error',
'status_class': 'error',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'error': str(e),
'discovery_source': discovery_source,
'index': i
}
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Mark as complete
state['phase'] = 'discovered'
state['status'] = 'discovered'
state['discovery_progress'] = 100
# Add activity for discovery completion
source_label = discovery_source.upper()
add_activity_item("", f"Deezer Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
logger.info(f"Deezer discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
# Sync discovery results back to mirrored playlist
_sync_discovery_results_to_mirrored('deezer', playlist_id, state.get('discovery_results', []), discovery_source, profile_id=state.get('_profile_id', 1))
except Exception as e:
logger.error(f"Error in Deezer discovery worker: {e}")
if playlist_id in deezer_discovery_states:
deezer_discovery_states[playlist_id]['phase'] = 'error'
deezer_discovery_states[playlist_id]['status'] = f'error: {str(e)}'
finally:
_resume_enrichment_workers(_ew_state, 'Deezer discovery')
def convert_deezer_results_to_spotify_tracks(discovery_results):