danvics/soulsync
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

PR5e: lift _run_spotify_public_discovery_worker to core/discovery/spotify_public.py

Browse source 
Fifth lift in the PR5 discovery-workers series. Pulls the 278-line
public-Spotify-link discovery worker out of `web_server.py` into its
own focused module under `core/discovery/`. Pure 1:1 lift — wrapper
keeps the original entry-point name.

What the Spotify Public discovery worker does:

1. Pause enrichment workers (release shared resources).
2. For each track:
   - Cancellation gate (state['cancelled']).
   - Normalize artists to plain string list (handles dict + str inputs).
   - Discovery cache lookup; cache hit short-circuits the search and
     populates display fields from the cached match.
   - SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
     (shared search helper, returns tuple for Spotify or dict for iTunes).
   - On Spotify match: build `match_data` preserving track_number /
     disc_number from raw API data; image extracted from album images
     or track object fallback; release_date filled from track.release_date
     when album dict is missing it.
   - On iTunes match: dict result → match_data with source set to
     discovery_source; image extracted from album images.
   - Save matched result to discovery cache.
   - On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.

This worker is structurally close to the Deezer worker (see PR5d) but
intentionally diverges on:
- Track-data field names (`spotify_public_track` vs `deezer_track`).
- Artist normalization (Spotify Public can pass dicts or strings).
- No mirrored-playlist DB writeback (sync is handled separately).

Dependencies injected via `SpotifyPublicDiscoveryDeps` (12 fields) —
spotify_public_discovery_states, spotify_client, plus 10 callable
helpers (pause/resume enrichment, get_active_discovery_source,
get_metadata_fallback_client, get_discovery_cache_key, get_database,
validate_discovery_cache_artist, search_spotify_for_tidal_track,
build_discovery_wing_it_stub, add_activity_item).

Diff vs original after `deps.X` → global X normalization is **zero
differences** — 278 lines orig = 278 lines lifted, byte-identical body
(including all whitespace, comments, log strings).

Tests: 10 new under tests/discovery/test_discovery_spotify_public.py
covering cache hit short-circuit, dict-artist normalization, Spotify
tuple match (track/disc preservation), iTunes dict match path, Wing It
fallback, cancellation, completion phase update, activity feed entry,
top-level error handler, per-track error handling.

Full suite: 1118 passing (was 1108). Ruff clean.
This commit is contained in:
Broque Thomas 2026-04-28 13:13:41 -07:00
parent f48580694d
commit c5e06691e3
3 changed files with 670 additions and 276 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

337
core/discovery/spotify_public.py Normal file
View file

@ -0,0 +1,337 @@
"""Background worker for public Spotify-link playlist discovery.
`run_spotify_public_discovery_worker(url_hash, deps)` is the function the
spotify-public discovery start-endpoint submits to its executor to match
each public Spotify playlist track against Spotify (preferred) or iTunes
(fallback):
1. Pause enrichment workers (release shared resources).
2. For each track:
- Cancellation gate (state['cancelled']).
- Discovery cache lookup; cache hit short-circuits the search and
populates display fields from the cached match.
- SimpleNamespace duck-type → `_search_spotify_for_tidal_track`
(shared search helper, returns tuple for Spotify or dict for iTunes).
- On Spotify match: build `match_data` preserving track_number /
disc_number from raw API data, image extracted from album images
or track object fallback, release_date filled from track.release_date
when album dict is missing it.
- On iTunes match: dict result populated as `match_data`, source set
to discovery_source, image extracted from album images.
- Save matched result to discovery cache.
- On miss: Wing It stub stored as 'wing-it' status.
3. After all tracks: phase='discovered', activity feed entry.
4. On error: state['phase']='error' + status with error string.
5. Finally: resume enrichment workers.
Lifted verbatim from web_server.py. Wide dependency surface (Spotify and
iTunes clients, multiple metadata helpers, state dict, shared tidal
search helper) all injected via `SpotifyPublicDiscoveryDeps`.
"""
from __future__ import annotations
import logging
import time
import types
from dataclasses import dataclass
from typing import Any, Callable
logger = logging.getLogger(__name__)
@dataclass
class SpotifyPublicDiscoveryDeps:
"""Bundle of cross-cutting deps the Spotify Public discovery worker needs."""
spotify_public_discovery_states: dict
spotify_client: Any
pause_enrichment_workers: Callable[[str], dict]
resume_enrichment_workers: Callable[[dict, str], None]
get_active_discovery_source: Callable[[], str]
get_metadata_fallback_client: Callable[[], Any]
get_discovery_cache_key: Callable
get_database: Callable[[], Any]
validate_discovery_cache_artist: Callable
search_spotify_for_tidal_track: Callable
build_discovery_wing_it_stub: Callable
add_activity_item: Callable
def run_spotify_public_discovery_worker(url_hash, deps: SpotifyPublicDiscoveryDeps):
"""Background worker for Spotify Public discovery process (Spotify preferred, iTunes fallback)"""
_ew_state = {}
try:
_ew_state = deps.pause_enrichment_workers('Spotify Public discovery')
state = deps.spotify_public_discovery_states[url_hash]
playlist = state['playlist']
# Determine which provider to use — respect user's configured primary source
discovery_source = deps.get_active_discovery_source()
use_spotify = (discovery_source == 'spotify') and deps.spotify_client and deps.spotify_client.is_spotify_authenticated()
# Initialize fallback client if needed
itunes_client_instance = None
if not use_spotify:
itunes_client_instance = deps.get_metadata_fallback_client()
logger.info(f"Starting Spotify Public discovery for: {playlist['name']} (using {discovery_source.upper()})")
# Store discovery source in state for frontend
state['discovery_source'] = discovery_source
successful_discoveries = 0
tracks = playlist['tracks']
for i, sp_track in enumerate(tracks):
if state.get('cancelled', False):
break
try:
track_name = sp_track['name']
track_artists_raw = sp_track.get('artists', [])
# Normalize artists to list of strings
track_artists = []
for a in track_artists_raw:
if isinstance(a, dict):
track_artists.append(a.get('name', ''))
else:
track_artists.append(str(a))
track_id = sp_track.get('id', '')
track_album = sp_track.get('album', '')
if isinstance(track_album, dict):
track_album_name = track_album.get('name', '')
else:
track_album_name = track_album or ''
track_duration_ms = sp_track.get('duration_ms', 0)
logger.info(f"[{i+1}/{len(tracks)}] Searching {discovery_source.upper()}: {track_name} by {', '.join(track_artists)}")
# Check discovery cache first
cache_key = deps.get_discovery_cache_key(track_name, track_artists[0] if track_artists else '')
try:
cache_db = deps.get_database()
cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
if cached_match and deps.validate_discovery_cache_artist(track_artists[0] if track_artists else '', cached_match):
logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {track_name} by {', '.join(track_artists)}")
# Extract display-friendly artist string from cached match
cached_artists = cached_match.get('artists', [])
if cached_artists:
cached_artist_str = ', '.join(
a if isinstance(a, str) else a.get('name', '') for a in cached_artists
)
else:
cached_artist_str = ''
cached_album = cached_match.get('album', '')
if isinstance(cached_album, dict):
cached_album = cached_album.get('name', '')
result = {
'spotify_public_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album_name,
'duration_ms': track_duration_ms,
},
'spotify_data': cached_match,
'match_data': cached_match,
'status': 'Found',
'status_class': 'found',
'spotify_track': cached_match.get('name', ''),
'spotify_artist': cached_artist_str,
'spotify_album': cached_album,
'spotify_id': cached_match.get('id', ''),
'discovery_source': discovery_source,
'index': i
}
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
continue
except Exception as cache_err:
logger.error(f"Cache lookup error: {cache_err}")
# Create a SimpleNamespace duck-type object for _search_spotify_for_tidal_track
track_ns = types.SimpleNamespace(
id=track_id,
name=track_name,
artists=track_artists,
album=track_album_name,
duration_ms=track_duration_ms
)
# Use the search function with appropriate provider
track_result = deps.search_spotify_for_tidal_track(
track_ns,
use_spotify=use_spotify,
itunes_client=itunes_client_instance
)
# Create result entry
result = {
'spotify_public_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album_name,
'duration_ms': track_duration_ms,
},
'spotify_data': None,
'match_data': None,
'status': 'Not Found',
'status_class': 'not-found',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'discovery_source': discovery_source
}
match_confidence = 0.0
if use_spotify and isinstance(track_result, tuple):
# Spotify: Function returns (Track, raw_data, confidence)
track_obj, raw_track_data, match_confidence = track_result
album_obj = raw_track_data.get('album', {}) if raw_track_data else {}
# Ensure album has a name — fall back to track_obj.album if raw_data was missing
if isinstance(album_obj, dict) and not album_obj.get('name') and track_obj.album:
album_obj['name'] = track_obj.album
elif not album_obj and track_obj.album:
album_obj = {'name': track_obj.album}
# Ensure release_date is present (raw Spotify data has it, but fallback may not)
if isinstance(album_obj, dict) and not album_obj.get('release_date'):
album_obj['release_date'] = getattr(track_obj, 'release_date', '') or ''
# Extract image URL from album data or track object
_album_images = album_obj.get('images', []) if isinstance(album_obj, dict) else []
_image_url = _album_images[0].get('url', '') if _album_images else (getattr(track_obj, 'image_url', '') or '')
match_data = {
'id': track_obj.id,
'name': track_obj.name,
'artists': track_obj.artists,
'album': album_obj,
'duration_ms': track_obj.duration_ms,
'external_urls': track_obj.external_urls,
'image_url': _image_url,
'source': 'spotify'
}
# Preserve track_number/disc_number from raw Spotify API data
if raw_track_data and raw_track_data.get('track_number'):
match_data['track_number'] = raw_track_data['track_number']
if raw_track_data and raw_track_data.get('disc_number'):
match_data['disc_number'] = raw_track_data['disc_number']
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = track_obj.name
result['spotify_artist'] = ', '.join(track_obj.artists) if isinstance(track_obj.artists, list) else str(track_obj.artists)
result['spotify_album'] = album_obj.get('name', '') if isinstance(album_obj, dict) else str(album_obj)
result['spotify_id'] = track_obj.id
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
elif not use_spotify and track_result and isinstance(track_result, dict):
# Fallback: Function returns a dict with track data (includes 'confidence' key)
match_confidence = track_result.pop('confidence', 0.80)
match_data = track_result
match_data['source'] = discovery_source
# Extract image URL from album images
_fb_album = match_data.get('album', {})
_fb_images = _fb_album.get('images', []) if isinstance(_fb_album, dict) else []
if _fb_images and 'image_url' not in match_data:
match_data['image_url'] = _fb_images[0].get('url', '')
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = match_data.get('name', '')
itunes_artists = match_data.get('artists', [])
result['spotify_artist'] = ', '.join(a if isinstance(a, str) else a.get('name', '') for a in itunes_artists) if itunes_artists else ''
result['spotify_album'] = match_data.get('album', {}).get('name', '') if isinstance(match_data.get('album'), dict) else match_data.get('album', '')
result['spotify_id'] = match_data.get('id', '')
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
# Save to discovery cache if match found
if result['status_class'] == 'found' and result.get('match_data'):
try:
cache_db = deps.get_database()
cache_db.save_discovery_cache_match(
cache_key[0], cache_key[1], discovery_source, match_confidence,
result['match_data'], track_name,
track_artists[0] if track_artists else ''
)
logger.info(f"CACHE SAVED: {track_name} (confidence: {match_confidence:.3f})")
except Exception as cache_err:
logger.error(f"Cache save error: {cache_err}")
# Auto Wing It fallback for unmatched tracks
if result['status_class'] == 'not-found':
sp_t = result.get('spotify_public_track', {})
stub = deps.build_discovery_wing_it_stub(
sp_t.get('name', ''),
', '.join(sp_t.get('artists', [])),
sp_t.get('duration_ms', 0)
)
result['status'] = 'Wing It'
result['status_class'] = 'wing-it'
result['spotify_data'] = stub
result['match_data'] = stub
result['spotify_track'] = sp_t.get('name', '')
result['spotify_artist'] = ', '.join(sp_t.get('artists', []))
result['wing_it_fallback'] = True
result['confidence'] = 0
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['wing_it_count'] = state.get('wing_it_count', 0) + 1
result['index'] = i
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Add delay between requests
time.sleep(0.1)
except Exception as e:
logger.error(f"Error processing track {i+1}: {e}")
# Add error result
result = {
'spotify_public_track': {
'name': sp_track.get('name', 'Unknown'),
'artists': sp_track.get('artists', []),
},
'spotify_data': None,
'match_data': None,
'status': 'Error',
'status_class': 'error',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'error': str(e),
'discovery_source': discovery_source,
'index': i
}
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Mark as complete
state['phase'] = 'discovered'
state['status'] = 'discovered'
state['discovery_progress'] = 100
# Add activity for discovery completion
source_label = discovery_source.upper()
deps.add_activity_item("", f"Spotify Link Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
logger.info(f"Spotify Public discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
except Exception as e:
logger.error(f"Error in Spotify Public discovery worker: {e}")
if url_hash in deps.spotify_public_discovery_states:
deps.spotify_public_discovery_states[url_hash]['phase'] = 'error'
deps.spotify_public_discovery_states[url_hash]['status'] = f'error: {str(e)}'
finally:
deps.resume_enrichment_workers(_ew_state, 'Spotify Public discovery')

308
tests/discovery/test_discovery_spotify_public.py Normal file
View file

@ -0,0 +1,308 @@
"""Tests for core/discovery/spotify_public.py — Spotify Public link discovery."""
from __future__ import annotations
from dataclasses import dataclass
import pytest
from core.discovery import spotify_public as dsp
# ---------------------------------------------------------------------------
# Fakes
# ---------------------------------------------------------------------------
@dataclass
class _FakeTrackObj:
id: str = 'sp-1'
name: str = 'Found'
artists: list = None
album: str = 'Album X'
duration_ms: int = 200000
image_url: str = ''
external_urls: dict = None
release_date: str = '2024-01-01'
def __post_init__(self):
if self.artists is None:
self.artists = ['Found Artist']
if self.external_urls is None:
self.external_urls = {}
class _FakeSpotifyClient:
def __init__(self, authenticated=True):
self._authenticated = authenticated
def is_spotify_authenticated(self):
return self._authenticated
class _FakeDB:
def __init__(self, cache_match=None):
self._cache_match = cache_match
self.cache_saves = []
def get_discovery_cache_match(self, t, a, src):
return self._cache_match
def save_discovery_cache_match(self, t, a, src, conf, data, raw_t, raw_a):
self.cache_saves.append((t, a, src, conf))
def _build_deps(
*,
states=None,
spotify_auth=True,
discovery_source='spotify',
cache_match=None,
search_result=None,
activity_log=None,
):
activity_log = activity_log if activity_log is not None else []
db = _FakeDB(cache_match=cache_match)
spotify = _FakeSpotifyClient(authenticated=spotify_auth)
itunes = object()
deps = dsp.SpotifyPublicDiscoveryDeps(
spotify_public_discovery_states=states if states is not None else {},
spotify_client=spotify,
pause_enrichment_workers=lambda label: {'paused': True},
resume_enrichment_workers=lambda state, label: None,
get_active_discovery_source=lambda: discovery_source,
get_metadata_fallback_client=lambda: itunes,
get_discovery_cache_key=lambda title, artist: (title.lower(), artist.lower()),
get_database=lambda: db,
validate_discovery_cache_artist=lambda artist, m: True,
search_spotify_for_tidal_track=lambda track, use_spotify, itunes_client: search_result,
build_discovery_wing_it_stub=lambda title, artist, dur: {
'name': title, 'artists': [artist], 'duration_ms': dur, 'wing_it': True
},
add_activity_item=lambda *a, **kw: activity_log.append((a, kw)),
)
deps._db = db
deps._spotify = spotify
deps._activity_log = activity_log
return deps
def _seed_state(url_hash, states, *, tracks=None, cancelled=False):
states[url_hash] = {
'cancelled': cancelled,
'playlist': {'name': 'Public Playlist', 'tracks': tracks or []},
'spotify_matches': 0,
'discovery_results': [],
'discovery_progress': 0,
}
def _track(track_id='id1', name='Track', artists=None, album='Album', duration_ms=180000):
return {
'id': track_id,
'name': name,
'artists': artists or ['Artist'],
'album': album,
'duration_ms': duration_ms,
}
# ---------------------------------------------------------------------------
# Cache hit
# ---------------------------------------------------------------------------
def test_cache_hit_short_circuits():
"""Cache hit appends Found result without live search."""
states = {}
cached = {
'id': 'spt-c',
'name': 'Cached',
'artists': ['CA'],
'album': {'name': 'CAlb'},
}
_seed_state('h1', states, tracks=[_track()])
deps = _build_deps(states=states, cache_match=cached)
dsp.run_spotify_public_discovery_worker('h1', deps)
state = states['h1']
assert state['spotify_matches'] == 1
result = state['discovery_results'][0]
assert result['status'] == 'Found'
assert result['spotify_track'] == 'Cached'
def test_dict_artists_normalized_to_strings():
"""Artists with dict format normalize to plain strings during track unpack."""
states = {}
track = {
'id': 'd1',
'name': 'T',
'artists': [{'name': 'A1'}, 'A2'],
'album': {'name': 'X'},
'duration_ms': 1000,
}
_seed_state('h2', states, tracks=[track])
deps = _build_deps(states=states, search_result=None)
dsp.run_spotify_public_discovery_worker('h2', deps)
result = states['h2']['discovery_results'][0]
sp = result['spotify_public_track']
assert sp['artists'] == ['A1', 'A2']
# ---------------------------------------------------------------------------
# Spotify path (tuple result)
# ---------------------------------------------------------------------------
def test_spotify_match_preserves_track_disc_numbers():
"""Spotify result tuple → match_data preserves track_number & disc_number."""
states = {}
raw = {
'album': {'name': 'A', 'release_date': '2024-05-05', 'images': [{'url': 'http://i'}]},
'track_number': 3,
'disc_number': 1,
}
track_obj = _FakeTrackObj()
_seed_state('h3', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=(track_obj, raw, 0.92))
dsp.run_spotify_public_discovery_worker('h3', deps)
md = states['h3']['discovery_results'][0]['match_data']
assert md['track_number'] == 3
assert md['disc_number'] == 1
assert md['album']['release_date'] == '2024-05-05'
# ---------------------------------------------------------------------------
# iTunes path
# ---------------------------------------------------------------------------
def test_itunes_dict_result_path():
"""Non-spotify dict result → match_data with source set to discovery_source."""
states = {}
track_data = {
'id': 'it-1', 'name': 'iT', 'artists': ['iA'],
'album': {'name': 'iAlb', 'images': [{'url': 'http://it'}]},
'duration_ms': 200000, 'confidence': 0.85,
}
_seed_state('h4', states, tracks=[_track()])
deps = _build_deps(
states=states, spotify_auth=False, discovery_source='itunes',
search_result=track_data,
)
dsp.run_spotify_public_discovery_worker('h4', deps)
result = states['h4']['discovery_results'][0]
assert result['confidence'] == 0.85
assert result['match_data']['source'] == 'itunes'
assert result['match_data']['image_url'] == 'http://it'
# ---------------------------------------------------------------------------
# Wing It fallback
# ---------------------------------------------------------------------------
def test_no_match_wing_it_fallback():
"""No match → Wing It stub stored, status='Wing It'."""
states = {}
_seed_state('h5', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=None)
dsp.run_spotify_public_discovery_worker('h5', deps)
state = states['h5']
assert state.get('wing_it_count') == 1
result = state['discovery_results'][0]
assert result['status'] == 'Wing It'
assert result['wing_it_fallback'] is True
# ---------------------------------------------------------------------------
# Cancellation
# ---------------------------------------------------------------------------
def test_cancellation_breaks_loop():
"""state['cancelled']=True bails on first iteration."""
states = {}
_seed_state('h6', states, tracks=[_track(), _track('id2')], cancelled=True)
deps = _build_deps(states=states)
dsp.run_spotify_public_discovery_worker('h6', deps)
assert states['h6']['discovery_results'] == []
# ---------------------------------------------------------------------------
# Completion
# ---------------------------------------------------------------------------
def test_completion_marks_phase_discovered():
"""Completion → phase + status = 'discovered', progress=100."""
states = {}
_seed_state('h7', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=None)
dsp.run_spotify_public_discovery_worker('h7', deps)
assert states['h7']['phase'] == 'discovered'
assert states['h7']['status'] == 'discovered'
assert states['h7']['discovery_progress'] == 100
def test_activity_feed_logged():
"""Completion logs activity feed entry with 'Spotify Link Discovery Complete'."""
states = {}
_seed_state('h8', states, tracks=[_track()])
deps = _build_deps(states=states, search_result=None)
dsp.run_spotify_public_discovery_worker('h8', deps)
args, _ = deps._activity_log[0]
title = args[1]
assert 'Spotify Link Discovery Complete' in title
# ---------------------------------------------------------------------------
# Error path
# ---------------------------------------------------------------------------
def test_top_level_error_marks_state_error():
"""Exception in main try → phase='error', status with error string."""
states = {}
_seed_state('herr', states, tracks=[_track()])
deps = _build_deps(states=states)
deps.get_active_discovery_source = lambda: (_ for _ in ()).throw(RuntimeError("boom"))
dsp.run_spotify_public_discovery_worker('herr', deps)
assert states['herr']['phase'] == 'error'
assert 'boom' in states['herr']['status']
def test_per_track_error_appends_error_result():
"""Per-track exception → 'Error' result entry, loop continues."""
states = {}
tracks = [_track('a'), _track('b')]
_seed_state('h9', states, tracks=tracks)
deps = _build_deps(states=states)
call_count = [0]
def search_side_effect(track, use_spotify, itunes_client):
call_count[0] += 1
if call_count[0] == 1:
raise RuntimeError("track boom")
return None
deps.search_spotify_for_tidal_track = search_side_effect
dsp.run_spotify_public_discovery_worker('h9', deps)
state = states['h9']
assert len(state['discovery_results']) == 2
assert state['discovery_results'][0]['status'] == 'Error'
assert state['discovery_results'][1]['status'] == 'Wing It'

301
web_server.py
View file

@ -27724,284 +27724,33 @@ def update_spotify_public_playlist_phase(url_hash):
return jsonify({"error": str(e)}), 500
# Spotify Public discovery worker logic lives in core/discovery/spotify_public.py.
from core.discovery import spotify_public as _discovery_spotify_public
def _build_spotify_public_discovery_deps():
"""Build the SpotifyPublicDiscoveryDeps bundle from web_server.py globals on each call."""
return _discovery_spotify_public.SpotifyPublicDiscoveryDeps(
spotify_public_discovery_states=spotify_public_discovery_states,
spotify_client=spotify_client,
pause_enrichment_workers=_pause_enrichment_workers,
resume_enrichment_workers=_resume_enrichment_workers,
get_active_discovery_source=_get_active_discovery_source,
get_metadata_fallback_client=_get_metadata_fallback_client,
get_discovery_cache_key=_get_discovery_cache_key,
get_database=get_database,
validate_discovery_cache_artist=_validate_discovery_cache_artist,
search_spotify_for_tidal_track=_search_spotify_for_tidal_track,
build_discovery_wing_it_stub=_build_discovery_wing_it_stub,
add_activity_item=add_activity_item,
)
def _run_spotify_public_discovery_worker(url_hash):
"""Background worker for Spotify Public discovery process (Spotify preferred, iTunes fallback)"""
_ew_state = {}
try:
_ew_state = _pause_enrichment_workers('Spotify Public discovery')
state = spotify_public_discovery_states[url_hash]
playlist = state['playlist']
return _discovery_spotify_public.run_spotify_public_discovery_worker(
url_hash, _build_spotify_public_discovery_deps()
)
# Determine which provider to use — respect user's configured primary source
discovery_source = _get_active_discovery_source()
use_spotify = (discovery_source == 'spotify') and spotify_client and spotify_client.is_spotify_authenticated()
# Initialize fallback client if needed
itunes_client_instance = None
if not use_spotify:
itunes_client_instance = _get_metadata_fallback_client()
logger.info(f"Starting Spotify Public discovery for: {playlist['name']} (using {discovery_source.upper()})")
# Store discovery source in state for frontend
state['discovery_source'] = discovery_source
successful_discoveries = 0
tracks = playlist['tracks']
for i, sp_track in enumerate(tracks):
if state.get('cancelled', False):
break
try:
track_name = sp_track['name']
track_artists_raw = sp_track.get('artists', [])
# Normalize artists to list of strings
track_artists = []
for a in track_artists_raw:
if isinstance(a, dict):
track_artists.append(a.get('name', ''))
else:
track_artists.append(str(a))
track_id = sp_track.get('id', '')
track_album = sp_track.get('album', '')
if isinstance(track_album, dict):
track_album_name = track_album.get('name', '')
else:
track_album_name = track_album or ''
track_duration_ms = sp_track.get('duration_ms', 0)
logger.info(f"[{i+1}/{len(tracks)}] Searching {discovery_source.upper()}: {track_name} by {', '.join(track_artists)}")
# Check discovery cache first
cache_key = _get_discovery_cache_key(track_name, track_artists[0] if track_artists else '')
try:
cache_db = get_database()
cached_match = cache_db.get_discovery_cache_match(cache_key[0], cache_key[1], discovery_source)
if cached_match and _validate_discovery_cache_artist(track_artists[0] if track_artists else '', cached_match):
logger.debug(f"CACHE HIT [{i+1}/{len(tracks)}]: {track_name} by {', '.join(track_artists)}")
# Extract display-friendly artist string from cached match
cached_artists = cached_match.get('artists', [])
if cached_artists:
cached_artist_str = ', '.join(
a if isinstance(a, str) else a.get('name', '') for a in cached_artists
)
else:
cached_artist_str = ''
cached_album = cached_match.get('album', '')
if isinstance(cached_album, dict):
cached_album = cached_album.get('name', '')
result = {
'spotify_public_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album_name,
'duration_ms': track_duration_ms,
},
'spotify_data': cached_match,
'match_data': cached_match,
'status': 'Found',
'status_class': 'found',
'spotify_track': cached_match.get('name', ''),
'spotify_artist': cached_artist_str,
'spotify_album': cached_album,
'spotify_id': cached_match.get('id', ''),
'discovery_source': discovery_source,
'index': i
}
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
continue
except Exception as cache_err:
logger.error(f"Cache lookup error: {cache_err}")
# Create a SimpleNamespace duck-type object for _search_spotify_for_tidal_track
track_ns = types.SimpleNamespace(
id=track_id,
name=track_name,
artists=track_artists,
album=track_album_name,
duration_ms=track_duration_ms
)
# Use the search function with appropriate provider
track_result = _search_spotify_for_tidal_track(
track_ns,
use_spotify=use_spotify,
itunes_client=itunes_client_instance
)
# Create result entry
result = {
'spotify_public_track': {
'id': track_id,
'name': track_name,
'artists': track_artists or [],
'album': track_album_name,
'duration_ms': track_duration_ms,
},
'spotify_data': None,
'match_data': None,
'status': 'Not Found',
'status_class': 'not-found',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'discovery_source': discovery_source
}
match_confidence = 0.0
if use_spotify and isinstance(track_result, tuple):
# Spotify: Function returns (Track, raw_data, confidence)
track_obj, raw_track_data, match_confidence = track_result
album_obj = raw_track_data.get('album', {}) if raw_track_data else {}
# Ensure album has a name — fall back to track_obj.album if raw_data was missing
if isinstance(album_obj, dict) and not album_obj.get('name') and track_obj.album:
album_obj['name'] = track_obj.album
elif not album_obj and track_obj.album:
album_obj = {'name': track_obj.album}
# Ensure release_date is present (raw Spotify data has it, but fallback may not)
if isinstance(album_obj, dict) and not album_obj.get('release_date'):
album_obj['release_date'] = getattr(track_obj, 'release_date', '') or ''
# Extract image URL from album data or track object
_album_images = album_obj.get('images', []) if isinstance(album_obj, dict) else []
_image_url = _album_images[0].get('url', '') if _album_images else (getattr(track_obj, 'image_url', '') or '')
match_data = {
'id': track_obj.id,
'name': track_obj.name,
'artists': track_obj.artists,
'album': album_obj,
'duration_ms': track_obj.duration_ms,
'external_urls': track_obj.external_urls,
'image_url': _image_url,
'source': 'spotify'
}
# Preserve track_number/disc_number from raw Spotify API data
if raw_track_data and raw_track_data.get('track_number'):
match_data['track_number'] = raw_track_data['track_number']
if raw_track_data and raw_track_data.get('disc_number'):
match_data['disc_number'] = raw_track_data['disc_number']
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = track_obj.name
result['spotify_artist'] = ', '.join(track_obj.artists) if isinstance(track_obj.artists, list) else str(track_obj.artists)
result['spotify_album'] = album_obj.get('name', '') if isinstance(album_obj, dict) else str(album_obj)
result['spotify_id'] = track_obj.id
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
elif not use_spotify and track_result and isinstance(track_result, dict):
# Fallback: Function returns a dict with track data (includes 'confidence' key)
match_confidence = track_result.pop('confidence', 0.80)
match_data = track_result
match_data['source'] = discovery_source
# Extract image URL from album images
_fb_album = match_data.get('album', {})
_fb_images = _fb_album.get('images', []) if isinstance(_fb_album, dict) else []
if _fb_images and 'image_url' not in match_data:
match_data['image_url'] = _fb_images[0].get('url', '')
result['spotify_data'] = match_data
result['match_data'] = match_data
result['status'] = 'Found'
result['status_class'] = 'found'
result['spotify_track'] = match_data.get('name', '')
itunes_artists = match_data.get('artists', [])
result['spotify_artist'] = ', '.join(a if isinstance(a, str) else a.get('name', '') for a in itunes_artists) if itunes_artists else ''
result['spotify_album'] = match_data.get('album', {}).get('name', '') if isinstance(match_data.get('album'), dict) else match_data.get('album', '')
result['spotify_id'] = match_data.get('id', '')
result['confidence'] = match_confidence
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
# Save to discovery cache if match found
if result['status_class'] == 'found' and result.get('match_data'):
try:
cache_db = get_database()
cache_db.save_discovery_cache_match(
cache_key[0], cache_key[1], discovery_source, match_confidence,
result['match_data'], track_name,
track_artists[0] if track_artists else ''
)
logger.info(f"CACHE SAVED: {track_name} (confidence: {match_confidence:.3f})")
except Exception as cache_err:
logger.error(f"Cache save error: {cache_err}")
# Auto Wing It fallback for unmatched tracks
if result['status_class'] == 'not-found':
sp_t = result.get('spotify_public_track', {})
stub = _build_discovery_wing_it_stub(
sp_t.get('name', ''),
', '.join(sp_t.get('artists', [])),
sp_t.get('duration_ms', 0)
)
result['status'] = 'Wing It'
result['status_class'] = 'wing-it'
result['spotify_data'] = stub
result['match_data'] = stub
result['spotify_track'] = sp_t.get('name', '')
result['spotify_artist'] = ', '.join(sp_t.get('artists', []))
result['wing_it_fallback'] = True
result['confidence'] = 0
successful_discoveries += 1
state['spotify_matches'] = successful_discoveries
state['wing_it_count'] = state.get('wing_it_count', 0) + 1
result['index'] = i
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Add delay between requests
time.sleep(0.1)
except Exception as e:
logger.error(f"Error processing track {i+1}: {e}")
# Add error result
result = {
'spotify_public_track': {
'name': sp_track.get('name', 'Unknown'),
'artists': sp_track.get('artists', []),
},
'spotify_data': None,
'match_data': None,
'status': 'Error',
'status_class': 'error',
'spotify_track': '',
'spotify_artist': '',
'spotify_album': '',
'error': str(e),
'discovery_source': discovery_source,
'index': i
}
state['discovery_results'].append(result)
state['discovery_progress'] = int(((i + 1) / len(tracks)) * 100)
# Mark as complete
state['phase'] = 'discovered'
state['status'] = 'discovered'
state['discovery_progress'] = 100
# Add activity for discovery completion
source_label = discovery_source.upper()
add_activity_item("", f"Spotify Link Discovery Complete ({source_label})", f"'{playlist['name']}' - {successful_discoveries}/{len(tracks)} tracks found", "Now")
logger.info(f"Spotify Public discovery complete ({source_label}): {successful_discoveries}/{len(tracks)} tracks found")
except Exception as e:
logger.error(f"Error in Spotify Public discovery worker: {e}")
if url_hash in spotify_public_discovery_states:
spotify_public_discovery_states[url_hash]['phase'] = 'error'
spotify_public_discovery_states[url_hash]['status'] = f'error: {str(e)}'
finally:
_resume_enrichment_workers(_ew_state, 'Spotify Public discovery')
def convert_spotify_public_results_to_spotify_tracks(discovery_results):