Commit graph

443 commits

Author SHA1 Message Date
Broque Thomas
cf5da04439 Roll LB Weekly / Top series into single rolling mirrors (Phase 1c.2.1)
ListenBrainz publishes "Weekly Jams for X" / "Weekly Exploration
for X" with a fresh MBID every week, and "Top Discoveries of YYYY
for X" / "Top Missed Recordings of YYYY for X" with a fresh MBID
every year. Auto-mirroring those per-period yielded one mirrored-
playlist row per week/year — useless for Auto-Sync schedules
because the underlying LB playlist never updates, only a brand new
playlist replaces it. The user accumulates 100+ dead Weekly Jams
rows per year if they discover regularly.

This commit collapses each family into a single ROLLING mirror
keyed by a synthetic ``source_playlist_id`` (e.g.
``lb_weekly_jams_Nezreka``). Each new period UPSERTs into the same
row, so the user gets one stable Auto-Sync schedule per series
that automatically picks up the latest period's tracks on every
refresh. Non-series LB playlists (user-created, collaborative,
Last.fm radios for a specific seed) continue to mirror under
their per-playlist MBID as before. Per-period LB playlists are
still visible + usable on the LB Sync tab — only the mirror layer
collapses.

- ``core/playlists/lb_series.py`` (new) — series-detect helper
  with regex patterns + canonical-name + LIKE-pattern template
  for each known LB family. Exposes
  ``detect_series(title)``, ``is_series_synthetic_id(id)``, and
  ``list_series_synthetic_ids()`` so both the JS auto-mirror hook
  and the LB adapter can speak the same language.
- ``GET /api/listenbrainz/series-detect?title=...`` — thin HTTP
  shim around ``detect_series`` so the auto-mirror JS doesn't
  duplicate the regex.
- ``ListenBrainzPlaylistSource.get_playlist`` now recognizes
  synthetic series ids — it queries the LB cache for the newest
  cache row whose title matches the series' LIKE pattern and
  resolves to that row's MBID before fetching tracks. The mirror's
  meta keeps the synthetic id so refreshes always re-resolve to
  the latest period.
- ``_mirrorListenBrainzAfterDiscovery`` (sync-services.js) calls
  the new detect endpoint when discovery completes — if a match
  comes back it swaps the per-period MBID for the synthetic id +
  the canonical name. Existing Last.fm radio routing logic stays
  intact (Last.fm radios aren't a series).
- ``ListenBrainzManager._cleanup_per_period_series_mirrors`` —
  one-shot consolidation sweeper runs in ``_cleanup_old_playlists``
  + deletes any legacy per-period mirror rows so the consolidated
  rolling mirror is the only one left. Idempotent — only matches
  per-period titles ("Weekly Jams for ..., week of ...") and never
  the canonical rolling-mirror titles ("ListenBrainz Weekly
  Jams").
- 11 new tests pin the detector + synthetic-id helpers; 236 total
  across adapter + automation + lb-series suites green.
2026-05-26 15:49:49 -07:00
Broque Thomas
246503066b Fold provider-matching into PlaylistSource contract (Phase 1b)
Adds ``discover_tracks(tracks) -> List[NormalizedTrack]`` to the
PlaylistSource interface. Sources whose tracks already carry
provider IDs (Spotify, Tidal, Qobuz, YouTube, Deezer, Spotify
public, iTunes link, SoulSync Discovery) inherit a no-op default;
ListenBrainz + Last.fm override to run the matching engine.

This closes the last gap before LB / Last.fm / SoulSync Discovery
can land as Sync-page mirror sources: the refresh handler now
calls ``source.discover_tracks(...)`` whenever a source returns
tracks with ``needs_discovery=True``, so mirrored LB rows arrive
already discovered + ready for the sync pipeline. Previously, LB
playlists ran through a separate state-machine worker tied to the
Discover-page UI, with results stored in ``discovery_cache``
instead of ``mirrored_playlist_tracks.extra_data``.

Changes:

- ``core/playlists/sources/base.py`` — PlaylistSource switches from
  Protocol to ABC so a concrete default for ``discover_tracks``
  can live on the base class. The four real-work methods stay
  ``@abstractmethod``; instantiating an adapter that forgets one
  fails loudly at construction.
- ``core/discovery/matching.py`` (new) — pure ``match_mb_tracks``
  helper that runs Strategy-1-only matching-engine queries against
  Spotify (primary) or iTunes (fallback). No state machine, no
  discovery-cache writes, no wing-it stub — that richer flow stays
  in ``core/discovery/listenbrainz.py`` for the Discover-page UI.
- ``ListenBrainzPlaylistSource`` + ``LastFMPlaylistSource`` take
  an optional ``discover_callable`` constructor arg. Last.fm reuses
  the LB implementation since the track shape is identical.
- ``bootstrap.build_playlist_source_registry`` accepts a
  ``discover_callable`` kwarg and wires it into LB + Last.fm
  adapters.
- ``web_server.py`` boot constructs the discovery callable from the
  existing matching engine + ``_discovery_score_candidates`` +
  Spotify / iTunes clients, passes through to the registry.
- ``refresh_mirrored.py`` adds a small ``_maybe_discover`` helper
  that calls ``source.discover_tracks(...)`` between fetch and
  ``to_mirror_track_dict`` projection — only fires when at least
  one track has ``needs_discovery=True``, so the normal Spotify /
  Tidal / etc. refresh path stays a zero-cost pass-through.

Tests:

- 5 new adapter tests: default no-op pass-through, LB discovery
  with mixed matches/misses, LB no-callable fallback, Last.fm
  shares the LB implementation, mirror-dict spotify_hint emit.
- 1 new automation test: end-to-end LB refresh with a stub
  discover_callable proves the matched_data lands in
  ``mirror_playlist_tracks.extra_data`` after the registry
  refresh + discover hop.

225 tests across adapter + automation suites green.
2026-05-26 13:07:01 -07:00
Broque Thomas
8c41b05fe8 Refactor refresh_mirrored to use unified PlaylistSource registry
Phase 1a of the Discover-to-Sync unification. The mirrored-playlist
refresh handler used to branch per-source through a ~190-line
if/elif chain (Spotify, Spotify public, Deezer, Tidal, YouTube).
Each branch hand-built its own ``extra_data`` JSON for the matched-
data block. With every new source we considered for Sync-page mirror
support (ListenBrainz, Last.fm radio, SoulSync Discovery, iTunes
link), that chain would have grown a new elif.

This commit lifts the per-source logic into the existing adapter
layer and collapses the dispatch to a registry lookup:

- ``core/playlists/sources/deezer.py`` — new adapter so the registry
  covers every source the refresh handler previously branched on.
- ``core/playlists/sources/bootstrap.py`` — single helper that builds
  a populated registry from injected getter callables. Both
  ``web_server.py`` boot and the automation test fixtures call it,
  so the two construction paths can't drift.
- ``core/playlists/sources/base.py`` — ``to_mirror_track_dict``
  projection helper centralises the NormalizedTrack → DB-row
  conversion (including the discovered/matched_data and
  spotify_hint extra_data shapes the downstream sync + wishlist
  consumers already expect).
- Spotify adapter now populates ``extra['discovered']`` + an
  ``extra['matched_data']`` block when fetching via the authed API,
  so Spotify mirrors keep landing pre-discovered (matches the
  pre-refactor contract pinned by
  ``test_spotify_refresh_writes_to_db``).
- Spotify-public adapter populates ``extra['spotify_hint']`` so the
  discovery worker can skip its search step and jump straight to
  enrichment for the known track ID.
- All artist-name fields now project to first-artist-only across
  every adapter — matches the pre-refactor mirror_playlist DB shape
  (``t.artists[0]``).

``refresh_mirrored.py`` shrinks ~190 → ~80 lines and keeps:

- the file/beatport unrefreshable-source filter,
- URL extraction from ``description`` via ``require_refresh_url``
  for spotify_public + youtube,
- the Spotify-public → authed-Spotify fallback when the user is
  signed in (handler-level branch, not in any adapter),
- the Tidal-not-authenticated soft-skip log (skip, not error),
- existing-extra_data preservation across refreshes,
- the ``playlist_changed`` automation event emit on track-set delta.

Test scaffolding:

- ``_build_deps`` in ``tests/automation/test_handlers_playlist.py``
  now builds a default registry from the passed clients via
  ``build_playlist_source_registry``, so existing refresh tests
  exercise the same path without per-test changes. New tests cover
  Tidal-not-authed soft-skip, Deezer refresh writes plain tracks,
  YouTube refresh reads URL from description, and Spotify-public
  uses authed Spotify when signed in.
- 4 new adapter tests for Deezer projection +
  ``to_mirror_track_dict`` (minimal track, Spotify matched_data,
  Spotify-public spotify_hint).
- ``playlist_source_registry`` field on ``AutomationDeps`` defaults
  to ``None`` so the other 5 automation test files (which don't
  exercise refresh_mirrored) keep working unchanged.

220 tests across automation + adapter suites green.
2026-05-26 12:52:39 -07:00
Broque Thomas
c5898c3b9b Add unified PlaylistSource adapter layer (Phase 0)
Groundwork for unifying Discover-page playlists (ListenBrainz, Last.fm
radio, SoulSync Discovery) with Sync-page playlists (Spotify, Tidal,
Qobuz, YouTube, Spotify public, iTunes link). All nine sources now
expose the same `PlaylistSource` Protocol so callers stop having to
branch per-source.

This commit only adds the abstraction — no dispatch sites collapse to
the registry yet, no DB or UI changes. Adapters wrap existing clients
via injected getter callables to avoid eager imports of web_server.py
globals.

- core/playlists/sources/base.py — PlaylistMeta, NormalizedTrack,
  PlaylistDetail dataclasses + PlaylistSource Protocol with
  supports_listing / supports_refresh / requires_auth capability
  flags. needs_discovery flag on NormalizedTrack marks tracks that
  carry raw MB metadata (LB, Last.fm) vs tracks already matched to a
  provider ID (everything else).
- core/playlists/sources/registry.py — thread-safe lazy-factory
  registry with instance caching + re-register invalidation.
- nine adapters in core/playlists/sources/ wrapping SpotifyClient,
  TidalClient, QobuzClient, spotify_public_scraper, the YouTube +
  iTunes-link parsers (via injected callables), ListenBrainzManager,
  Last.fm radio rows in the ListenBrainz cache, and
  PersonalizedPlaylistManager.
- tests/test_playlist_sources_adapters.py — 18 tests covering each
  adapter's field projection with fake backing clients, plus
  registry lazy-construct + cache + re-register invalidation.

Phase 1 will collapse refresh_mirrored.py's per-source if/elif chain
to a registry lookup and surface ListenBrainz as a Sync-page tab.
2026-05-26 12:22:09 -07:00
Broque Thomas
b5755d6307 Trust user manual picks past AcoustID verification (#701)
When a task failed AcoustID verification and got quarantined, opening
the candidates modal and manually picking a different file would just
re-quarantine it. The manual-pick path through
`_attempt_download_with_candidates` ran full post-processing with no
quarantine bypass — so if the alternate file disagreed with AcoustID's
stored metadata too (common for live versions, remasters, regional
title differences, fingerprint coverage gaps) the file landed right
back in quarantine. User got stuck in the loop.

The Approve button on quarantined rows already handles the "I want
this exact file" case via `_skip_quarantine_check='all'`. The
candidates modal handles the "I want a different file" case — same
user intent, opposite direction, but the bypass plumbing didn't carry
through.

`/api/downloads/task/<id>/download-candidate` already sets
`task['_user_manual_pick'] = True`. `attempt_download_with_candidates`
now reads that flag under tasks_lock alongside `used_sources` and,
when set, injects `_skip_quarantine_check='acoustid'` plus
`_user_manual_pick=True` into the stored `matched_downloads_context`
entry. The acoustid-only scope is deliberate: integrity + bit-depth
gates still run because those check the new file's actual condition
(corruption, sample rate) rather than its identity — only the
metadata-mismatch gate is the user-override case.

Auto-search picks (the normal task-worker path) leave the flag unset
and continue to run full AcoustID verification, preserving the
existing safety net for non-user-initiated downloads.

Tests:
- positive: manual-pick task → stored context has
  `_skip_quarantine_check='acoustid'` and `_user_manual_pick=True`
- negative: auto-search task → stored context has neither key,
  AcoustID still runs as before

Full suite 3976 pass.
2026-05-26 09:56:49 -07:00
Broque Thomas
85ba93f16f Fix album-bundle staging match + wishlist provenance (#700, #698)
Root cause (#700): the Soulseek album-bundle path downloads whole
releases into a private staging dir, then per-track workers claim
those files via the staging-match shortcut. When slskd files arrived
without ID3 tags (common for FLAC rips), the staging cache fell back
to the filename stem as the title — and stems shaped like
"Artist - Album - 03 - Title" could not clear the 0.80 title-
similarity threshold against the clean Spotify track name. Every
track in the album went not_found, the batch ended "failed" in the
Downloads UI with an empty queue, and the bundle-downloaded files
just sat unused in staging.

Fix: in _staging_title_variants, add a trailing-title variant by
extracting the segments after a bare track-number block (e.g. "03")
between " - " delimiters. Conservative — only fires when a clear
digit segment is present, so real song titles with dashes like
"Hold Me - Live" are left intact. Generated as an additional variant
alongside the existing raw/compacted/feat-stripped/bonus-stripped
forms, so behavior on already-matching files is unchanged.

Downstream (#698): the album-bundle staging miss pushed every failed
track to the wishlist labelled as a playlist track, and a couple of
fallback paths in ensure_wishlist_track_format and the slskd-result
reconstruction hardcoded album_type='single' / total_tracks=1 on the
stored album dict. On wishlist requeue the path builder saw
album_type='single' and routed the download through single_path,
dumping the file in the Singles tree even though it belonged to an
album. (Running Reorganize would fix it because the DB album linkage
was still correct, but the file landed in the wrong place first.)

Fixes:
- new resolve_wishlist_source_type_for_batch() returns 'album' for
  is_album_download batches; wishlist_failed.py now calls it instead
  of hardcoding 'playlist'
- build_wishlist_source_context() threads album_context /
  artist_context / is_album_download from the batch into the wishlist
  row so future requeue logic has authoritative routing data
- the non-dict-album fallback in ensure_wishlist_track_format and
  the slskd-result reconstruction default album_type='album' (and
  total_tracks=0 = unknown) instead of lying with 'single'/1; the
  existing setdefault chain handles dict-shaped album data unchanged

Tests:
- 2 staging-match tests pin the new tail-extraction behavior against
  a realistic untagged slskd stem, plus a negative test that confirms
  a dash-in-title without a digit segment still does NOT extract a
  variant
- 2 payload tests pin the album_type='album' default for both
  fallback paths
- 4 processing tests pin resolve_wishlist_source_type_for_batch()
  and the album-context threading in build_wishlist_source_context()

3974 pass; no behavioural change on already-working flows.
2026-05-26 07:12:49 -07:00
Broque Thomas
96e6ba0ed7 Preserve Navidrome album cover art
Expose Navidrome album coverArt as a Subsonic getCoverArt thumbnail so library refreshes keep a real album-art URL. Preserve existing album thumb_url when an incoming server album has no thumbnail, preventing manual or server-corrected covers from being cleared and later replaced by loose missing-cover searches. Add regression tests for Navidrome album thumbnails and DB thumb preservation.
2026-05-25 19:51:38 -07:00
Broque Thomas
dad1b5109e Add _build_library_tag_db_data helper
Extract repeated DB tag payload construction into a new _build_library_tag_db_data(track_data, album_genres) helper and replace in multiple endpoints. The helper builds the metadata dict (title, artist_name, track_artist, album_title, year, genres, track/disc numbers, bpm, track_count, thumb_url) and populates artists_list by splitting track_artist on ';'. Added tests (tests/test_library_tag_payload.py) to verify artists_list creation, genre propagation, thumb_url selection, and fallback behavior when track_artist is missing. This reduces duplication and ensures consistent tag payloads across tag-preview, batch preview, and tag-writing flows.
2026-05-25 18:48:45 -07:00
Broque Thomas
a65ba7e6a3 Add node:test contract for auto-sync.js helpers
Cin review: no JS tests covered the autoSync* helpers, so the
timezone fix shipped without a regression test in the layer where the
bug actually lived. New `tests/static/test_auto_sync.mjs` runs under
`node --test` (built-in runner, no extra deps) and pins:

- `autoSyncTriggerForHours` / `autoSyncHoursFromTrigger` round-trip
  for 1h, 4h, 12h, 24h, 48h, 168h. Catches off-by-one in the day vs
  hour conversion that backs the schedule board's drag-drop.
- `autoSyncBucketLabel` / `autoSyncIntervalLabel` formatting +
  pluralization.
- `autoSyncSourceLabel` known + unknown + falsy.
- Predicates (`autoSyncCanSchedulePlaylist`,
  `autoSyncIsPipelineAutomation`, `autoSyncPlaylistIdFromAutomation`,
  `autoSyncIsScheduleOwned`) including the `owned_by`-flag /
  legacy-name-prefix split from the previous commit.
- `buildAutoSyncScheduleState` partitions board-owned schedules from
  custom pipelines correctly.
- `autoSyncNextRunLabel` parses the naive UTC timestamp as UTC, not
  local — exactly the regression that took an hour to diagnose this
  session. Includes a past-time check ("due now") and a multi-day
  case.
- `getMirroredSourceRef` source_ref/description-URL/source_playlist_id
  resolution order.

Cross-realm note: vm-sandbox return values fail `deepStrictEqual`
against host-realm objects even when shape matches, so a small
`deepShapeEqual` helper round-trips through JSON for structural
comparison. The `_autoParseUTC` stub mirrors the real implementation
in stats-automations.js so the timezone test exercises both files end
to end.

`tests/test_auto_sync_js.py` is the pytest shim — shells out to
`node --test` and surfaces failures inline. Skips cleanly when node
isn't on PATH or is older than 22, matching the existing
discover-section-controller test pattern.

Also updated SPLIT_MODULES in tests/test_script_split_integrity.py to
include the new auto-sync.js — the onclick-coverage check was failing
because `openAutoSyncScheduleModal` (referenced from index.html via
the Sync page button) now lives in a module the integrity scanner
wasn't searching.

39 new JS test cases, all green via `node --test` and via the pytest
wrapper.
2026-05-25 00:01:34 -07:00
Broque Thomas
9b086c5a65 Add owned_by column for Auto-Sync schedule ownership
The Auto-Sync schedule board was detecting its own automations by
checking `group_name === 'Playlist Auto-Sync' || name.startsWith('Auto-Sync:')`.
That's fragile — renaming the row from the Automations page silently
hands ownership back to the read-only Automation Pipelines tab and the
board stops managing it.

This commit replaces the string convention with an explicit
`automations.owned_by` TEXT column:

- Migration `_add_automation_owned_by_column` adds the column and
  backfills `'auto_sync'` for existing rows that match the legacy
  `group_name`/`name`-prefix pattern, so users running the migration
  don't lose their schedules.
- `database.create_automation` and `database.update_automation` accept
  `owned_by` (the latter via its `allowed` kwarg set).
- `core/automation/api.py` forwards `owned_by` on both POST and PUT.
  Missing field is left as None, preserving today's behavior for every
  caller that doesn't opt in.
- The Auto-Sync schedule board posts `owned_by: 'auto_sync'` and the
  detection helper now prefers that signal, falling back to the legacy
  name/group convention so any hand-rolled rows still show up.

Tests: three new cases in `tests/automation/test_automation_api.py`
covering create-with-owned-by, create-without (defaults to None), and
update set/clear. The fake DB grew the matching kwarg.
2026-05-24 23:40:22 -07:00
Broque Thomas
feb6778af4 Address Cin review: extract helpers, indexed pool fetch, tidy nits
Three changes folded into one perf+cleanup pass:

1. Indexed fast path for the per-artist pool fetch. The previous
   `search_tracks(artist=name)` call hit
   `unidecode_lower(artists.name) LIKE ?`, a function-in-WHERE that
   can't use `idx_artists_name`. New `MusicDatabase.get_artist_tracks_indexed`
   does a two-step lookup: exact-name match (indexed) plus a
   case-insensitive fallback, then `tracks WHERE artist_id IN (...)`
   via `idx_tracks_artist_id`. Drops per-artist fetch from seconds to
   milliseconds for the common case. The sync helper falls back to
   the old LIKE-based `search_tracks` only when the indexed lookup
   finds nothing, preserving diacritic recall and `tracks.track_artist`
   feature-artist matches with zero regression.

2. Public text-normalization helper. Lifted the body of
   `MusicDatabase._normalize_for_comparison` into
   `core/text/normalize.py:normalize_for_comparison` so callers outside
   the database layer (matching engine, sync pool, future import-side
   comparisons) don't reach across the module boundary into a
   leading-underscore "private" method. The DB method now delegates,
   so existing internal call sites stay untouched. Sync's lazy pool
   now imports the public helper.

3. Artist-name walker extracted. `_artist_name` at module level in
   `services/sync_service.py` replaces two near-identical inline
   str-or-dict-or-fallback walkers (one in `sync_playlist`, one in
   `_find_track_in_media_server`). Returns `''` for None instead of
   the literal string `'None'`.

Plus three small tidies from the same review:

- `_POOL_FETCH_LIMIT = 10000` constant in place of the literal at the
  pool-fetch call site.
- Trimmed the verbose docstring + comment block on the pool helper.
- Set-intersection predicate for the trigger-shape reset in
  `core/automation/api.py` instead of a two-line `or` chain.

Also removed the duplicate `_get_active_media_client()` call at
sync_service.py:212/214 — pre-existing wart that was sitting in the
same block I was editing.

Tests: 21 new tests across `tests/database/`, `tests/sync/`, and
`tests/text/`, plus updates to the existing pool tests to cover the
new fast/fallback split. Full suite stays green (3953 passing).
2026-05-24 23:33:55 -07:00
Broque Thomas
687bb0ca2c Add tests for next_run reset and lazy candidate pool
`tests/automation/test_automation_api.py` gains three update_automation
tests covering the schedule-shape reset:
- trigger_config change blanks next_run
- trigger_type change blanks next_run
- non-trigger field (name) leaves next_run alone

`tests/sync/test_sync_candidate_pool.py` is new — nine tests for the
lazy artist track pool in PlaylistSyncService:
- candidate_pool=None disables pooling and skips the DB call
- first lookup for an artist fetches and caches
- second lookup for the same artist reuses the cache (zero DB calls)
- empty result still cached so the next call short-circuits without SQL
- defensive None return coerced to []
- search_tracks exception returns None and does NOT poison the cache
- pool key is normalized so casing variants share a single fetch
- different artists get separate pool entries
- server_source plumbing survives the trip to search_tracks

All assertions go through fakes / MagicMock — no real DB, no
web_server.py import, no AST-parsing.
2026-05-24 22:58:48 -07:00
Broque Thomas
f83c671570 Add direct mirrored playlist pipeline runs
Expose playlist-native run and status endpoints that reuse the shared mirrored playlist pipeline engine while routing progress into playlist UI state.

Add a Run Pipeline action to mirrored playlist cards and modals with live status polling, and make the shared pipeline lock atomic for manual and scheduled callers.
2026-05-24 19:54:04 -07:00
Broque Thomas
547e499121 Expose mirrored playlist source-ref health
Return normalized source_ref metadata from mirrored playlist APIs so the UI no longer has to infer editable refresh links from description fields. Accept Spotify embed URLs during source-ref repair and add coverage for source-ref health reporting.
2026-05-24 19:32:39 -07:00
Broque Thomas
73bd2db547 Harden playlist pipeline source refresh
Centralize mirrored playlist source reference normalization so edited links and IDs are stored consistently. Preserve URL-backed refresh refs, surface missing-source refresh failures, count background sync failures in pipeline summaries, and retry guarded automation skips after a short delay instead of losing a scheduled run. Add focused coverage for source refs, mirrored playlist source updates, refresh failures, and guarded retry behavior.
2026-05-24 19:31:00 -07:00
Broque Thomas
a7ca7ddfad Harden album bundle fallback flow
Delay torrent and usenet album-bundle dispatch until missing-track analysis confirms there is work to do, matching the Soulseek album flow and avoiding release downloads for already-owned albums.

Clear private album-bundle staging state when a release-level source intentionally falls back to per-track mode so workers can use the normal staging/search path instead of an empty private bundle directory.

Verified by user: focused downloads master tests passed, 2 passed.
2026-05-24 16:15:36 -07:00
BoulderBadgeDad
a1222d5a8f
Merge pull request #686 from kettui/feat/react-migration-import
feat(webui): migrate import page to React
2026-05-24 14:16:05 -07:00
Broque Thomas
0bea332aed Preserve album bundle track numbers
Keep album-bundle staging from replacing known per-track album numbers with the filename parser's default when staged files do not expose a real track number. Carry staging tag numbers through the cache, fall back to task metadata for private release staging, and cap hybrid album batches to one worker when Soulseek is first in the source order.
2026-05-24 13:59:44 -07:00
Antti Kettunen
caa6534ee8
feat(import): show MusicBrainz variants
- pass release metadata through album search normalization
- surface release format, country, label, and disambiguation in React import cards
- add coverage for search normalization and import route rendering
2026-05-24 21:29:22 +03:00
Antti Kettunen
56f642aadd
test(import): remove stale frontend pytest
- delete the source-text guard for the old album lookup cache pattern\n- keep the import-page source-routing contract covered by Vitest route tests\n- avoid duplicating frontend behavior checks across pytest and the webui test suite
2026-05-24 21:17:21 +03:00
Broque Thomas
9a0e3b4011 Persist completed downloads in downloads view
Include a capped recent tail of database-backed download history in the unified Downloads page so completed Deezer and other streaming downloads remain visible after runtime tasks are cleaned up or the container restarts. Use persistent download history for the dashboard finished count, keep live tasks authoritative for active rows, avoid showing the local clear-completed action for persisted history rows, and cover history hydration/deduping/capping in status tests.
2026-05-24 10:02:00 -07:00
Broque Thomas
4ca3f70bf3 Show MusicBrainz release variants in import
Expand matched MusicBrainz release groups into concrete releases for specific album searches so import users can choose the correct edition by track count, format, country, and disambiguation. Preserve distinct MusicBrainz release IDs instead of deduping same-title variants, carry release metadata through import matching, and surface those details on album result cards. Add coverage for variant preservation and release-group expansion.
2026-05-24 09:33:19 -07:00
Broque Thomas
7bee424686 Escape dash-leading YouTube search queries
Fix manual YouTube searches for video IDs that begin with a dash by escaping leading '-' before building yt-dlp ytsearch expressions. This preserves normal search terms and already escaped user input while preventing yt-dlp from treating the ID as search syntax.

Add regression coverage for both YouTube download search and video search paths. Fixes #684.
2026-05-24 08:54:47 -07:00
Broque Thomas
f68afe80c8 Update #524 lookup pattern test for consolidated renderer (#681)
The #681 commit (eba7f61e) collapsed the inline duplicate card-renderer
inside importPageSearchAlbum into a single _renderSuggestionCard call.
The structural test for the issue #524 lookup-cache pattern was still
counting inline cache writes and expecting >=2, which started failing
in CI now that the search-results renderer routes through the shared
helper.

Rewritten to assert the actual invariant of the consolidated design:

* _renderSuggestionCard contains exactly one _albumLookup write
* No other inline write exists (a second write means a caller is
  re-implementing the renderer instead of calling the helper — the
  exact duplication the #524 fix consolidated away)

Same regression guard, matches the new architecture.
2026-05-24 01:46:36 -07:00
Broque Thomas
9769d8be19 Fetch all Qobuz favorite tracks for discovery
Remove the implicit 500-track cap from Qobuz Favorite Tracks so the Sync page discovers the same number of tracks shown on the playlist card. Keep an explicit limit parameter for callers that want a capped fetch.

Add tests covering the default full-pagination behavior and explicit limit handling.
2026-05-24 01:17:38 -07:00
Broque Thomas
a34eae1445 Add Qobuz playlist sync to Sync page (#677)
Qobuz joins Tidal and Deezer as a first-class playlist sync source.
New Qobuz tab on the Sync page lists user playlists + a virtual
Favorite Tracks entry, and clicks route through the same discovery →
sync → download pipeline the other services already use.

Backend:
* core/qobuz_client.py — new get_user_playlists, get_playlist,
  get_user_favorite_tracks, get_user_favorite_tracks_count. Returns
  normalized dicts (matches Deezer client shape, not Tidal's
  dataclasses) so the discovery worker can iterate directly without
  duck-typing. Virtual `qobuz-favorites` ID dispatches to favorites
  fetcher inside get_playlist — same trick Tidal uses with
  COLLECTION_PLAYLIST_ID. Both list endpoints paginate against
  Qobuz's 500-cap limit.
* core/discovery/qobuz.py — new worker module. Mirrors
  core/discovery/deezer.py: pause enrichment, iterate tracks,
  hit discovery cache, fall back to _search_spotify_for_tidal_track,
  build wing-it stub on miss, sync results to mirrored playlist.
* web_server.py — adds /api/qobuz/playlists, /playlist/<id>,
  /discovery/start/<id>, /discovery/status/<id>, /discovery/update_match,
  /playlists/states, /state/<id>, /reset/<id>, /delete/<id>,
  /update_phase/<id>, /sync/start/<id>, /sync/status/<id>,
  /sync/cancel/<id>. One-for-one with the Tidal + Deezer endpoint
  sets. Qobuz discovery executor registered for clean shutdown.

Frontend:
* webui/static/sync-services.js — full handler set (loadQobuzPlaylists,
  createQobuzCard, openQobuzDiscoveryModal, startQobuzDiscoveryPolling,
  startQobuzPlaylistSync, startQobuzSyncPolling, cancelQobuzSync,
  startQobuzDownloadMissing, rehydrateQobuzDownloadModal, etc.).
  Reuses the shared YouTube discovery modal via fake `qobuz_<id>`
  urlHash and is_qobuz_playlist flag. Shared switch statements in
  getModalActionButtons / generateTableRowsFromState / Wing It helpers
  in downloads.js gain new isQobuz branches alongside the existing
  per-service ones.
* webui/index.html — new Qobuz tab button + content div, slotted
  between Deezer and Deezer Link.
* webui/static/style.css — new .qobuz-icon for the tab icon.
* webui/static/core.js — qobuzPlaylists / qobuzPlaylistStates /
  qobuzPlaylistsLoaded globals.

Followed the existing per-service pattern verbatim rather than
refactoring the duplicated transformers across Tidal / Deezer /
Spotify-public / YouTube / Mirrored — that refactor is its own follow-up
PR per the "don't break Tidal/Deezer" scope discipline. Adding the 6th
copy of a proven pattern is lower risk than collapsing 5 working
services behind a new abstraction.

Tests:
* tests/test_qobuz_playlists.py — 12 tests covering pagination,
  normalization, favorites virtual-ID routing, artist-name fallback
  chain (performer → album.artist → 'Unknown Artist'), and
  unauthenticated short-circuits.
2026-05-23 23:27:36 -07:00
Broque Thomas
eba7f61e04 Surface metadata source on Import album results (#681)
Import album search silently fell through to the next source in
METADATA_SOURCE_PRIORITY when the configured primary returned zero
matches — intentional behavior shared with the auto-import worker
(see core/auto_import_worker.py:1316). With MusicBrainz selected and
a query MB couldn't resolve, users saw Deezer cards with no indication
their primary was bypassed.

Backend now echoes `primary_source` on /api/import/search/albums,
/api/import/search/tracks, and /api/import/staging/suggestions.
Frontend renders a per-card 'via {source}' badge when the served
source differs from the primary, plus a banner above the grid when
every card came from a fallback source. Fallback semantics unchanged.

Also collapses an inline duplicate of _renderSuggestionCard inside
importPageSearchAlbum into a single shared renderer.

Regression test pins the contract: response carries primary_source +
per-album source when the chain falls back.
2026-05-23 16:22:17 -07:00
Broque Thomas
6c226613bf Add Soulseek album bundle downloads
Route primary Soulseek album downloads through the album-bundle staging flow, reusing preflight-selected folders when available. In hybrid mode, only the first configured source can claim whole-album bundle behavior so later Soulseek fallback keeps the existing per-track/source-reuse path.

Allow Soulseek album bundles to stage completed tracks when some same-source transfers fail or time out, and keep partial bundles from blocking per-track fallback. Add coverage for dispatcher gating, master flow ordering, task-worker staged-miss behavior, and Soulseek bundle polling.
2026-05-23 15:08:21 -07:00
Broque Thomas
5d1f3c1b48 Fix Picard albumartist orphan false positives
Teach the orphan file detector to match tracked files by both track artist and album artist. This prevents Picard-style albumartist/album (year)/track layouts from being reported as orphans when the DB track artist differs from the album artist.

Also check file albumartist tags during fallback matching and add a regression test for the reported Picard folder layout.
2026-05-22 08:43:57 -07:00
Broque Thomas
4179926899 Fix missing album placeholder asset path
Update Import Music album and queue artwork fallbacks to use the shipped /static/placeholder-album.png asset instead of the nonexistent /static/placeholder.png path.

Replace the remaining static UI fallback to the missing placeholder path and add a regression test that fails if static JS references it again.
2026-05-22 08:34:42 -07:00
Broque Thomas
a41eccbe3c Fix Usenet settings reload without restart
Refresh registry-backed download plugins when settings are saved so cached Prowlarr clients pick up new indexer credentials immediately. This preserves active download state by reloading existing plugin instances instead of rebuilding the registry.

Add regression coverage for orchestrator reload fanout and the Usenet plugin's cached ProwlarrClient refresh path.
2026-05-22 08:28:56 -07:00
Broque Thomas
763888e671 Support legacy HiFi track manifests
Add fallback support for public hifi-api instances that expose playback through /track/ instead of /trackManifests/. The capability checker now accepts either manifest shape, and downloads can use direct URLs decoded from the legacy base64 manifest.

Tests cover legacy instance capability detection and download-manifest fallback while preserving the newer trackManifests path.
2026-05-21 18:11:14 -07:00
Broque Thomas
fae13226e5 Check HiFi download capability via manifests
Probe public HiFi instances with the same trackManifests endpoint used by real downloads instead of the legacy /track endpoint. This prevents compatible instances from being falsely labeled search-only in Settings.

Centralize HiFi instance capability checks in HiFiClient and reuse manifest URI parsing with the download path.

Tests cover manifest-based capability detection, no legacy /track probe, and limited instances without a manifest URI.
2026-05-21 18:04:10 -07:00
Broque Thomas
b9af4ef4ef Handle transient SQLite IO during maintenance
Keep full refresh moving when post-clear VACUUM hits a transient disk I/O error, and retry clear_server_data once when the clear step itself sees the same transient SQLite failure.

Retry metadata cache maintenance writes once on transient disk I/O errors so first-attempt cache jobs do not fail when an immediate retry would succeed.

Tests cover best-effort VACUUM, clear retry behavior, and cache maintenance retry behavior.
2026-05-21 17:50:30 -07:00
Broque Thomas
f1d4f78e0e Repair stale media schema during refresh
Ensure upgraded databases have the tracks.file_size and albums.api_track_count columns after all legacy migrations run. Add defensive repair paths for Jellyfin track imports and album track-count caching so stale schemas self-heal instead of dropping full-refresh track imports.

Tests cover legacy schema repair and api_track_count self-repair.
2026-05-21 17:41:54 -07:00
Broque Thomas
8012f41ef7 fix(album-completeness): block cross-artist auto-fill
Reported bug: filling Jamiroquai's "Light Years" single pulled in
Gut's "Light Years" album tracks (different artist, completely
different genre — track titles like "Wound Fuck" and "Eat My Cum"
made the contamination obvious). The Album Completeness auto-fill
was the only file-copying path with a loose 0.50 SequenceMatcher
artist gate, which let unrelated candidates through whenever the
title matched well.

Two-stage defense now sits on the only album-fill code path
(_fix_incomplete_album in core/repair_worker.py):

- Stage 1 — _album_fill_target_artist_allows_track. Pre-search
  gate: before doing any library lookup for a missing track,
  refuse to operate if the missing track's source artist(s)
  don't match the target album's artist. Compilation albums
  (album_artist in {'various artists', 'various', 'soundtrack'})
  bypass the gate so legitimate VA releases still work. Empty
  source-artist metadata also bypasses for backward compat with
  older missing-track records that don't carry per-track artist.
- Stage 2 — _album_fill_artist_names_match. Replaces the old
  0.50 SequenceMatcher with an alias-aware 0.82 threshold that
  uses core.matching.artist_aliases when available (handles
  diacritic variants like Beyoncé/Beyonce and known stage names)
  with a normalized-similarity fallback if the aliases module
  isn't importable. Skipped candidates are logged at debug so a
  later support ticket can show what was rejected and why.

Tests in tests/test_repair_worker_album_fill.py reproduce the
exact reported scenario: target album "Light Years" by Gut +
missing track from a Jamiroquai source → skipped with a logged
warning, no copy attempted, wishlist not poisoned. Second test
covers Stage 2 directly with a wrong-artist library candidate.
Existing test_perform_album_fill_copy_branch still passes.

Note: this fix prevents NEW cross-artist contamination via
Album Completeness. It does not clean up the data anomaly that
made Gut's library entry appear to have a "Light Years" album
in the first place — that's a separate data-quality issue worth
investigating if it recurs.
2026-05-21 16:49:12 -07:00
Broque Thomas
dee200b267 Add torrent usenet PR notes and test updates
Adds the PR description for the torrent/usenet release-staging feature and updates drifted tests for the new plugin registry entries and search exclude_sources signature.

Verified with the focused pytest command covering cancellation and default registry source registration.
2026-05-21 15:28:48 -07:00
Broque Thomas
6c9b43225a Add torrent and usenet release staging support
Adds torrent/usenet as release-oriented download sources with album-bundle staging, live progress reporting, and post-processing that selects the requested audio file from completed releases instead of blindly importing the first file.

Keeps album-bundle behavior gated to single-source torrent/usenet album downloads, excludes release sources from hybrid album per-track searches, and allows hybrid non-album tracks to use release results safely.

Improves staged-release matching for featured/bonus track filenames while preserving version mismatches, records torrent/usenet provenance in library history, and updates service/status UI labels.

Covers the flow with focused lifecycle, status, staging, validation, task worker, post-processing, and import side-effect tests.
2026-05-21 14:22:21 -07:00
Broque Thomas
8b0de9eb76 fix(downloads): harden album bundle staging
Route torrent and Usenet album bundles through private per-batch staging so Auto-Import cannot race public staging or duplicate imports.

Expose album-bundle progress in batch status and render it on the Downloads page while the external client is still downloading.

Tighten release handoff safety by rejecting archive path traversal, ignoring torrent candidates without a usable URL, and skipping Soulseek source reuse for torrent/Usenet batches.

Tests: .venv/bin/python -m pytest tests/downloads/test_downloads_status.py tests/test_album_bundle_dispatch.py tests/downloads/test_downloads_staging.py tests/test_torrent_usenet_plugins.py
2026-05-20 21:39:06 -07:00
Broque Thomas
440c3624f3 refactor(staging): inject batch-field accessor instead of importing runtime_state
Per code review: the album-bundle provenance override added in an
earlier commit reached into ``core.runtime_state.download_batches``
directly from inside the staging matcher. Sibling modules
shouldn't import each other's globals — the existing StagingDeps
pattern is the canonical way to inject everything else this helper
needs.

- core/downloads/staging.py: new optional ``get_batch_field``
  callable on ``StagingDeps`` (defaults to None for backward compat
  with any caller that doesn't know about it yet). The inline
  ``from core.runtime_state import download_batches`` is gone; the
  helper now calls ``deps.get_batch_field(batch_id,
  'album_bundle_source')`` and falls back to 'staging' when None
  is returned. Accessor exceptions are swallowed with a debug log
  so a deleted batch mid-process can't break the staging match.
- web_server.py: ``_build_staging_deps`` injects a small
  ``_staging_get_batch_field`` helper that wraps the tasks_lock +
  download_batches dict access. Centralises the lock semantics in
  one place — the staging module no longer needs to know about
  the lock or the dict.
- tests/test_staging_album_provenance.py: 5 new tests covering the
  full matrix — torrent override applied, usenet override applied,
  no override falls back to 'staging', missing accessor (default
  None) falls back to 'staging', accessor raising falls back to
  'staging'. Each test seeds + cleans a synthetic task in
  runtime_state so the test doesn't bleed state across the suite.
2026-05-20 20:43:35 -07:00
Broque Thomas
ad59bf05a1 refactor(downloads): lift album-bundle gate into its own module
Per code review: ~90 lines of inline gate logic in
``run_full_missing_tracks_process`` was inflating an already-580-
line worker function and was non-testable in isolation. Lifted to
``core/downloads/album_bundle_dispatch.py`` with two entry points:

- ``is_eligible(mode, is_album, album_name, artist_name)`` — pure
  predicate, no side effects, easy to assert against. Splits the
  gate decision from the resolution + run step so tests can pin
  the gate semantics without standing up a plugin.
- ``try_dispatch(...)`` — full flow. Returns True iff the master
  worker should stop (gate fired + failed); False = engaged-and-
  succeeded OR didn't engage, both fall through to per-track.

State access is now decoupled from ``runtime_state``:
- New ``BatchStateAccess`` Protocol with two methods
  (``update_fields``, ``mark_failed``).
- Concrete impl ``_BatchStateAccessImpl`` lives in master.py and
  wraps the tasks_lock + dict ops the original inline code did.
- Injected via parameter so the dispatch module never imports
  ``download_batches`` / ``tasks_lock`` directly.

Same goes for the plugin resolver and config getter — both
injected, so the dispatcher works against any orchestrator /
config implementation (including the in-test fakes).

Behavior unchanged. The master worker call site is now 11 lines
of boilerplate instead of 90 lines of inline conditional. Plugin
contract (``download_album_to_staging`` return dict shape)
unchanged.

- core/downloads/album_bundle_dispatch.py: new module owning the
  gate + execution. ~150 lines including docstrings and the
  Protocol definition.
- core/downloads/master.py: gate call site shrunk to a single
  ``if _album_bundle_dispatch.try_dispatch(...): return``. New
  ``_BatchStateAccessImpl`` class implements the Protocol against
  the existing ``download_batches`` dict + ``tasks_lock`` so the
  dispatcher gets injected access instead of importing them.
- tests/test_album_bundle_dispatch.py: 16 new tests covering the
  pure predicate (album-required, mode allowlist, name validation,
  case insensitivity), the resolver-failure fall-through
  (plugin missing, plugin lacks method, resolver raises), the
  success path returning False so per-track flows, the failure
  path returning True with state.mark_failed called, plugin-raise
  treated as a normal failure, whitespace stripping on names,
  and progress-callback mirroring lifecycle events into batch
  state.
2026-05-20 20:29:50 -07:00
Broque Thomas
670a2db95e refactor(downloads): extract album_bundle shared helpers + atomic copy
Per code review: the album-bundle helpers (release picker + staging
collision suffix) were defined as private symbols in torrent.py and
imported by usenet.py through ``from core.download_plugins.torrent
import _pick_best_album_release, _unique_staging_path``. Sibling
plugins shouldn't reach into each other's private surface — leaky
module boundary, and the underscore prefix says don't import.

Also addressed two latent issues at the same time:

- The Auto-Import sweep race: my plugin copied audio files into
  staging via plain ``shutil.copy2``, which exposes a partial file
  at the audio extension for the duration of the copy. The Auto-
  Import worker filters by audio extension when scanning Staging
  (AUDIO_EXTENSIONS in core/auto_import_worker.py), so a mid-flight
  scan could pick up a truncated file. Fix: copy to a
  ``.tmp.<random>`` sidecar first, then atomically rename via
  ``Path.replace`` (which is ``os.replace`` — atomic on the same
  filesystem). Auto-Import sees the file either at its final name
  or not at all.

- The 6-hour poll timeout was a hard-coded magic constant. Users
  with slow private trackers or large box sets would silently time
  out after 6h. Both the timeout and the poll interval are now
  read from config (``download_source.album_bundle_timeout_seconds``
  / ``..._poll_interval_seconds``) with safe fallback to the
  existing defaults when unset / non-numeric.

- core/download_plugins/album_bundle.py: new module owns the
  shared surface — ``pick_best_album_release`` (with quality_guess
  passed in as a parameter to avoid the circular import that would
  result from this module trying to know about torrent.py's title
  parser), ``unique_staging_path``, ``atomic_copy_to_staging``,
  ``copy_audio_files_atomically``, ``get_poll_interval``,
  ``get_poll_timeout``. Module-level size constants and quality
  weights live here too. Usenet's grabs-as-popularity-proxy is
  built into the picker so both plugins get the right behavior
  without divergent local logic.
- core/download_plugins/torrent.py: drops the local helpers + the
  hard-coded poll constants, imports from album_bundle. Per-track
  download flow still uses module-level ``_POLL_TIMEOUT_SECONDS``
  / ``_POLL_INTERVAL_SECONDS`` aliases (read from config once at
  import time, same as before from a per-track perspective).
- core/download_plugins/usenet.py: drops the imports of the
  torrent.py private helpers; everything goes through album_bundle
  now. Stops the cross-plugin private-import leak that started
  this whole refactor.
- tests/test_album_bundle.py: 23 new tests covering the picker
  heuristic (empty input, singleton drop, FLAC preference, grabs
  fallback for usenet, size-floor / ceiling boundaries), the
  collision-suffix logic, the atomic-copy invariant (concurrent
  scanner thread asserts it never observes a partial audio file
  during five sequential copies), the failure-skip behavior of the
  batch copier, and the config-driven poll cadence including
  garbage-input fallback.
- tests/test_torrent_usenet_plugins.py: existing picker tests
  updated to call the new module-level helpers instead of the
  former torrent.py privates.
2026-05-20 20:26:30 -07:00
Broque Thomas
c990ce079d feat(downloads): album-bundle flow for torrent/usenet single-source mode
Fixes the core architectural mismatch between indexer-based sources
and the per-track search-and-pick contract every other download
plugin satisfies. Prowlarr returns release-level torrents and NZBs;
searching for "Luther (with SZA)" against the GNX album torrent
scores near-zero on track-title similarity. Per-track candidate
validation rejects every result, every track in the batch flips
to not_found. The album-name fallback added in an earlier commit
papers over it for some cases but doesn't fix the fundamental
behavior: the user wanted the whole album.

New album-bundle flow does what the user actually wanted:
1. Gate fires inside core/downloads/master.py BEFORE the per-track
   analysis loop, strictly when the batch has an album context AND
   download_source.mode is 'torrent' or 'usenet' (single-source —
   hybrid stays per-track to preserve fallback to Soulseek / etc.).
2. Plugin's new download_album_to_staging method searches Prowlarr
   ONCE for the album as a whole ('<artist> <album>'), filters to
   the right protocol, runs results through _pick_best_album_release.
3. Picker prefers seeded FLAC over low-seeded MP3, drops single-
   track torrents that snuck in via the 40 MB size floor (single
   tracks are typically ~10 MB), falls back to most-seeded when
   every candidate is below the floor.
4. Picked release goes to the active adapter (qBit / Transmission /
   Deluge for torrent; SAB / NZBGet for usenet). Polls until
   complete with progress mirrored into the batch state so the
   Downloads page can show meaningful status.
5. On completion the existing archive_pipeline walks the save dir
   (extracting archives if any), every audio file gets copied into
   the staging folder via _unique_staging_path so concurrent batches
   don't collide.
6. Gate exits, master worker continues into the normal per-track
   flow. Each track task hits try_staging_match early in the worker
   and finds its file by fuzzy title match — no Prowlarr search
   ever fires per-track, no candidate rejection, files flow through
   the existing post-processing pipeline (tags, AcoustID, library
   import).

Gate is strictly opt-in. Three orthogonal conditions must all hold:
batch_is_album, mode in ('torrent', 'usenet'), and the plugin must
expose download_album_to_staging. Any other source / hybrid mode /
non-album batch flows through the master worker unchanged. The
existing per-track torrent path still works for basic-search
single-track grabs.

- core/download_plugins/torrent.py: download_album_to_staging plus
  _pick_best_album_release and _unique_staging_path helpers (shared
  with the usenet plugin). _poll_album_download mirrors the existing
  poll loop with progress callback emission.
- core/download_plugins/usenet.py: parallel implementation reusing
  the picker + staging helpers. Different state set ('failed' vs
  'error') from the usenet adapter contract.
- core/downloads/master.py: ~90-line gate right after batch context
  loading. Mirrors plugin lifecycle into batch state under
  ``album_bundle_*`` keys so the Downloads page can render progress
  while the torrent/usenet job runs (per-track tasks don't exist
  yet during this phase). Failed bundle download fails the batch
  with a meaningful error; missing plugin / context falls back to
  the per-track flow with a warning.
- tests/test_torrent_usenet_plugins.py: 5 new tests pinning the
  album picker preferences (FLAC over MP3 with comparable size +
  better seeders, size floor drops singles, fallback when all
  small), staging-path collision suffix, and the not-configured
  short-circuit.
2026-05-20 18:48:48 -07:00
Broque Thomas
478fd25dd6 fix(downloads): pre-fill artist/title so search UI doesn't show download URL
Real-world test surfaced the bug — torrent results displayed
'by download?apikey=c15d6f69...&link=...' as the uploader / artist
in the basic search UI. The cause is TrackResult.__post_init__:
when artist is None it runs parse_filename_metadata on the bare
filename, and our filename starts with the indexer's download URL
(needed so download() can recover the URL later). The auto-parser
treats the URL as 'artist' and ships it to the UI.

Fix:
- core/download_plugins/torrent.py: new _parse_release_title()
  splits 'Artist - Title' / 'Artist - Album' out of the release
  title and strips trailing [FLAC] / (2016) tags. Falls back to
  ('', cleaned_title) when no dash is found, and explicitly
  rejects URL-looking strings as an extra defence. The projection
  pre-fills both artist and title on TrackResult, so __post_init__
  skips the auto-parse entirely. When the release title has no
  dash, artist defaults to the indexer name so the UI shows
  'by Indexer' instead of a URL.
- core/download_plugins/usenet.py: imports the new helper and
  applies the same fix.
- tests/test_torrent_usenet_plugins.py: 5 tests for the new
  helper (dash split, trailing-tag stripping, no-dash fallback,
  multiple-dash preservation, URL-prefix rejection). Existing
  projection tests updated to assert artist + title come through
  parsed correctly, plus a new test pinning the indexer-name
  fallback for titles without a dash so the URL-leak regression
  can't return.
2026-05-20 18:06:23 -07:00
Broque Thomas
080b1aa1b4 feat(downloads): wire torrent + usenet as live download sources
The payoff for the previous five commits. Two new download
sources slot into the existing DownloadSourcePlugin contract,
backed by Prowlarr (search) + the torrent or usenet client
adapter (transfer) + archive_pipeline (post-extract walk). They
appear in the Download Source dropdown next to Soulseek / Tidal /
Lidarr / etc. and also participate in hybrid mode.

Pipeline (both plugins, mirror shape):
1. search(query) → ProwlarrClient.search filtered to the right
   protocol, projected into TrackResult / AlbumResult shapes the
   existing search UI already speaks. Filename field encodes the
   indexer's download URL (or magnet URI for torrents) so
   download() can recover it later.
2. download() → decodes URL, hands it to the active adapter
   (qBittorrent / Transmission / Deluge for torrent; SABnzbd /
   NZBGet for usenet), spawns a background poll thread that
   tracks progress + reports the adapter-reported save_path.
3. On 'seeding' / 'completed' → archive_pipeline walks the save
   directory, extracts any archives the downloader didn't
   already unpack, picks the first audio file as the canonical
   file_path. Matches the Lidarr client's single-track-pick
   contract — picking which specific track to import happens in
   post-processing.

- core/download_plugins/torrent.py: TorrentDownloadPlugin +
  module-level helpers (_decode_filename, _guess_quality_from_title,
  _parse_indexer_id_filter, _adapter_state_to_display, _row_to_status).
  Uses get_active_torrent_adapter() so a settings change to the
  client type takes effect without restart.
- core/download_plugins/usenet.py: UsenetDownloadPlugin —
  parallel shape, reuses the torrent module's helpers. Different
  enough states (no seeding, no magnet) to warrant its own class
  but cheap to keep in lockstep.
- core/download_plugins/registry.py: register 'torrent' and
  'usenet' plugins. Per the registry docstring this is the only
  wiring point needed — the orchestrator picks them up
  automatically via the iteration helpers.
- webui/index.html: 'Torrent Only (via Prowlarr)' + 'Usenet Only
  (via Prowlarr)' added to the Download Source dropdown. New
  redirect card (#prowlarr-source-redirect) explains that the
  actual config lives on the Indexers & Downloaders tab —
  shown whenever torrent or usenet is in the active source set.
- webui/static/settings.js: HYBRID_SOURCES gets two new entries
  so hybrid mode can pick them up. updateDownloadSourceUI now
  toggles the redirect card based on active sources.
- tests/test_torrent_usenet_plugins.py: 23 tests covering pure
  helpers (filename encode/decode round-trip incl. magnet URIs,
  quality guesser, state mapping), search projection logic
  (protocol filter, drops without URLs, magnet-preferred-over-URL,
  filename encoding, neutralised soulseek-specific score fields),
  is_configured (both prowlarr + adapter required), finalize
  (picks first audio file, errors on empty dir / missing save_path),
  clear/get_all lifecycle, DownloadSourcePlugin protocol
  conformance, and registry membership.
2026-05-20 17:22:19 -07:00
Broque Thomas
5f126584f9 feat(downloads): add archive_pipeline module for torrent/usenet downloads
Shared helper the upcoming torrent and usenet download plugins
both compose against. Narrow surface — no matching, no tagging,
no library import. Just walks audio files and extracts archives
when needed.

Why a separate module: usenet downloaders (SABnzbd, NZBGet)
already auto-extract by default, and Lidarr's import pipeline
extracts before SoulSync sees the files. The only client that
sometimes leaves an archive behind is a torrent client when the
album was packed as a .rar — most music torrents ship loose but
not all. Centralising the walk + extract logic means both new
plugins can do the same thing, and a future direct-archive source
(zip download from a private site, etc.) plugs in for free.

- core/archive_pipeline.py:
  - AUDIO_EXTENSIONS / ARCHIVE_EXTENSIONS constants (audio set
    matches core/imports/file_ops.py quality_tiers).
  - is_archive(path) handles compound extensions (.tar.gz etc).
  - walk_audio_files(directory) — recursive, case-insensitive.
  - find_archives_in_dir(directory) — top-level only (don't
    surprise-extract sample / proof folders inside a torrent).
  - extract_archive(archive_path, extract_to=None) — handles
    .zip, .tar variants, .rar (optional rarfile dep), .7z
    (optional py7zr dep). Optional deps warn-and-skip if absent.
  - extract_all_in_dir + collect_audio_after_extraction — the
    one-shot helpers the download plugins call after a download
    completes.
  - Path-traversal protection: every archive member's resolved
    path must stay inside the destination — first violator aborts
    the extract without writing anything. Applies to zip, tar,
    and rar.
- tests/test_archive_pipeline.py: 21 tests covering the walker
  (nested dirs, case-insensitive, ignores non-audio), archive
  detection (compound extensions, missing files), zip extraction
  + path-traversal rejection, tar.gz + tar path-traversal,
  multi-archive directory, mixed-loose-and-archived collection.
2026-05-20 17:05:00 -07:00
Broque Thomas
e4ca56b499 test: cover Prowlarr + torrent + usenet adapters
54 mocked unit tests pinning the parse + dispatch behavior of the
new indexer and downloader plumbing. No live services required —
HTTP is mocked at the requests-library boundary, RPC is mocked at
the _rpc_sync helper.

Coverage:
- core/prowlarr_client.py: parse_indexer / parse_result with
  category-shape variants, search query encodes repeated
  ``categories=`` and ``indexerIds=`` keys, check_connection hits
  the right endpoint with the right header.
- core/torrent_clients/qbittorrent.py: login sends the Referer
  CSRF header, login failure surfaces, parse_status normalises
  field names, eta <= 0 becomes None.
- core/torrent_clients/transmission.py: bare host URL is rewritten
  to /transmission/rpc, 409 + X-Transmission-Session-Id is
  renegotiated and the retry carries the new id, torrent-add
  surfaces torrent-duplicate hashes, eta -1 becomes None.
- core/torrent_clients/deluge.py: requires password to be configured,
  magnet vs HTTP URL hit different RPC methods, progress is
  normalised from 0-100 to 0-1.
- core/usenet_clients/sabnzbd.py: parse_timeleft handles HH:MM:SS
  and the MM:SS fallback, queue + history merge into a single
  get_all, addurl vs addfile are dispatched on the input type.
- core/usenet_clients/nzbget.py: requires URL + username + password,
  mb_value prefers the 64-bit size split over the legacy MB field,
  add_nzb base64-encodes raw bytes, GroupFinalDelete vs GroupDelete
  is picked by the delete_files flag, non-numeric job IDs fail fast.
- state mapping tables for all five adapters get explicit assertions
  so future refactors can't silently lose a native state value.

WHATS_NEW entry covers the test addition; no VERSION_MODAL_SECTIONS
entry — internal infrastructure, not user-facing.
2026-05-20 15:43:18 -07:00
Broque Thomas
3375b6c4bd Handle non-JSON Tidal auth responses
Detect JSON decode-like exceptions from Tidal's token endpoint and return a safer, more actionable error message. Adds a _looks_like_json_decode_error helper and special-cases that error in check_device_auth to log the non-JSON response and advise disabling VPN/proxy/network filtering and restarting SoulSync. A test was added to ensure the user-facing message does not leak the raw exception text while still returning an error status. Other errors continue to fall back to the existing behavior.
2026-05-20 14:04:45 -07:00
Broque Thomas
2fc08e199e Enforce duration tolerance for strict sources
Add duration tolerance logic and pre-download rejection for structured sources (tidal, qobuz, hifi, deezer_dl, amazon) when candidate duration deviates beyond allowed tolerance. Introduces helper functions _duration_tolerance_seconds and _duration_mismatch_exceeds_integrity_tolerance and uses resolve_duration_tolerance from core.imports.file_integrity. Log and skip candidates that would fail post-processing integrity checks to avoid wasted downloads. Update tests to include matching engine stub and new cases covering rejection and acceptance based on duration tolerance; also adjust imports and test fixtures.
2026-05-20 11:24:57 -07:00
Broque Thomas
136d665c8a feat(webui): cache artwork images on disk
Add a disk-backed image cache with hashed browser URLs, SQLite metadata, size/type validation, stale fallback, and per-image fetch locking. Route normalized artwork through /api/image-cache while keeping /api/image-proxy as a compatibility shim, and align browser max-age with the image cache TTL. Add focused tests for cache behavior and image URL normalization.
2026-05-20 10:43:47 -07:00