Commit graph

2490 commits

Author SHA1 Message Date
Broque Thomas
e309370862 Source picker: rename Soulseek icon to "Basic Search"
That source icon hits /api/search — raw slskd file results, the same
flow the UI historically labelled "Basic Search" before the source-icon
row replaced the dropdown. Reverting the label avoids implying it
returns Soulseek-flavoured metadata results in the same shape as the
other source icons.

Backend route + endpoint name unchanged; this is display-only.
2026-04-27 16:03:56 -07:00
Broque Thomas
b94cbd7dd7 Search lift: pre-merge parity polish for cin's review
Three drifts caught in line-by-line review against the pre-lift
web_server.py. All addressed for strict 1:1 behavior parity.

1. /api/enhanced-search/source/<src> now returns plain JSON
   `{"artists":[],"albums":[],"tracks":[],"available":false}` (or
   `{"videos":[],"available":false}` for youtube_videos) when the
   source's client isn't available, matching the original endpoint
   contract. Previously streamed an NDJSON `{"type":"done"}` line
   instead.

   Restructured by splitting the orchestrator into resolve+stream
   helpers:
   - `resolve_client(source_name, deps)` — already existed, used
     for /api/enhanced-search single-source mode
   - `resolve_youtube_videos_client(deps)` — new, returns the
     soulseek_client.youtube subclient or None
   - `stream_metadata_source(source_name, query, client)` — pure
     NDJSON generator, caller resolves client first
   - `stream_youtube_videos(query, youtube_client, run_async)` —
     same shape for the yt-dlp path

   The route now decides plain-JSON-vs-stream based on resolution
   result, mirroring the original control flow exactly.

2. core/search/library_check.py — reverted the defensive `(x or '')`
   and `getattr(plex_client, 'server', None) is not None` patterns
   to original byte-for-byte (`x.get('name', '')`,
   `plex_client.server`, no try/except around `get_plex_config`).
   Lift PR shouldn't change crash semantics; if the original raises
   on malformed input, mine should too. Pre-existing edge cases get
   their own follow-up PR.

3. core/search/stream.py — same revert: `soulseek_client.youtube`
   instead of `getattr(..., 'youtube', None)` etc.

Also removed the module-level `EMPTY_SOURCE` from sources.py and
moved its (per-call) duplicate into _fan_out_response as a local —
the original used a per-request local dict and the identity-check
behavior depends on that. Module-level was a footgun for future
mutations.

789 tests still pass (95 search), ruff clean.
2026-04-27 15:32:48 -07:00
Broque Thomas
47b4663091 Search cache: preserve falsy provider returns to match original behavior
Line-by-line review of the search lift caught one drift: cache.get_cache_key
was coercing falsy provider returns ('', None, 0) to 'unknown' / False.
Original web_server.py only fell back to those sentinels on exception, not
on falsy success values.

Real-world impact: low — get_active_media_server() and get_primary_source()
return non-empty strings in practice. But cache keys are tuples with no
schema enforcement, so any drift here can silently fragment the cache.
Restored 1:1 parity with original semantics.

Added test covering the falsy-success path so this can't drift again.

789 tests pass, ruff clean.
2026-04-27 15:19:52 -07:00
Broque Thomas
fd7b56e58c Lift /api/search and /api/enhanced-search/* into core/search/
Routes moved to thin parse-args/jsonify handlers; logic now lives in
six focused modules under core/search/. 720 lines deleted from
web_server.py; 109 added back as wrappers; ~700 lines of new core code
plus ~700 lines of tests.

Module split:
- core/search/cache.py — TTL+LRU cache for enhanced-search responses,
  keyed by (query, active_server, fallback_source, hydrabase_active,
  source_tag) so config changes don't poison stale entries.
- core/search/sources.py — per-kind metadata search (artists/albums/
  tracks) and the multi-kind ThreadPoolExecutor that fans them out.
- core/search/library_check.py — library + wishlist presence check
  with Plex thumb URL resolution; profile-aware wishlist with legacy
  fallback for older DBs missing the profile_id column.
- core/search/stream.py — single-track preview search; effective stream
  mode resolution, query-variant generation, retry walk, matching
  engine integration.
- core/search/basic.py — flat Soulseek file search, quality-sorted.
- core/search/orchestrator.py — main enhanced-search dispatch
  (short-query fast path, single-source bypass, hydrabase-primary fan
  out, alternate source list builder), NDJSON streaming generator
  for /source/<src>, and the SearchDeps dataclass that bundles the
  cross-cutting deps.

Routes pass clients (spotify, hydrabase, hydrabase_worker, soulseek)
and helpers (config_manager, fix_artist_image_url,
_is_hydrabase_active, _get_metadata_fallback_*, _run_background_
comparison, run_async, dev_mode_enabled_provider) into core/search via
a SearchDeps bundle built per-request. fix_artist_image_url stays in
web_server.py because it touches 31 other call sites.

Behavior preserved 1:1:
- Same response shapes (db_artists, spotify_artists, spotify_albums,
  spotify_tracks, primary_source, metadata_source, alternate_sources,
  source_available)
- Same NDJSON line ordering (artists/albums/tracks as they finish, plus
  done marker)
- Same per-kind exception swallowing
- Same hydrabase-worker mirror on dev mode
- Same cache key shape (5-tuple) and TTL/LRU semantics
- Same stream-track effective-mode resolution including the
  Soulseek-coerce-to-YouTube edge case
- Same library-check Plex thumb URL rewriting and wishlist fallback
  for older DBs

Tests: 94 new (cache TTL/LRU/key, sources happy/partial/all-fail,
library presence with library + wishlist + thumbs, stream effective
mode + query gen + retry, orchestrator client resolution + short
query + single source + fan-out alternates + hydrabase primary +
NDJSON drain). Full suite: 788 passing (was 694).

Ruff clean.
2026-04-27 15:07:11 -07:00
BoulderBadgeDad
ac004f5ecf
Merge pull request #390 from Nezreka/refactor/lift-stats-to-core
Lift /api/stats/* and /api/listening-stats/* into core/stats/
2026-04-27 14:31:51 -07:00
Broque Thomas
f51b75da7e Lift /api/stats/* and /api/listening-stats/* into core/stats/
Stats route logic moves into core/stats/queries.py as pure-ish functions
that take dependencies (database, image-url fixer, listening worker) as
arguments. The 13 route handlers in web_server.py shrink to thin
parse-args / jsonify wrappers.

What moved to core/stats/queries.py:
- stats_cached: 3-key metadata cache lookup + image url fix-up
- stats_overview / timeline / genres / library_health / db_storage
- stats_top_artists / top_albums / top_tracks: top-N + DB enrichment
- stats_recent: listening_history readback
- stats_resolve_track: title+artist -> file_path lookup for playback
- listening_stats_sync: spawns daemon thread that runs worker._poll
- listening_stats_status: stats payload, with None-worker fallback shape

No behavior change. Same response shapes, same error handling, same
silent-except on per-row enrichment failure. fix_artist_image_url
stays in web_server.py and is passed through as a callback so we
don't have to lift its config_manager / media-server dependencies in
this PR.

Adds tests/stats/test_stats_queries.py — 27 tests covering happy
paths, edge cases, image-url plumbing, worker glue.

Ruff clean. 694 tests pass (was 667 + 27 new).
2026-04-27 14:27:03 -07:00
BoulderBadgeDad
c4626ae503
Merge pull request #389 from Nezreka/fix/post-pr378-lint-cleanup
Drop stale post-PR378 redefs and fix B009
2026-04-27 13:51:03 -07:00
Broque Thomas
313b5677a5 Drop stale post-PR378 redefs and fix B009
Lifted-then-not-deleted leftovers from the PR378 merge:
- web_server.py `_resolve_album_group` and `_build_final_path_for_track`
  were already imported at module top from `core/imports/`. Removed the
  shadowing local copies.
- Mutagen reimports (FLAC/MP4/OggVorbis) at L17736-17738 shadowed the
  top-of-file imports. Picture/MP4Cover/MP4FreeForm were unused. Dropped
  the whole block.
- core/imports/context.py: `getattr(artist, "name")` -> `artist.name`
  (B009).

Ruff clean, 667 tests pass.
2026-04-27 13:39:16 -07:00
BoulderBadgeDad
f7b01f476a
Merge pull request #378 from kettui/refactor/extract-import-pipelines
Refactor import and related post-processing pipelines
2026-04-27 13:19:59 -07:00
Antti Kettunen
02305096a3
Tighten metadata and import safety
- Normalize album import track display handling so queue labels and match rows stay consistent
- Bound MusicBrainz caches and avoid caching transient lookup failures
- Stop swallowing programmer errors in source enrichment helpers
- Restore import config test seams without reintroducing lazy imports
- Guard task completion calls and fix the Windows path test expectation
- Keep file lock tracking from growing without bound
2026-04-27 20:28:05 +03:00
Antti Kettunen
9315e74bea
Broaden import and metadata test coverage
- Cover search_result fallback normalization and ambiguous album detection.
- Add staging metadata, multi-disc path, and MusicBrainz enrichment cases.
- Move the single-track context test next to the imports code it exercises.
2026-04-27 19:55:07 +03:00
Antti Kettunen
4f236baa6d
Fix import normalization and task completion locking
- Promote legacy _source into source during import normalization.
- Keep the normalized import context neutral after stripping aliases.
- Avoid re-entering tasks_lock when marking completed download tasks.
2026-04-27 19:55:07 +03:00
Antti Kettunen
6ee119ffa9
Fix DummyConfigManager position in album completeness job test 2026-04-27 19:55:06 +03:00
Antti Kettunen
4c819681a1
Move single-track resolver; fix wishlist cleanup
- keep single-track import lookup in imports/resolution.py
- normalize simple-download search_result data before wishlist matching
- run wishlist cleanup for simple-download post-processing
- keep source-only artist detail on resolved names and MB short-circuit
2026-04-27 19:55:06 +03:00
Antti Kettunen
9321fc4ad2
Small cleanup 2026-04-27 19:55:06 +03:00
Antti Kettunen
d04573f397
Fix single import source handling
- pass the selected manual match through singles import
- keep the import context source-aware so artist and album stay correct
- avoid treating non-Spotify IDs as wishlist Spotify IDs
- make wishlist logging and local variable names source-neutral
2026-04-27 19:54:45 +03:00
Antti Kettunen
594c8c1b93
Cleanup duplicated code
Leftovers from the earlier recovery mission
2026-04-27 19:54:45 +03:00
Antti Kettunen
9b2b6d856f
Split runtime builders into owning modules
- Move the import pipeline runtime factory into core.imports.pipeline
- Move the metadata runtime factory into core.metadata.enrichment
- Keep the web server wiring thin and drop the shared glue module
- Add contract tests that keep the two runtime bundles separate
2026-04-27 19:54:45 +03:00
Antti Kettunen
bcab54095e
Group metadata tests under tests/metadata
- Move the metadata and MusicBrainz-related tests into a dedicated tests/metadata subfolder.
- Keep the rest of the suite flat for now.
- Preserve the existing test filenames so the change stays organizational rather than behavioral.
2026-04-27 19:54:44 +03:00
Antti Kettunen
9e496397da
Move shared metadata helpers into package
- Relocate the shared metadata helper module from core/metadata_common.py into core/metadata/common.py.
- Update the new metadata package, the import pipeline, and the web entrypoint to use the package-scoped helper.
- Keep the shared config, mutagen, file-lock, and tag-writing helpers centralized without touching unrelated files.
2026-04-27 19:54:44 +03:00
Antti Kettunen
9656dbd46a
Thread runtime through metadata enrichment
- Pass the live runtime bundle into the shared metadata facade so worker-backed source enrichment can actually run.
- Forward runtime from the import pipeline and web-server wrapper into embed_source_ids.
- Add a regression test that verifies the runtime object reaches the source-ID embedding path.
2026-04-27 19:54:44 +03:00
Antti Kettunen
8319c6679f
Move new metadata helpers into a package
- Keep existing metadata_cache and metadata_service at the top level for now
- Move the new branch-local metadata helpers under core/metadata
- Share MusicBrainz release cache state from core.metadata.source and update import sites
2026-04-27 19:54:44 +03:00
Antti Kettunen
bdef127dd6
Lift shared runtime state into core
- Move app-wide task and activity registries out of core/imports
- Share one runtime-state module across the web server, API, and import pipeline
- Keep import-specific helpers focused on context and post-processing
2026-04-27 19:54:44 +03:00
Antti Kettunen
e10df4caf2
Rehome import helpers into core/imports
- Move import flow modules into a dedicated package
- Update app and test imports to the new namespace
- Group the import-focused tests under tests/imports
2026-04-27 19:54:44 +03:00
Antti Kettunen
b9269b4f16
Tighten metadata helper boundaries
- remove stale wrapper helpers from web_server and metadata_common
- import provider helpers directly in metadata_source
- keep the metadata modules' public surface explicit
2026-04-27 19:54:43 +03:00
Antti Kettunen
edd9048f86
Checkpoint metadata runtime cleanup
- remove runtime from metadata helper APIs where it only carried config, logger, mutagen, and database access
- keep runtime only for the source-ID enrichment path that still needs live worker handles
- add the new metadata helper modules and update the tests to match the slimmer interfaces
2026-04-27 19:54:43 +03:00
Antti Kettunen
6872e5080d
Refine import module boundaries
- Move filename and staging helpers into their canonical modules
- Extract album naming and grouping from path handling
- Update import and test call sites to the new layout
2026-04-27 19:54:43 +03:00
Antti Kettunen
0bbf44809f
Move the import flows and related post-processing pipelines into separate modules
- Extract the import pipeline, album import, staging, path, file ops, guards, runtime state, side effects, and metadata enrichment out of .
- Canonicalize the refactored import path around  and remove legacy , , , and  request shapes from the import endpoints.
- Make album and track metadata lookups follow the configured provider priority instead of hard-coding Spotify, while still falling back when needed.
- Update the import routes and frontend payloads to use the new core helpers.
- Add coverage for the extracted helpers and the refactored import flows.

PS. apologies to anyone who might check this commit out - the intention was to start small, but things kinda snowballed out of control at some point since the logic just kept going on and on, and everything kinda had to be changed all at once for it all to make any sense
2026-04-27 19:54:43 +03:00
BoulderBadgeDad
eb442da728
Merge pull request #387 from Nezreka/feat/service-worker-and-pwa-manifest
Service worker for cover art + PWA manifest
2026-04-26 22:30:19 -07:00
Broque Thomas
f11b91a5c6 Service worker for cover art + PWA manifest
Addresses #365 (reported by JohnBaumb), parts 3 & 5. Client-side
IDB / sessionStorage data cache (part 4) deferred to its own PR.

Cover art on Library and Discover used to re-fetch from the source
CDN on every page visit. Now a service worker caches images locally
in CacheStorage with cache-first strategy — second visit serves art
instantly with zero network round-trips. PWA manifest added so the
app is installable to home screen / desktop.

Service worker (`webui/static/sw.js`):
- Cache-first for images: 10 known CDN hosts (Spotify, Last.fm,
  Apple, Deezer, Discogs, MusicBrainz CAA, YouTube thumbnails) plus
  the local `/api/image-proxy` endpoint plus same-origin .png/.jpg/
  .webp/.gif/.svg paths. Cross-origin file-extension matches are
  refused so we don't accidentally cache trackers.
- Stale-while-revalidate for `/static/*`: serve cached instantly,
  refresh in background. Combined with the existing `?v=static_v`
  cache-bust, deploys still ship live (different query → different
  cache entry, old ages out).
- HTML / API / everything else: no caching, pass through.
- Cache-versioned (CACHE_VERSION = 'v1'); activate handler wipes any
  cache whose name doesn't match the current version.
- skipWaiting + clients.claim so deploys propagate to open tabs
  without requiring a full close-and-reopen.

PWA manifest (`webui/static/manifest.json`):
- Standalone display mode, theme color #1db954 (matches --accent-rgb).
- Two icons (192, 512) with both `any` and `maskable` purpose,
  generated from favicon.png with aspect-preserving transparent
  padding so the existing logo lands inside the safe zone for
  OS-applied masks.

Wiring:
- `web_server.py` adds a `/sw.js` route that serves the file from
  root scope (a service worker only controls URLs at or below its
  served path; `/static/sw.js` would scope to `/static/*` only).
  `Cache-Control: no-cache` on the SW response so deploys propagate
  on next page load instead of being pinned by the 1yr static cache
  the rest of /static/ uses.
- `webui/index.html` adds the manifest link, theme-color meta, and
  an apple-touch-icon for iOS.
- `webui/static/init.js` registers the SW on `window.load`.
  Feature-detected — no-op on browsers without serviceWorker support
  or on non-secure origins (SW requires https or localhost).

One bug caught + fixed during line-by-line self-review:
`_staleWhileRevalidate` could return null to `respondWith()` when
both the cache miss AND the network fetch failed (the `.catch(() =>
null)` collapsed the rejection to null, which then short-circuited
through the falsy chain). Now explicitly awaits the network promise
and falls back to `Response.error()` when it resolves to null —
matches the `_cacheFirst` pattern.

Browser-verified: sw.js registers, status "activated and is running"
in DevTools. 603 tests pass.
2026-04-26 22:17:52 -07:00
BoulderBadgeDad
7bc7936371
Merge pull request #386 from Nezreka/feat/static-cache-and-discover-cache-headers
Feat/static cache and discover cache headers
2026-04-26 21:41:57 -07:00
Broque Thomas
5d9e5e5781 Discover cache: switch from public to private
Self-review nit on b0e7dae. Discover data is user-specific (hero
artists from your watchlist, similar artists from your taste,
recently-played derivations, etc.) — `Cache-Control: public` would let
intermediate proxies (corporate caching proxy, Cloudflare with cache
rules, Nginx with proxy_cache) store one user's response and serve
it to another. Privacy leak.

Switched to `private, max-age=300`. Browser-only cache, proxies skip.
Static assets stay `public` (shared content — everyone gets the same
library.js). Streaming and backup endpoints already correct
(`no-cache` and `no-store` respectively).

603 tests pass.
2026-04-26 21:27:46 -07:00
Broque Thomas
b0e7dae7c6 Cache static assets 1y + cache discover GETs 5min
Addresses #365 (reported by JohnBaumb), parts 1 & 2 of the proposal.
Service worker, client-side IDB/sessionStorage, and PWA manifest
deferred to follow-up PRs.

1. Static asset cache (CSS/JS/icons/fonts).
   `SEND_FILE_MAX_AGE_DEFAULT` flipped from 0 to 31536000 (1 year) in
   production. Safe because every static URL is bust-tagged with
   `?v=static_v` (computed once per process start), so each server
   restart effectively invalidates every cached asset for every user.
   Within a single deploy, repeat page loads hit zero round-trips on
   static files — was a 304 round-trip per asset before.
   Dev override (`SOULSYNC_WEB_DEV_NO_CACHE=1`) keeps it at 0 so
   iterating on JS/CSS doesn't need a server restart between edits.

   Collateral fixes from the bump:
   - Music streaming endpoint (L16140): `response.headers.add('Cache-Control',
     'no-cache')` → bracket-assign. Under the old max-age=0, send_file
     set `no-cache` and `.add()` duplicated harmlessly. Under the new
     max-age=31536000, `.add()` would APPEND a second Cache-Control
     value → two conflicting headers, browser-undefined behavior.
     Bracket-assign replaces.
   - Backup download endpoint (L25181): explicit `Cache-Control:
     no-store` on the response so DB backups don't inherit the new
     long max-age — sensitive content, must never cache.

2. Discover GET browser cache (5 min).
   New `@app.after_request` hook scoped to `/api/discover/` and
   `/api/discovery/` paths, GET method, 2xx responses only. Sets
   `Cache-Control: public, max-age=300`. Skipped when the endpoint
   already set its own Cache-Control. Toggling between Discover
   sections within 5 min serves from browser cache, no backend hit.

   Try/except wraps the hook body and logs a warning if anything
   throws — never let a header-tagging bug turn a successful response
   into a 500. (Logging instead of `pass` since silent except-pass is
   exactly the anti-pattern issue #369 is about.)

Audited every other Cache-Control set site in web_server.py — only
the two `send_file` callers needed adjustment. Range-branch streaming
uses `Response()` directly, unaffected by the config change.

603 tests pass.
2026-04-26 21:16:41 -07:00
BoulderBadgeDad
e0573729ed
Merge pull request #385 from Nezreka/fix/settings-endpoint-no-auth
Gate /api/settings endpoints behind admin profile
2026-04-26 20:18:28 -07:00
Broque Thomas
01b7d50311 Gate /api/settings endpoints behind admin profile
Closes #370 (reported by JohnBaumb).

The /api/settings endpoint and three siblings (/log-level,
/config-status, /verify) had no auth check — any logged-in profile
could read or modify service tokens, OAuth secrets, and API keys.
Cin's "minimum" suggestion from the issue: gate to admin profile.

Added an `admin_only` decorator near `get_current_profile_id` that
returns 403 when the current profile isn't admin (id=1). Applied
to all four endpoints.

Auth model note (documented in the decorator docstring): SoulSync's
existing model is "trust local network" — single-admin / no-multi-
profile installs default `get_current_profile_id()` to 1, so the
gate is a no-op for solo users. The decorator is meaningful in
multi-profile setups where non-admin sessions exist. Tightening to
real per-request auth is out of scope.

Did NOT consolidate with api/settings.py (Cin's "better" suggestion):
that endpoint uses API-key auth (for external tools), the web_server.py
copy uses session/profile auth (for the web UI). Different consumers,
different auth models — merging would break one or the other.

603 tests pass.
2026-04-26 20:01:01 -07:00
BoulderBadgeDad
9e8b5aee3b
Merge pull request #383 from Nezreka/fix/socketio-cors-wildcard
Lock down Socket.IO CORS — same-origin default + opt-in allow-list
2026-04-26 19:27:16 -07:00
Broque Thomas
dd4cf130d7 Socket.IO CORS: handle self-review nits
Six items from a Cin-style line-by-line pass on PR #383:

- resolve_cors_origins: list of non-string entries (`[None, 123]`) now
  drops them instead of coercing to junk strings like `'None'`/`'123'`.
- will_reject: backwards-compat shim removed. Production callers always
  pass `request.scheme` (Flask-guaranteed); the shim only existed for
  tests/non-Flask callers and made the production code path branchier
  than necessary. Tests now pass scheme explicitly.
- maybe_log: redundant `if not origin` early-return dropped. will_reject
  handles missing origin (engineio's own behavior — server.py:207).
- RejectionLogger.__init__: `int(dedup_cap)` wrapped in try/except so
  bad-type input falls back to DEFAULT_DEDUP_CAP instead of raising.
- web_server.py: docstring on the before_request hook explains why the
  hook fires on every request (Flask doesn't scope before_request to a
  path prefix; the early-return string compare is the cheapest option).
- settings.js: cors-origins URL regex tightened from `[^\s/]+` to
  `[^\s/?#]+` so query/fragment chars don't pass validation. Engineio
  would silently fail to match those anyway; better to flag at save.

Test changes:
- parametrize gained an explicit `scheme` column (12 cases updated).
- New explicit case: scheme-mismatch rejects (engineio compares full
  `{scheme}://{host}` strings).
- `test_will_reject_falls_back_to_host_only_when_no_scheme_info`
  deleted — the shim it tested is gone.
- `test_will_reject_honors_x_forwarded_host` now passes scheme info.

Net: -9 production lines, -3 test lines. Production code path is
straight-line. 603 tests pass.
2026-04-26 19:24:43 -07:00
Broque Thomas
efd2960629 Merge remote-tracking branch 'origin/dev' into fix/socketio-cors-wildcard
# Conflicts:
#	webui/static/helper.js
2026-04-26 18:38:11 -07:00
BoulderBadgeDad
3666e65a2e
Merge pull request #384 from Nezreka/fix/entrypoint-pip-install-on-startup
Pin yt-dlp in requirements.txt, drop pip install from entrypoint
2026-04-26 18:34:22 -07:00
Broque Thomas
22fda5dd94 Trim yt-dlp pin comment, drop misleading WHATS_NEW page link
Self-review nits on PR #384:

- requirements.txt: 5-line comment for one pin → 1 line. Rationale
  lives in commit body and #367; no need to repeat in-tree.
- helper.js: dropped `page: 'settings'` from the yt-dlp WHATS_NEW
  entry. Settings page has no yt-dlp UI; the link would have
  navigated users somewhere irrelevant.

553 tests pass.
2026-04-26 18:30:12 -07:00
Broque Thomas
77a781caba Pin yt-dlp in requirements.txt, drop pip install from entrypoint
Closes #367 (reported by JohnBaumb).

The Docker entrypoint ran `pip install -U yt-dlp --quiet --no-cache-dir`
on every container start. Three problems with that:

- Non-deterministic startup: each restart could pick up a different
  yt-dlp version, making "works on my machine" debugging harder.
- Network dependency at boot: PyPI being slow/unreachable gated the
  app coming up.
- In-place upgrades inside running containers can race with active
  yt-dlp invocations and aren't a great pattern.

Picked Option A from the issue: pin to an exact version in
requirements.txt (`yt-dlp==2026.3.17`) and remove the entrypoint
install entirely. yt-dlp comes baked into the image now via the
existing `pip install -r requirements.txt` in the Dockerfile.

Tradeoff: YouTube fixes ship via SoulSync releases now instead of
"next container restart". The pin is documented inline with how to
bump it.

Net change: -3 entrypoint lines, requirements.txt pin tightened,
WHATS_NEW '2.4.1' block opened (entries hidden until version bumps).

553 tests pass.
2026-04-26 18:02:20 -07:00
Broque Thomas
0f24739e27 Socket.IO CORS: polish — match engineio exactly, bound dedup, validate URLs
Self-review pass on the security fix uncovered five issues, all fixed
here:

1. will_reject scheme handling. Engineio compares full {scheme}://{host}
   strings, not just hostnames. A TLS-terminating proxy can leave the
   backend seeing http while the browser's Origin is https — engineio
   rejects, but the original predictor said "allow" → no helpful log
   line. Added request_scheme + forwarded_proto params, build full
   candidate strings to match engineio.

2. EITHER-forwarded-header rule. Engineio adds the forwarded candidate
   when EITHER X-Forwarded-Proto OR X-Forwarded-Host is present (it
   falls back to HTTP_HOST for the missing one). The original predictor
   only added it when forwarded_host was set — false negative for
   misconfigs sending only X-Forwarded-Proto. Now mirrors engineio.

3. will_reject incorrectly rejected missing-Origin requests. Engineio
   (server.py:207: `if origin: validate`) skips CORS validation when
   no Origin header is sent — non-browser clients (curl etc.) are
   intentionally permitted. The original code rejected them. Test was
   asserting the wrong behavior. Both fixed.

4. RejectionLogger had unbounded dedup set growth. A hostile actor
   opening connections from many distinct fake origins would fill
   memory unboundedly. Capped at 100 unique origins (configurable);
   when cap hit, one overflow notice is emitted and further rejections
   are silently dropped until restart.

5. Lock pattern: the overflow log path called logger.warning() while
   holding the dedup lock, inconsistent with the normal path. Fixed
   to pick the message under the lock and log after release. Critical
   section is now minimal and uniform.

Plus polish:
- Stale module docstring fixed (said "empty list" instead of "None").
- settings.js validates each cors_origins line against a URL regex on
  save; toasts a one-shot warning if entries are malformed (resolver
  silently filters them, but user gets feedback now).
- web_server.py wiring passes request.scheme + X-Forwarded-Proto so
  the predictor has full proxy info.

Tests:
- 51 unit tests in tests/test_socketio_cors.py (was 45). New cases:
  * scheme comparison (5 cases including TLS-terminating proxies)
  * forwarded_proto-alone misconfig
  * missing-origin matches engineio (was asserting wrong behavior)
  * dedup cap with overflow + reset
  * default cap is reasonable (uses public DEFAULT_DEDUP_CAP constant)

Engineio behavior independently verified by reading engineio/server.py
and engineio/base_server.py source. Predictor mirrors both files.

604 tests pass.
2026-04-26 17:32:22 -07:00
Broque Thomas
013eebf350 Lock down Socket.IO CORS — same-origin default + opt-in allow-list
Closes #366 (reported by JohnBaumb).

Socket.IO was initialized with `cors_allowed_origins='*'`, accepting
WebSocket connections from any origin. A malicious site could open a
WS to a user's local SoulSync instance and exfiltrate live progress /
toast / activity events.

This commit:

- Defaults to engineio's same-origin behavior (`cors_allowed_origins=None`),
  which automatically honors X-Forwarded-Host so reverse proxies that
  send that header (Caddy / Traefik by default, properly-configured
  Nginx) work transparently.
- Adds a `security.cors_origins` config setting + Settings → Security
  textarea where users behind unusual proxies / Electron wrappers /
  cross-origin integrations can whitelist their origin. Accepts comma
  or newline separated values; `*` on its own line opts back into the
  legacy wildcard with a startup-warning log.
- Logs a clear warning the first time engineio rejects each unique
  origin, naming the rejected Origin and request Host and pointing
  users to the settings field. Without this, engineio silently 403s
  the upgrade and the user just sees a half-broken UI with no clue
  why. Threadsafe dedup so a hostile origin can't spam logs.

Logic lives in `core/socketio_cors.py` (resolver, rejection
predictor, dedup logger class, startup-status emitter) — pure
functions, no Flask dependency. `web_server.py` adds 23 lines of
wiring and imports.

Important catch during review: my first pass used `cors_allowed_origins=[]`
as the "secure default." Reading engineio's source revealed `[]` actually
means "DISABLE CORS HANDLING" (engineio/server.py:202: `if cors_allowed_origins != []:`)
— identical security to `'*'`. Fixed to use `None` (engineio's actual
same-origin sentinel) and pinned with a regression test that asserts
the resolver never returns `[]` for any input shape.

Tests:
- tests/test_socketio_cors.py — 45 unit tests covering 19 resolver shape
  cases (None, empty, whitespace, comma, newline, garbage types, lists),
  the `[]`-must-never-be-returned security regression, 12 rejection
  prediction cases, X-Forwarded-Host handling, dedup logger behavior,
  threadsafe race (8 threads × 50 hammers → exactly 1 warning), and
  startup-status emitter outputs.

Frontend:
- Settings → Security gains an "Allowed WebSocket Origins" textarea
  with help text explaining same-origin default + when to add a domain
  + the `*` opt-out.
- helper.js — new '2.4.1' WHATS_NEW block (hidden until version bump)
  with a chill-voice entry describing the change.

Conftest.py left at `'*'` — test environment, no security concern.

598 tests pass.
2026-04-26 16:27:10 -07:00
BoulderBadgeDad
f9f0d80ab8
Merge pull request #382 from Nezreka/refactor/changelog-single-source
Refactor/changelog single source
2026-04-26 14:46:31 -07:00
Broque Thomas
04ff287c72 Rewrite changelog entries in user voice
Trimmed the WHATS_NEW '2.4.0' block (27 entries) and the full
VERSION_MODAL_SECTIONS array (23 sections) from the diagnostic-paragraph
style I'd been defaulting to into something terse and casual:

- Descriptions are 1-2 short sentences instead of multi-clause writeups.
- Modal feature bullets capped at 3-7 short items each.
- Stripped parenthetical credits from titles (no more "(kettui Review)",
  "(Images, Counts, Title Hints)" — those belong in git history, not UI).
- Lowercase casual tone throughout description bodies.
- No reporter handles in entry text.

Net: 176 insertions / 194 deletions. helper.js parses, 553 tests pass.
2026-04-26 14:25:30 -07:00
Broque Thomas
7714b51a50 Lift version modal data into helper.js, delete /api/version-info
The version modal pulled its content from /api/version-info — a 295-line
hand-curated Python dict in web_server.py. The "What's New" panel pulled
its content from WHATS_NEW in helper.js. Same release notes, two files,
two languages, hand-edited at every release — drift was inevitable
(and happened: the kettui-fix entries I added recently differed in
detail between the two surfaces).

This commit makes helper.js the single editing surface:

- Adds VERSION_MODAL_SECTIONS const in helper.js right beside WHATS_NEW,
  with a comment block documenting the relationship: WHATS_NEW is the
  per-version detailed log used by the helper popover; VERSION_MODAL_SECTIONS
  is the curated highlight reel shown by the sidebar version button. Both
  edited at release time, both in the same file.
- Rewires showVersionInfo() in downloads.js to read from those consts
  directly. No backend round-trip; the changelog content ships in the
  same JS bundle the browser already loaded.
- Deletes the /api/version-info route and its 295-line version_data dict.
- Updates the line-39 comment to drop the now-stale "version-info endpoint"
  reference.

Note: this is collocation, not true unification. WHATS_NEW and
VERSION_MODAL_SECTIONS are still two distinct structures with overlapping
content, linked by a comment convention rather than a shared schema. A
deeper refactor (e.g. a `featured` flag on WHATS_NEW entries that the
modal aggregates) was rejected as out-of-scope — the curated section
titles ("Earlier in v2.3", "Recent Fixes") aren't 1:1 mappable to
WHATS_NEW entries. Saving for a follow-up if the drift problem persists.

Risk audit:
- Load order: helper.js loads at line 7967, downloads.js at line 7873.
  Both classic scripts execute synchronously before any clickable
  interaction, so showVersionInfo (only invoked on the version-button
  onclick) always sees both consts defined.
- populateVersionModal() unchanged — receives the same {title, subtitle,
  sections: [{title, description, features, usage_note?}]} shape.
- Stale-cache window during deploy: old downloads.js hitting a 404 on
  the deleted endpoint falls through to the existing catch + toast path
  ("Failed to load version information"). Cache-buster ?v=static_v
  resolves on next page load.

553 tests pass. helper.js + downloads.js parse cleanly. No residual
references to /api/version-info anywhere in the repo.
2026-04-26 13:32:30 -07:00
Broque Thomas
edb2c2044f Delete dead historical changelog strings from web_server.py
_OLD_V22_NOTES (655 lines) and _OLD_V2_NOTES (556 lines) were
triple-quoted Python strings holding old release-notes JSON. No code
references them — `grep _OLD_V22_NOTES|_OLD_V2_NOTES` returns only
the definitions themselves. They were leftover from earlier
version-info refactors and have been sitting in the file unread.

Pure deletion. No behavior change.
2026-04-26 13:22:07 -07:00
Broque Thomas
ac30e21b3d Sidebar version button: v2.3 → v2.4.0
Forgot to bump the hardcoded label in index.html during the 2.4.0
version commit. _getCurrentVersion() reads this textContent, so the
What's New surfacing logic was still seeing 2.3.
2026-04-26 10:20:02 -07:00
Broque Thomas
8ed6ccbb4e Bump version to 2.4.0 for dev → main release
- _SOULSYNC_BASE_VERSION → 2.4.0 (was 2.39).
- Migrate WHATS_NEW key '2.40' → '2.4.0', strip unreleased flags off
  the 27 entries shipping in this release, set release date.
- Replace parseFloat() version compare with proper int-tuple semver
  comparator — parseFloat('2.4.0') and parseFloat('2.4.1') both return
  2.4, which would have made future patch bumps invisible to the
  What's New surfacing logic.
2026-04-26 10:18:50 -07:00
BoulderBadgeDad
13b4578067
Merge pull request #379 from Nezreka/feat/reorganize-queue-and-status-panel
Library reorganize: FIFO queue with live status panel
2026-04-26 09:05:15 -07:00