YouTube now gates downloadable formats behind JS challenges (nsig); yt-dlp
needs a JavaScript runtime (Deno — its only default-enabled one) to solve
them, and its STABLE channel can lag months behind a breaking YouTube change.
Users hit "Requested format is not available" on every stream and music-video
download with no hint why. Neither piece is fixable via requirements.txt:
Deno isn't a Python package, and a version floor can only ever resolve stable.
- Dockerfile: install Deno in the runtime image (official installer,
auto-detects amd64/arm64, build fails early via `deno --version` if it
breaks; unzip added for the installer) and build with the yt-dlp NIGHTLY
channel (`pip install -U --pre "yt-dlp[default]"`) on top of requirements
- core/youtube_client.py: one-time startup warning when deno isn't on PATH,
naming the exact failure it causes and the install command — instead of
letting users debug a cryptic yt-dlp format error
- requirements.txt: annotate the yt-dlp line with the stable-lag caveat, the
nightly upgrade command, and the Deno requirement
- README: Deno + nightly notes in Prerequisites and the Python (No Docker)
install section; Docker bundles both automatically
- Verified end-to-end: fetch_public_playlist_full pulled all 236 tracks of the
test playlist via SpotipyFree (the library handles the client-auth that 429'd
the raw approach). Name + tracks correct.
- requirements.txt: declare spotipyFree>=1.1.2 as a normal pip dependency (like
spotDL, also MIT — aggregation, not vendored) + websockets (a transitive dep
SpotipyFree/spotapi needs that pip doesn't pull automatically). Code still
soft-imports + falls back to embed, so it's never a hard runtime requirement.
- meta fetch uses limit=1 (name/owner only) so we don't pull the whole list
twice. 9 tests green.
The in-house anonymous-token path is blocked by Spotify (429 without the web
player's rotating client-auth). Switch the full-fetch to SpotipyFree — the
maintained no-creds spotipy drop-in spotDL uses, which tracks that machinery.
- core/spotify_public_api.fetch_public_playlist_full now uses a SpotipyFree
client (playlist + playlist_items + next), normalising the spotipy-shaped
items to the embed scraper's shape. Injectable client_factory keeps it
unit-testable without the library or network. Dropped the dead in-house
token/pagination code.
- Licensing: SpotipyFree is GPL-3.0, so it is NOT bundled/required (SoulSync is
MIT). Optional, user-installed: the import is soft, and on ImportError (or any
failure) fetch_spotify_public falls back to the embed scraper (~100). So the
shipped project stays cleanly MIT and the link path never regresses.
- requirements.txt: documents it as a commented optional extra
(pip install SpotipyFree) with the GPL/MIT rationale.
- 9 tests: normalisation, pagination past 100, library-missing -> raises (->
fallback), and the embed-fallback orchestration.
Needs a live click-through with SpotipyFree installed to confirm the exact
class/method names match (SpotipyFree.Spotify / playlist / playlist_items).
PR 1 (commit 6ad85e27) shipped the ``next_run_at`` pure function as
foundation plumbing. PR 2 wires the engine through it and adds
``monthly_time`` as a real registered trigger type. After this PR
``core/automation_engine.py`` no longer has its own datetime
arithmetic for daily / weekly schedules — every next-run computation
flows through one function with one set of defensive fallbacks.
Net user-visible change: zero (no UI surface for monthly_time yet —
that's PR 3). New ``monthly_time`` trigger is reachable only via
direct API for now.
**Engine refactor:**
- ``_finish_run`` — collapsed three inline branches (daily_time
arithmetic, weekly_time arithmetic, fallback schedule arithmetic)
into a single ``next_run_at(...)`` call with ``_dt_to_db_str``
normalising the aware-UTC result to the engine's naive-UTC string
convention. Retry-delay short-circuit preserved. Exception
swallowing preserved (logged at debug, writes None next_run).
- ``_setup_daily_time_trigger`` + ``_setup_weekly_time_trigger`` +
new ``_setup_monthly_time_trigger`` — three near-identical methods
collapsed into one ``_setup_timed_trigger`` skeleton. Each public
method is now a one-line dispatch passing trigger_type to the
shared helper with a human-readable label for the debug log.
- Existing ``_next_weekly_occurrence`` deleted — its logic now lives
in ``core/automation/schedule.py:_next_weekly`` (lifted in PR 1).
- New ``_dt_to_db_str(dt)`` module-level helper normalises aware-UTC
→ naive-UTC string. Centralised so a tz mistake here surfaces in
one place. Aware non-UTC datetimes converted to UTC first
(defensive against a future bug that passes the wrong tz).
- New ``_resolve_system_default_tz()`` reads the server's local IANA
tz via ``tzlocal``. Cached at module import (the host's tz doesn't
change while the process runs). Falls back to UTC when ``tzlocal``
is missing — defensive for minimal Docker images.
- New ``self._default_tz`` engine attribute reads from
``automation.default_timezone`` config first, falls back to the
system-detected IANA name. Override path lets users on weird
setups pin a specific tz without touching env vars.
**Convergence fix (intentional behaviour change):**
Old ``_setup_daily_time_trigger`` / ``_setup_weekly_time_trigger``
didn't check the DB for an existing future ``next_run`` — they'd
recompute from scratch on every engine startup, overwriting manual
edits or pending retries. The interval path (``_setup_schedule_trigger``)
already had this check. The new shared ``_setup_timed_trigger``
brings daily / weekly in line: existing-future next_run wins over
freshly-computed delay. Treat this as a correctness fix, not a
breaking change — the old behaviour was an inconsistency, not a
deliberate choice.
**Backward-compat:**
- Existing ``schedule`` / ``daily_time`` / ``weekly_time`` rows
continue to work unchanged. The ``_trigger_handlers`` registry
keeps every historic key.
- Existing rows without an explicit ``tz`` field use
``self._default_tz`` (server-local IANA via ``tzlocal``) —
preserves "every Monday 09:00 server-local" behaviour on
non-UTC servers. Pre-fix the engine used naive
``datetime.now()`` which is also server-local; net effect is
identical wall-clock time, just routed through a tz-aware
pipeline that handles DST correctly (the May 2026 "next in 8h"
bug fix class).
- Engine boots even when ``tzlocal`` is missing — the resolver
falls back to UTC silently. Existing tests would catch a hard
dependency on tzlocal here.
**``tzlocal>=5.0`` added to requirements.txt** alongside
``tzdata>=2024.1`` from PR 1. Both libraries are small and stable;
``tzlocal`` returns a clean IANA name across Windows / Linux /
Docker, sidestepping the platform-specific tz detection mess.
**Tests:** 20 new in ``tests/automation/test_engine_schedule_integration.py``:
- ``_dt_to_db_str`` x3 (aware UTC, aware non-UTC converted to UTC,
naive assumed UTC)
- ``_resolve_system_default_tz`` x2 (returns IANA string, falls back
to UTC without tzlocal)
- ``_finish_run`` dispatch through next_run_at for each trigger type
(schedule, daily_time, weekly_time, monthly_time)
- Retry-delay short-circuits next_run_at
- next_run_at returns None → DB next_run cleared
- next_run_at raises → engine swallows + writes None
- Event triggers skipped (no scheduled next-run)
- ``self._default_tz`` passed through to next_run_at
- monthly_time registered in _trigger_handlers
- All historic trigger types kept registered
- ``_setup_monthly_time_trigger`` arms timer + writes DB
- ``_setup_timed_trigger`` honours existing future DB next_run
- Skip-with-log when next_run_at returns None
- End-to-end no-mock smoke for monthly_time
260 automation suite tests pass; the 240 from PR 1's branch plus 20
new integration tests. Ruff clean.
No WHATS_NEW entry — UI doesn't expose monthly_time yet (PR 3),
and the backward-compat path preserves existing daily/weekly
schedule timing.
Self-review pass on ec4a55c1 — applying the standing kettui-grade
rule (see memory/feedback_always_build_kettui_grade.md). Three issues
that would have surfaced on review:
1. Silent tz fallback to UTC
``_resolve_tz`` returned UTC when the IANA name was unknown — no
log, no warning. User on a host without ``tzdata`` who configures
``America/Los_Angeles`` got schedules running silently at UTC
offset with no way to debug. Now logs WARNING once per unknown
name (deduped via ``_UNKNOWN_TZ_WARNED`` set so a misconfigured
row doesn't spam every poll cycle) and the log line names BOTH
real causes — typo or missing tzdata — so the user can fix from
a single grep.
2. ``weeks`` unit drift from engine
I added ``'weeks': 86400*7`` to ``_INTERVAL_MULTIPLIERS`` but the
engine's existing ``_calc_delay_seconds`` only recognises
minutes/hours/days. Until PR 2 collapses both paths through this
function, any row whose config snuck through with ``unit='weeks'``
would get scheduled by the engine as 1-hour and by this function
as 7-day — drift between two live implementations. Dropped
``weeks`` from the map to match the engine. Added a comment
pinning the map to the engine's contract and a regression test
that asserts ``unit='weeks'`` falls back to the same hours
default the engine produces.
3. DST edge cases unverified
The module docstring claims DST-aware via ``zoneinfo`` but no test
pinned the spring-forward gap (02:30 LA on DST-Sunday doesn't
exist) or fall-back ambiguity (01:30 LA on fall-Sunday happens
twice). Three new tests:
- ``test_dst_spring_forward_lands_after_the_gap`` — pins that the
function doesn't crash + lands on a real instant past ``now``.
- ``test_dst_fall_back_handles_ambiguous_local_time`` — pins
zoneinfo's default-earlier-instant resolution for ambiguous
local times (01:30 PDT vs 01:30 PST → picks PDT).
- ``test_weekly_across_dst_boundary_keeps_local_wall_clock`` —
pins that a "every Sunday at 09:00 LA" schedule keeps the
local wall clock across the boundary even though the UTC
equivalent shifts by an hour. This is the exact bug class
that caused the May 2026 "next in 8h" tz mismatch.
Also loosened ``tzdata==2026.2`` to ``tzdata>=2024.1``. IANA tz data
changes a few times a year for real-world DST policy updates; pinning
to one snapshot would freeze the app's tz knowledge to the build date
and miss future government-mandated rule changes.
41 schedule tests pass (5 new); 240 across the full automation suite.
Ruff clean.
Backend plumbing for upcoming weekly + monthly Auto-Sync schedules.
PR 1 of 4 in the schedule-types feature — see
``memory/project_auto_sync_schedule_types.md`` for the full plan.
Net behaviour change in this PR: zero. The automation engine still
computes next_run via its existing inline ``_calc_delay_seconds`` /
``_next_weekly_occurrence`` helpers; this module is unused until PR 2
wires the engine through. Lands separately so the foundation can sit
on dev for a beat before the engine change.
``core/automation/schedule.py:next_run_at(trigger_type, trigger_config,
now_utc, default_tz)``:
- Pure function. ``now_utc`` injected (tests freeze time without
monkeypatching ``datetime.now``); ``default_tz`` injected (so daily /
weekly / monthly schedules compute against the USER's timezone, not
the server's — the same class of bug that produced the May 2026
"Auto-Sync next in 8h" timezone fix).
- Returns aware-UTC ``datetime`` ready to serialise to the DB
``next_run`` column, or ``None`` for unrecognised / event-based
triggers (callers should not write a next_run for those).
- Naive ``now_utc`` inputs are assumed UTC for defensive symmetry
with the engine's DB-string parser convention.
Trigger types covered:
- ``schedule``: ``{interval: N, unit: 'minutes'|'hours'|'days'|'weeks'}``
— matches engine's existing ``_calc_delay_seconds``. Unknown unit
defaults to hours; zero/negative interval clamps to 1 (preserves
the engine's guard against scheduling for the past); non-numeric
interval falls back to 1.
- ``daily_time``: ``{time: 'HH:MM', tz: '<IANA>'}`` — DST-aware via
``zoneinfo``; ``tz`` falls back to ``default_tz``; unknown IANA
string falls back to UTC; garbage ``time`` falls back to 00:00.
- ``weekly_time``: ``{time, days: ['mon',...], tz}`` — empty / all-
invalid ``days`` list means "every day" (matches engine fallback);
abbreviations case-insensitive; 8-day scan finds the next match.
- ``monthly_time``: ``{time, day_of_month: 1-31, tz}`` — NEW shape.
Day clamped to [1, 31]. Months too short for the target day clamp
to the LAST valid day rather than skipping a month (standard cron
convention; running a day early in February is less surprising
than missing the whole month). 12-iteration loop cap so a
pathological config can't infinite-loop.
Tests (36 cases, all passing):
- Interval: every unit, unknown-unit fallback, zero/negative/garbage
interval clamp, tz field ignored on interval (wall-clock-independent).
- Daily: today-at-future-time runs today, today-at-past-time rolls to
tomorrow, exact-match rolls to tomorrow (no schedule-now-then-schedule-
again-immediately), user-tz vs server-tz, default_tz fallback,
garbage time / unknown tz defensive returns.
- Weekly: same-day-still-future qualifies, same-day-past rolls to next
allowed day, wraps across week boundary, empty days = every day,
garbage abbreviations dropped, case-insensitive, tz across day
boundary (LA Wednesday evening is Thursday UTC).
- Monthly: target day this month, rolls to next month when passed,
Feb 31 → Feb 28 / Feb 29 leap year, day_of_month above 31 / below
1 clamp, Dec → Jan year roll, user-tz pre-midnight edge case.
- Result-shape contract: every returned datetime is aware UTC at
offset zero (engine relies on this when serialising to the
``next_run`` string column).
Added ``tzdata==2026.2`` to requirements.txt. Windows ``zoneinfo`` and
minimal Docker base images ship without the system tz database;
without ``tzdata`` ``ZoneInfo('America/Los_Angeles')`` raises
``ZoneInfoNotFoundError`` and the helper silently falls back to UTC.
No WHATS_NEW entry — no user-visible behaviour change in this PR.
PR 2 (engine wire-through) will land the user-facing changelog entry
when ``monthly_time`` becomes a real schedulable trigger.
User reported (eN1gma) the dev nightly Docker image fails to start
with ``ModuleNotFoundError: No module named 'requests'`` despite
``requests>=2.31.0`` being correctly listed in requirements.txt.
Local Docker builds + python imports both work — the issue is a
poisoned GHA Docker layer cache: the ``pip install -r requirements.txt``
step is cached based on the file's content hash, so once a bad
layer (e.g. an aborted/incomplete pip install from a previous run)
makes it into the cache, every subsequent build reuses it.
Touching this comment changes the requirements.txt hash, which
forces ``cache-from: type=gha`` in dev-nightly.yml to skip the
poisoned layer and run a fresh ``pip install``. The next dev nightly
build (or push-to-dev triggered build) will produce a clean image.
No functional change.
Self-review nits on PR #384:
- requirements.txt: 5-line comment for one pin → 1 line. Rationale
lives in commit body and #367; no need to repeat in-tree.
- helper.js: dropped `page: 'settings'` from the yt-dlp WHATS_NEW
entry. Settings page has no yt-dlp UI; the link would have
navigated users somewhere irrelevant.
553 tests pass.
Closes#367 (reported by JohnBaumb).
The Docker entrypoint ran `pip install -U yt-dlp --quiet --no-cache-dir`
on every container start. Three problems with that:
- Non-deterministic startup: each restart could pick up a different
yt-dlp version, making "works on my machine" debugging harder.
- Network dependency at boot: PyPI being slow/unreachable gated the
app coming up.
- In-place upgrades inside running containers can race with active
yt-dlp invocations and aren't a great pattern.
Picked Option A from the issue: pin to an exact version in
requirements.txt (`yt-dlp==2026.3.17`) and remove the entrypoint
install entirely. yt-dlp comes baked into the image now via the
existing `pip install -r requirements.txt` in the Dockerfile.
Tradeoff: YouTube fixes ship via SoulSync releases now instead of
"next container restart". The pin is documented inline with how to
bump it.
Net change: -3 entrypoint lines, requirements.txt pin tightened,
WHATS_NEW '2.4.1' block opened (entries hidden until version bumps).
553 tests pass.
Switch the web UI from Werkzeug's built-in server to Gunicorn for a more stable production deployment path.
Keep a separate dev config so local runs still reload quickly, while the production path uses a dedicated WSGI entrypoint and cleaner startup behavior.
The main motivation is to reduce the websocket teardown noise and make the server behavior more predictable under the app's mostly background-driven workload.
Adds a full public REST API at /api/v1/ with 32 endpoints covering library, search, downloads, wishlist, watchlist, playlists, system status, and settings. Includes API key authentication (Bearer token), per-endpoint rate limiting, and consistent JSON response format. API keys can be generated and managed from the Settings page. No changes to existing functionality — the API delegates to the same backend services the web UI uses.
Add optional post-download audio fingerprint verification using AcoustID.
Downloads are verified against expected track/artist using fuzzy string
matching on AcoustID results. Mismatched files are quarantined and
automatically added to the wishlist for retry.
- AcoustID verification with title/artist fuzzy matching (not MBID comparison)
- Quarantine system with JSON metadata sidecars for failed verifications
- fpcalc binary auto-download for Windows, macOS (universal), and Linux
- MusicBrainz enrichment worker with live status UI and track badges
- Settings page AcoustID section with real-fingerprint connection test
- Source reuse for album downloads to keep tracks from same Soulseek user
- Enhanced search queries for better track matching
- Bug fixes: wishlist tracking, album splitting, regex & handling, log rotation
