soulsync/core/automation/handlers/database_update.py
Broque Thomas 017553193f Extract automation handlers (3/3): maintenance + misc, finishing the lift
Final commit of the automation-handler refactor. With this commit
every closure that used to live in
`web_server._register_automation_handlers` is now a top-level
function in `core/automation/handlers/`.

Handlers extracted in this commit:

- start_database_update + deep_scan_library
    -> core/automation/handlers/database_update.py
    Both share the db_update_state monitoring pattern (poll until
    status flips, stall detection emits warning at 10 min, 2-hour
    outer timeout). Lifted into a shared `_run_with_progress` helper
    inside the module so the per-handler bodies stay tiny.

- run_duplicate_cleaner -> core/automation/handlers/duplicate_cleaner.py
- start_quality_scan    -> core/automation/handlers/quality_scanner.py

- clear_quarantine, cleanup_wishlist, update_discovery_pool,
  backup_database, refresh_beatport_cache
    -> core/automation/handlers/maintenance.py
    Grouped because each body is short (~20-50 lines) and they share
    no state — splitting into per-handler files would just add import
    noise.

- clean_search_history, clean_completed_downloads, full_cleanup
    -> core/automation/handlers/download_cleanup.py
    Grouped because all three reach the download orchestrator,
    tasks_lock, and download_batches/download_tasks accessors. The
    full_cleanup multi-step orchestration shares phase-detection
    logic with clean_completed_downloads.

- run_script         -> core/automation/handlers/run_script.py
- search_and_download -> core/automation/handlers/search_and_download.py

`AutomationDeps` grew with the new dependency surface:
- get_db_update_state + db_update_lock + db_update_executor +
  run_db_update_task + run_deep_scan_task
- get_duplicate_cleaner_state + duplicate_cleaner_lock +
  duplicate_cleaner_executor + run_duplicate_cleaner
- get_quality_scanner_state + quality_scanner_lock +
  quality_scanner_executor + run_quality_scanner
- download_orchestrator + run_async + tasks_lock +
  get_download_batches + get_download_tasks +
  sweep_empty_download_directories + get_staging_path
- docker_resolve_path + get_current_profile_id +
  get_watchlist_scanner + get_app + get_beatport_data_cache
- set_db_update_automation_id (writes the legacy global so the live
  DB-update progress callbacks still living in web_server.py keep
  emitting against the active automation card)

`web_server._register_automation_handlers` is now ~50 lines: build
deps once, call register_all. The 667-line block of remaining
closure definitions and engine register calls is gone.

The final orphan was the `_db_update_automation_id` module global —
the DB-update progress callbacks at line ~14080 still read it
directly, so the extracted database_update handler propagates the
automation id through `deps.set_db_update_automation_id` (a closure
in web_server that writes the global). When the legacy callbacks
get extracted in a future PR the setter goes away.

Tests:
- tests/automation/test_handlers_maintenance.py adds 21 boundary
  tests covering every newly-extracted handler shape: guard
  short-circuits (already-running returns skipped), deps wiring
  (set_db_update_automation_id called with the right id),
  exception swallow contract, status returns, path-traversal
  blocked in run_script, source-mode skip in clean_search_history,
  active-batch skip in clean_completed_downloads, etc.
- 3244 tests pass (was 3223 — 21 new), no regression.

web_server.py: 35,593 -> 34,220 lines (-1,373 net across 3 commits).
Issue #1 from the extraction punch list is now COMPLETE.
2026-05-15 11:24:35 -07:00

136 lines
5.3 KiB
Python

"""Automation handlers: ``start_database_update`` and
``deep_scan_library`` actions.
Lifted from ``web_server._register_automation_handlers`` (the
``_auto_start_database_update`` and ``_auto_deep_scan_library``
closures). Both share the same ``db_update_state`` / executor / lock
infrastructure -- the only difference is which task they submit
(``run_db_update_task`` vs ``run_deep_scan_task``).
Pattern: pre-set state to running, submit task to executor, then
poll the state dict until it transitions away from ``running``.
Stall-detection emits a warning every 10 minutes when progress
hasn't budged. 2-hour outer timeout caps the worst case.
"""
from __future__ import annotations
import time
from typing import Any, Dict
from core.automation.deps import AutomationDeps
_TIMEOUT_SECONDS = 7200 # 2 hours — covers the worst large-library case
_STALL_WARNING_SECONDS = 600 # 10 minutes without progress = stall
_POLL_INTERVAL_SECONDS = 3
_INITIAL_DELAY_SECONDS = 1
def auto_start_database_update(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
"""Run a full or incremental DB update via ``run_db_update_task``."""
return _run_with_progress(
config, deps,
task=deps.run_db_update_task,
task_args=(config.get('full_refresh', False), deps.config_manager.get_active_media_server()),
initial_phase='Initializing...',
stall_label='Database update',
finished_extras=lambda: {'full_refresh': str(config.get('full_refresh', False))},
timeout_label='Database update timed out after 2 hours',
)
def auto_deep_scan_library(config: Dict[str, Any], deps: AutomationDeps) -> Dict[str, Any]:
"""Run a deep library scan via ``run_deep_scan_task``."""
return _run_with_progress(
config, deps,
task=deps.run_deep_scan_task,
task_args=(deps.config_manager.get_active_media_server(),),
initial_phase='Deep scan: Initializing...',
stall_label='Deep scan',
finished_extras=lambda: {},
timeout_label='Deep scan timed out after 2 hours',
)
def _run_with_progress(
config: Dict[str, Any],
deps: AutomationDeps,
*,
task,
task_args: tuple,
initial_phase: str,
stall_label: str,
finished_extras,
timeout_label: str,
) -> Dict[str, Any]:
"""Shared poll-and-wait body for both DB-update handlers."""
automation_id = config.get('_automation_id')
state = deps.get_db_update_state()
if state.get('status') == 'running':
return {'status': 'skipped', 'reason': 'Database update already running'}
deps.state.db_update_automation_id = automation_id
# Sync legacy module global so the DB-update progress callbacks
# (still living in web_server.py) emit against this automation.
deps.set_db_update_automation_id(automation_id)
with deps.db_update_lock:
state.update({
'status': 'running', 'phase': initial_phase,
'progress': 0, 'current_item': '', 'processed': 0, 'total': 0,
'error_message': '',
})
deps.db_update_executor.submit(task, *task_args)
# Monitor progress (callbacks handle card updates, we just block until done).
time.sleep(_INITIAL_DELAY_SECONDS)
poll_start = time.time()
last_progress_time = time.time()
last_progress_val = 0
while time.time() - poll_start < _TIMEOUT_SECONDS:
time.sleep(_POLL_INTERVAL_SECONDS)
with deps.db_update_lock:
current_status = state.get('status', 'idle')
current_progress = state.get('progress', 0)
if current_status != 'running':
break
# Stall detection — if no progress change in 10 minutes, warn.
if current_progress != last_progress_val:
last_progress_val = current_progress
last_progress_time = time.time()
elif time.time() - last_progress_time > _STALL_WARNING_SECONDS:
deps.update_progress(
automation_id,
log_line=f'{stall_label} appears stalled — waiting...',
log_type='warning',
)
last_progress_time = time.time() # Reset so warning repeats every 10 min.
else:
# 2-hour timeout reached.
deps.update_progress(
automation_id, status='error',
phase='Timed out', log_line=timeout_label, log_type='error',
)
return {'status': 'error', 'reason': 'Timed out', '_manages_own_progress': True}
# Finished/error callback already updated the card — return matching status.
with deps.db_update_lock:
final_status = state.get('status', 'unknown')
if final_status == 'error':
return {
'status': 'error',
'reason': state.get('error_message', 'Unknown error'),
'_manages_own_progress': True,
}
with deps.db_update_lock:
stats = {
'status': 'completed', '_manages_own_progress': True,
'artists': state.get('total', 0),
'albums': state.get('total_albums', 0),
'tracks': state.get('total_tracks', 0),
'removed_artists': state.get('removed_artists', 0),
'removed_albums': state.get('removed_albums', 0),
'removed_tracks': state.get('removed_tracks', 0),
}
stats.update(finished_extras())
return stats