diff --git a/README.md b/README.md index f4f679f6..23139412 100644 --- a/README.md +++ b/README.md @@ -4,152 +4,83 @@ # SoulSync - Intelligent Music Discovery & Automation Platform -**Bring Spotify-quality music discovery to your self-hosted library.** SoulSync automates music collection with intelligent discovery algorithms, multi-source downloads, and zero manual intervention. +**Spotify-quality music discovery for self-hosted libraries.** Automates downloads, curates playlists, monitors artists, and organizes your collection with zero manual effort. -> **IMPORTANT**: Configure file sharing in slskd before use. The Soulseek community bans users who only download without sharing. Set up shared folders at `http://localhost:5030/shares`. +> **IMPORTANT**: Configure file sharing in slskd to avoid Soulseek bans. Set up shared folders at `http://localhost:5030/shares`. -> **Development Status**: New features are developed for the **Web UI**. The Desktop GUI receives maintenance and bug fixes only. - -**Community**: [Discord Server](https://discord.gg/ePx7xYuV) | **Support**: [GitHub Issues](https://github.com/Nezreka/SoulSync/issues) | **Donate**: [Ko-fi](https://ko-fi.com/boulderbadgedad) +**Community**: [Discord](https://discord.gg/ePx7xYuV) | **Support**: [GitHub Issues](https://github.com/Nezreka/SoulSync/issues) | **Donate**: [Ko-fi](https://ko-fi.com/boulderbadgedad) --- -## The Problem +## What It Does -You want Spotify's discovery features (Release Radar, Discovery Weekly, personalized playlists) but for your local music library. Existing tools require manual playlist management or lack intelligent discovery. +SoulSync bridges streaming services to your media server with automated discovery: -## The Solution - -SoulSync bridges streaming services to your self-hosted media server with **automated discovery and collection**: - -1. **Monitors artists** for new releases automatically -2. **Generates personalized playlists** using custom recommendation algorithms -3. **Downloads missing tracks** from Soulseek with FLAC priority -4. **Enriches metadata** with lyrics, album art, and proper tags -5. **Organizes files** using customizable templates -6. **Syncs with Plex/Jellyfin/Navidrome** automatically +1. **Monitors artists** → Automatically detects new releases +2. **Generates playlists** → Release Radar, Discovery Weekly, Seasonal, Decade/Genre mixes +3. **Downloads missing tracks** → From Soulseek, Beatport charts, playlists +4. **Enriches metadata** → LRC lyrics, album art, proper tags +5. **Organizes files** → Custom templates for clean folder structures +6. **Syncs media server** → Plex, Jellyfin, or Navidrome stay updated --- ## Key Features -### Intelligent Discovery System +### Discovery Engine -**Release Radar** - 30 new tracks from your watchlist, updated daily -- Monitors 100+ artists automatically -- Only includes releases from the last 30 days -- Curated using popularity scoring +**Release Radar** - 30 new tracks from watchlist artists (updates daily) -**Discovery Weekly** - 50 tracks from similar artists you don't own -- Custom algorithm: 20 popular + 20 mid-tier + 10 deep cuts +**Discovery Weekly** - 50 tracks from similar artists using custom algorithm +- 20 popular + 20 mid-tier + 10 deep cuts - Built from 1000+ track discovery pool -- Updated every 24 hours +- Refreshes every 24 hours -**Seasonal Playlists** - Auto-generated themed collections -- Halloween, Christmas, Valentine's Day, Summer, Spring, Autumn -- Smart keyword matching and genre analysis -- Active during appropriate months +**Seasonal Playlists** - Halloween, Christmas, Valentine's, Summer, Spring, Autumn (auto-generated) **Personalized Playlists** (12+ types) - Recently Added, Top Tracks, Forgotten Favorites -- Decade Playlists (1960s-2020s) -- Genre Playlists (15 parent categories, 50+ sub-genres) -- Daily Mixes, Hidden Gems, Popular Picks -- Custom Playlist Builder (seed with artists) +- Decade Playlists (1960s-2020s), Genre Playlists (15 categories) +- Daily Mixes, Hidden Gems, Popular Picks, Custom Builder -**ListenBrainz Integration** -- Import recommendation playlists -- Sync user and collaborative playlists -- Access community-curated content +**ListenBrainz** - Import recommendation and community playlists -**Beatport Integration** - Electronic music discovery -- Browse by genre (House, Techno, Trance, Drum & Bass, etc.) -- Top 100, Hype Charts, DJ Charts -- Staff Picks, New Releases, Latest Tracks -- Per-genre hero sections and featured content +**Beatport** - Electronic music charts by genre (House, Techno, Trance, etc.) +- Top 100, Hype Charts, DJ Charts, Staff Picks ### Multi-Source Downloads -**Primary Sources** -- **Soulseek**: FLAC-priority with automatic quality selection -- **Beatport Charts**: Electronic music with Spotify matching -- **Spotify Playlists**: Public and private playlist sync -- **Tidal Playlists**: Alternative streaming source -- **YouTube Playlists**: Fallback option +**Sources**: Soulseek (FLAC priority), Beatport charts, Spotify/Tidal/YouTube playlists -**Smart Download Pipeline** +**Features** - Quality profiles: Audiophile, Balanced, Mobile -- Automatic format fallback (FLAC → MP3 → other) -- Duplicate prevention against existing library -- Batch processing with concurrent downloads -- Automatic retry on failure (30-minute intervals) +- Automatic format fallback (FLAC → MP3) +- Duplicate prevention against library +- Batch processing with retry logic +- Synchronized lyrics (LRC) for every track -### Advanced Matching Engine +### Advanced Matching -**Text Normalization** -- Unicode handling (handles KoЯn, Björk, etc.) -- Special character preservation (A$AP Rocky) -- Accent normalization (Beyoncé → Beyonce) -- Abbreviation expansion (feat. → featured, pt. → part) +- Unicode/accent handling (KoЯn, Björk, A$AP Rocky) +- Fuzzy matching with confidence scoring +- Album variation detection (Deluxe, Remastered, etc.) +- Multi-strategy: exact → normalized → fallback -**Fuzzy Matching** -- Multi-strategy: exact → normalized → Unicode fallback -- Album variation handling (Deluxe, Remastered, Platinum Edition) -- Artist name preservation (doesn't break "Daryl Hall & John Oates") -- Confidence scoring with configurable thresholds +### Automation -### Automation & Monitoring +**Watchlist** - Monitor unlimited artists, auto-discover similar artists via music-map.com -**Watchlist System** -- Monitor unlimited artists for new releases -- Automatic similar artist discovery via music-map.com -- Occurrence-based ranking (tracks artist overlap) -- Configurable scan intervals +**Wishlist** - Failed downloads retry every 30 minutes automatically -**Wishlist System** -- Tracks failed downloads automatically -- Auto-retry every 30 minutes -- Granular management (remove tracks or entire albums) -- Source tracking (playlist, album, manual) - -**Background Tasks** -- Database synchronization with media server -- Discovery pool population (50 artists × 10 releases) -- Seasonal content updates -- Library completion tracking +**Background Tasks** - Database sync, discovery pool updates, seasonal content ### Library Management -**Tools** -- **Quality Scanner**: Find low-bitrate files to replace -- **Duplicate Cleaner**: Identify redundant tracks -- **Completion Tracking**: See album progress percentages -- **Enhanced Search**: Unified search across Spotify, library, and Soulseek - -**Metadata Enhancement** -- Synchronized lyrics (LRC format) via LRClib.net -- Album art embedding -- Proper ID3/Vorbis tags -- Custom file organization templates - -**File Organization** -- Template-based paths: `$albumartist/$album/$track - $title` -- Separate templates for albums, singles, playlists -- Client-side validation -- Automatic fallback on errors - -### Media Server Integration - -**Supported Servers** -- Plex (with library selection) -- Jellyfin (with multi-library support) -- Navidrome - -**Features** -- Automatic library scanning after downloads -- Database caching for fast access -- Incremental updates -- Connection testing and validation +- **Quality Scanner** - Find low-bitrate files to replace +- **Duplicate Cleaner** - Identify redundant tracks +- **Completion Tracking** - Album progress percentages +- **Enhanced Search** - Unified search across Spotify, library, Soulseek +- **Template Organization** - `$albumartist/$album/$track - $title` (fully customizable) --- @@ -158,17 +89,12 @@ SoulSync bridges streaming services to your self-hosted media server with **auto ### Docker (Recommended) ```bash -# Using docker-compose curl -O https://raw.githubusercontent.com/Nezreka/SoulSync/main/docker-compose.yml docker-compose up -d - -# Or run directly -docker run -d -p 8008:8008 boulderbadgedad/soulsync:latest - # Access at http://localhost:8008 ``` -### Python (Web UI) +### Python ```bash git clone https://github.com/Nezreka/SoulSync @@ -178,376 +104,148 @@ python web_server.py # Open http://localhost:8008 ``` -### Desktop GUI - -```bash -git clone https://github.com/Nezreka/SoulSync -cd SoulSync -pip install -r requirements.txt -python main.py -``` - --- ## Quick Setup ### Prerequisites -- **slskd** running on port 5030 ([Download](https://github.com/slskd/slskd/releases)) -- **Spotify API** credentials ([Developer Dashboard](https://developer.spotify.com/dashboard)) -- **Tidal API** (optional) ([Developer Dashboard](https://developer.tidal.com/dashboard)) +- **slskd** on port 5030 ([Download](https://github.com/slskd/slskd/releases)) +- **Spotify API** credentials ([Dashboard](https://developer.spotify.com/dashboard)) - **Media Server** (optional): Plex, Jellyfin, or Navidrome -### API Configuration +### Configuration -**Spotify** -1. Create app at [Developer Dashboard](https://developer.spotify.com/dashboard) -2. Add redirect URI: `http://127.0.0.1:8888/callback` -3. Copy Client ID and Secret +1. **Spotify API** + - Create app → Add redirect: `http://127.0.0.1:8888/callback` + - Copy Client ID and Secret -**Tidal** (optional) -1. Create app at [Developer Dashboard](https://developer.tidal.com/dashboard) -2. Add redirect URI: `http://127.0.0.1:8889/callback` -3. Add scopes: `user.read`, `playlists.read` -4. Copy Client ID and Secret +2. **SoulSync Settings** + - Enter API credentials + - Configure slskd URL and API key + - Set download/transfer paths + - Connect media server (optional) + - **Configure slskd file sharing to avoid bans** -**Plex** (optional) -- Get token from media item URL: `?X-Plex-Token=YOUR_TOKEN` -- Server URL: `http://YOUR_IP:32400` - -**Jellyfin** (optional) -- Settings → API Keys → Generate new key -- Server URL: `http://YOUR_IP:8096` - -**Navidrome** (optional) -- Settings → Users → Generate API Token -- Server URL: `http://YOUR_IP:4533` - -### Initial Configuration - -1. Launch SoulSync and navigate to Settings -2. Enter API credentials for streaming services -3. Configure media server connection (if using) -4. Set slskd URL (`http://localhost:5030`) and API key -5. Configure download path and transfer path -6. Customize file organization templates (optional) -7. **Configure file sharing in slskd to avoid bans** - -### Docker-Specific Setup - -**Path Mapping** -```yaml -volumes: - - ./config:/app/config # Settings persist - - ./logs:/app/logs # Log files - - /mnt/c:/host/mnt/c:rw # Mount Windows drives - - /mnt/d:/host/mnt/d:rw -``` - -Use `/host/mnt/X/path` in settings where X is your drive letter. - -**OAuth Authentication from Remote Devices** - -Due to Spotify API requirements (127.0.0.1 mandatory, localhost banned), remote OAuth needs a workaround: - -1. Complete OAuth flow - redirected to `http://127.0.0.1:8888/callback?code=...` -2. Manually edit URL to your server IP: `http://192.168.1.5:8888/callback?code=...` -3. Press Enter to complete authentication - -See [DOCKER-OAUTH-FIX.md](DOCKER-OAUTH-FIX.md) for details. +3. **Docker OAuth Fix** (if accessing from remote device) + - Redirected to `http://127.0.0.1:8888/callback?code=...` + - Manually edit URL to server IP: `http://192.168.1.5:8888/callback?code=...` + - Spotify requires 127.0.0.1 (banned localhost Nov 2025) + - See [DOCKER-OAUTH-FIX.md](DOCKER-OAUTH-FIX.md) --- -## How It Works +## Who Should Use This -### Discovery Pipeline +**Perfect for:** +- Self-hosters with Plex/Jellyfin/Navidrome +- Music enthusiasts with 500+ album collections +- Electronic music fans (Beatport integration) +- Former Spotify users wanting local discovery -1. **Add artists to watchlist** → SoulSync monitors for new releases -2. **Fetch similar artists** → music-map.com provides 10 similar artists per watchlist artist -3. **Aggregate by occurrence** → Ranks similar artists by how many watchlist artists recommend them -4. **Build discovery pool** → Top 50 similar artists × 10 recent releases = ~500 albums -5. **Extract tracks** → Pool contains 1000-2000 tracks, rolling 1-year window -6. **Curate playlists** → Algorithms generate Release Radar, Discovery Weekly, Seasonal - -### Download Workflow - -1. **Source Selection** → User picks playlist, album, or uses discovery features -2. **Library Matching** → SoulSync checks existing library to avoid duplicates -3. **Quality Filtering** → Applies user-defined quality profile (FLAC priority) -4. **Download Queue** → Batches requests with configurable concurrency (default: 3) -5. **Metadata Enhancement** → Adds lyrics (LRC), album art, proper tags -6. **File Organization** → Moves to transfer folder using custom templates -7. **Media Server Sync** → Triggers library rescan, updates internal database - -### Automation Loop - -- **Every 30 minutes**: Wishlist retry for failed downloads -- **Every 24 hours**: Discovery pool refresh, playlist curation -- **On media server scan**: Database incremental update -- **On new release**: Watchlist triggers download if configured +**Not ideal for:** +- Casual users wanting simple sync +- Slow/metered internet connections +- Users uncomfortable with APIs or Docker --- -## Who Should Use SoulSync - -### Perfect For - -- **Self-hosters** with Plex, Jellyfin, or Navidrome libraries -- **Music enthusiasts** with 500+ album collections who want automated discovery -- **Electronic music fans** who follow Beatport charts -- **Former Spotify users** who want discovery features for local files -- **Power users** comfortable with API configuration and Docker - -### Not Ideal For - -- Casual users wanting simple one-click playlist sync -- Users on slow/metered internet (download-heavy workflow) -- People uncomfortable with terminal commands or API keys -- Those seeking streaming-only solutions (not a media server replacement) - ---- - -## Comparison with Alternatives +## Comparison | Feature | SoulSync | Lidarr | Headphones | Beets | |---------|----------|--------|------------|-------| -| **Custom Discovery Algorithm** | ✓ | ✗ | ✗ | ✗ | -| **Personalized Playlists** | 12+ types | Manual lists | ✗ | ✗ | -| **Beatport Integration** | ✓ (charts, genres) | ✗ | ✗ | ✗ | -| **ListenBrainz Playlists** | ✓ | ✗ | ✗ | ✗ | -| **Multi-Source Downloads** | Spotify/Tidal/YouTube | MusicBrainz | ✗ | ✗ | -| **Watchlist Monitoring** | ✓ (100+ artists) | ✓ (manual add) | ✓ | ✗ | -| **LRC Lyrics** | ✓ (auto) | ✗ | ✗ | Plugin | -| **Advanced Matching** | Unicode, fuzzy, confidence | Basic | Basic | ✓ | -| **Quality Scanner** | ✓ | ✗ | ✗ | ✓ | -| **Duplicate Cleaner** | ✓ | ✗ | ✗ | ✓ | -| **Web UI** | Modern Flask | ✓ | Basic | CLI only | -| **Template-Based Organization** | ✓ | ✗ | ✗ | ✓ | -| **Seasonal Playlists** | ✓ (auto) | ✗ | ✗ | ✗ | +| Custom Discovery Algorithm | ✓ | ✗ | ✗ | ✗ | +| Personalized Playlists (12+) | ✓ | ✗ | ✗ | ✗ | +| Beatport Integration | ✓ | ✗ | ✗ | ✗ | +| ListenBrainz Playlists | ✓ | ✗ | ✗ | ✗ | +| Multi-Source (Spotify/Tidal/YouTube) | ✓ | ✓ | ✗ | ✗ | +| Watchlist Monitoring | ✓ (100+) | ✓ | ✓ | ✗ | +| LRC Lyrics | ✓ | ✗ | ✗ | Plugin | +| Advanced Matching | ✓ | ✗ | ✗ | ✓ | +| Quality Scanner + Duplicate Cleaner | ✓ | ✗ | ✗ | ✓ | +| Template-Based Organization | ✓ | ✗ | ✗ | ✓ | +| Seasonal Playlists | ✓ | ✗ | ✗ | ✗ | -**SoulSync's Unique Position**: Only tool combining intelligent discovery (Release Radar, Discovery Weekly) with multi-source automation (Beatport charts, ListenBrainz) and self-hosted library management. +**SoulSync is the only tool combining intelligent discovery with multi-source automation and library management.** --- ## Architecture -### Technical Stack +**Scale**: 83,000+ lines Python, 120+ API endpoints, handles 10,000+ album libraries -- **Language**: Python 3.8+ -- **Web Framework**: Flask with 120+ API endpoints -- **Database**: SQLite with connection pooling and indexing -- **UI**: Modern JavaScript with real-time updates -- **Desktop**: PyQt6 (maintenance mode) +**Integrations**: Spotify, Tidal, Plex, Jellyfin, Navidrome, Slskd, ListenBrainz, LRClib, music-map.com, Beatport -### Service Integrations +**Stack**: Python 3.8+, Flask, SQLite, PyQt6 (desktop GUI in maintenance mode) -- **Spotify API**: Artist monitoring, playlist sync, metadata -- **Tidal API**: Alternative playlist source -- **Plex API**: Library scanning, metadata sync -- **Jellyfin API**: Multi-library support -- **Navidrome API**: Subsonic-compatible server -- **Slskd API**: Download management, search -- **ListenBrainz API**: Community playlists, recommendations -- **LRClib.net**: Synchronized lyrics -- **music-map.com**: Similar artist discovery - -### Core Components - -**Matching Engine** (`core/matching_engine.py`) -- Text normalization with Unicode support -- Fuzzy string matching with confidence scoring -- Album variation handling -- Special character preservation - -**Discovery System** (`core/watchlist_scanner.py`, `core/personalized_playlists.py`) -- Watchlist monitoring -- Discovery pool population -- Playlist curation algorithms -- Seasonal content generation - -**Download Pipeline** (`core/soulseek_client.py`, `services/sync_service.py`) -- Quality profile filtering -- Concurrent download management -- Automatic retry logic -- Batch processing - -**Metadata Enhancement** (`core/lyrics_client.py`) -- LRC lyrics fetching -- Album art embedding -- Tag normalization -- File organization - -### Database Schema - -- **Tracks**: Full track metadata with file paths -- **Albums**: Album info with completion tracking -- **Artists**: Artist profiles with watchlist status -- **Discovery Pool**: 1000-2000 track rotating pool -- **Seasonal Content**: Cached seasonal albums/tracks -- **Wishlist**: Failed downloads with retry tracking -- **Similar Artists**: Occurrence-ranked recommendations - -### Scale - -- **83,000+ lines** of Python code -- **120+ API endpoints** -- **15+ service clients** -- **20+ database tables** -- **Handles libraries of 10,000+ albums** +**Core Components**: +- Matching engine with Unicode/fuzzy logic +- Discovery system with custom algorithms +- Download pipeline with quality profiles +- Metadata enhancement (lyrics, art, tags) +- Template-based file organization --- ## File Organization -SoulSync uses customizable path templates with validation. - -### Default Structure - +**Default Structure** ``` -Transfer/ - Artist/ - Artist - Album/ - 01 - Track.flac - 01 - Track.lrc +Transfer/Artist/Artist - Album/01 - Track.flac ``` -### Template System +**Custom Templates** +- Albums: `$albumartist/$albumartist - $album/$track - $title` +- Singles: `$artist/$artist - $title/$title` +- Playlists: `$playlist/$artist - $title` +- Variables: `$artist`, `$albumartist`, `$album`, `$title`, `$track`, `$playlist` -**Available Variables** -- `$artist` - Track artist -- `$albumartist` - Album artist -- `$album` - Album name -- `$title` - Track title -- `$track` - Track number (zero-padded: 01, 02...) -- `$playlist` - Playlist name - -**Default Templates** -- **Albums**: `$albumartist/$albumartist - $album/$track - $title` -- **Singles**: `$artist/$artist - $title/$title` -- **Playlists**: `$playlist/$artist - $title` - -**Features** -- Client-side validation prevents invalid templates -- Automatic fallback on errors -- Reset to defaults button -- Changes apply immediately +**Features**: Client-side validation, automatic fallback, instant apply --- ## Troubleshooting -### Enable Debug Logging +**Enable Debug Logging**: Settings → Log Level → DEBUG → Check `logs/app.log` -Settings → Log Level → DEBUG (takes effect immediately) -Check `logs/app.log` for detailed information - -### Common Issues - -**Files not organizing properly** -- Verify transfer path points to your music library -- Check template syntax in Settings → File Organization -- Use "Reset to Defaults" if templates are broken -- Review logs for path-related errors - -**Docker drive access issues** -- Ensure drives are mounted in docker-compose.yml -- Restart Docker Desktop if mounts fail -- Verify paths use `/host/mnt/X/` prefix in settings - -**OAuth failing from remote devices** -- Spotify requires 127.0.0.1, not server IP -- Manually edit callback URL to use server IP -- See [DOCKER-OAUTH-FIX.md](DOCKER-OAUTH-FIX.md) - -**Wishlist tracks stuck** -- Remove items using delete buttons on wishlist page -- Auto-retry runs every 30 minutes -- Check logs for persistent download failures -- Verify slskd is running and accessible - -**Multi-library Plex/Jellyfin setups** -- Select correct library from dropdown in settings -- Test connection to verify credentials -- Check library permissions - -**Quality scanner finding false positives** -- Adjust quality profile thresholds -- Review format priorities (FLAC vs. MP3) -- Check logs for matching errors - ---- - -## Development - -### Project Structure - -``` -SoulSync/ -├── core/ # Core service clients -│ ├── spotify_client.py -│ ├── soulseek_client.py -│ ├── matching_engine.py -│ ├── watchlist_scanner.py -│ └── personalized_playlists.py -├── database/ # Database layer -│ └── music_database.py -├── services/ # Business logic -│ └── sync_service.py -├── webui/ # Web interface -│ ├── static/ -│ └── index.html -├── ui/ # Desktop GUI (PyQt6) -├── config/ # Configuration management -├── utils/ # Utilities and logging -└── web_server.py # Flask application (22k lines) -``` - -### Contributing - -Contributions welcome! Please: -1. Check existing issues before creating new ones -2. Follow existing code style -3. Add tests for new features -4. Update documentation +**Common Issues** +- **Files not organizing**: Verify transfer path, check template syntax, use "Reset to Defaults" +- **Docker paths**: Ensure drives mounted in docker-compose.yml, use `/host/mnt/X/` prefix +- **OAuth from remote**: Manually edit callback URL to server IP (Spotify requires 127.0.0.1) +- **Wishlist stuck**: Auto-retry runs every 30 mins, check logs for failures +- **Multi-library**: Select correct library in settings dropdown --- ## Roadmap -### Planned Features - -- WebSocket support (replace polling architecture) -- Batch wishlist operations (select 20 tracks → remove) +### Planned +- WebSocket support (replace polling) +- Batch wishlist operations - Download history browser UI -- Source reliability tracking (learn which Slskd users are best) -- Notification center (persistent toast history) -- Mobile-responsive UI improvements -- Playlist collaboration between SoulSync instances -- Smart bandwidth management (time-based rules) +- Source reliability tracking +- Notification center +- Mobile-responsive improvements ### Under Consideration - - MusicBrainz ID integration - Additional streaming sources (Deezer, Apple Music) -- Advanced playlist scheduling -- Export to external playlists (Spotify, Tidal) -- Machine learning for improved matching +- Playlist collaboration between instances +- Machine learning for matching --- ## License -[Include your license here] +MIT License - See [LICENSE](LICENSE) file for details --- ## Acknowledgments -- **slskd** - Soulseek daemon -- **music-map.com** - Similar artist data -- **LRClib.net** - Synchronized lyrics -- **Spotify, Tidal, Plex, Jellyfin, Navidrome** - API providers -- **Community contributors** - Feature requests and bug reports +**Services**: slskd, music-map.com, LRClib.net, Spotify, Tidal, Plex, Jellyfin, Navidrome + +**Community**: Contributors, testers, and users providing feedback ---