Expands variable validation to also include support for partial resolution.
In short, this will partially execute script code until it hits unresolved runtime variables, and store that as the new script representation. This functionality will allow for the upcoming `inspect` command, which will show users their script code in action without having to dry-run.
Adds the mechanism to partially resolve an entire script (aka every underlying variable in ytdl-sub). This will be used in the upcoming `inspect` sub-command where it will dump a subscription's raw contents at multiple levels of resolution.
Potentially useful to optimize script execution as well.
Adds the prebuilt preset ``Filter Duration``, which can include/exclude media based on its duration.
Supports the following override variables:
* ``filter_duration_min_s``
* ``filter_duration_max_s``
Usage:
```
Plex TV Show by Date | Filter Duration:
= Documentaries:
"~NOVA PBS":
url: "https://www.youtube.com/@novapbs"
filter_duration_min_s: 120 # Only download videos at least 2m long
= Sports:
"~Maple Leafs Highlights":
url: "https://www.youtube.com/@NHL"
filter_duration_max_s: 180 # Only get highlight videos less than 3m long
```
Adds the ability to make thumbnails square, both file and embedded. Usage:
```
my_preset:
square_thumbnail: True
```
Will soon enable this by default for all music-based presets with a toggle to disable.
Huge thanks to @Kentaro1043 for crafting the ffmpeg command for this!
Closes https://github.com/jmbannon/ytdl-sub/issues/383
Gives Emby first class support by creating prebuilt presets for each variant, including
- `Emby TV Show by Date`
- `Emby TV Show Collection`
- `Emby Music Videos`
Emby is identical to Jellyfin except in TV Show Collection, where it requires a `season.nfo`. A new plugin `StaticNfoTags` has been created to create those.
Huge thanks to @Kamaroth92 for the initial code for this feature!
- Update Ubuntu-based images to use Python 3.12
- Move hidden files from GUI image's main dir
- Update github workflows to use python 3.12
- Update fixtures
- Minor doc fixes
Adds ``Filter Keywords``, a preset that can include or exclude media with any of the listed keywords. Both keywords and title/description are lower-cased before filtering.
Supports the following override variables:
* ``title_include_keywords``
* ``title_exclude_keywords``
* ``description_include_keywords``
* ``description_exclude_keywords``
For best usage, use the `~` tilda subscription mode to set a subscription's list override variables.
Tilda mode allows override variables to be set directly underneath it.
```
Plex TV Show by Date | Filter Keywords:
= Documentaries:
"~NOVA PBS":
url: "https://www.youtube.com/@novapbs"
title_exclude_keywords:
- "preview"
- "trailer"
"~To Catch a Smuggler":
url: "https://www.youtube.com/@NatGeo"
title_include_keywords:
- "To Catch a Smuggler"
```
Enhances the music video presets by supporting the following syntax:
`subscriptions.yaml`
```
__preset__:
overrides:
music_video_directory: "/music_videos"
# Choose between Jellyfin/Kodi/Plex Music Videos preset:
# - Plex Music Videos:
# - Jellyfin Music Videos:
# - Kodi Music Videos:
#
"Plex Music Videos":
= Pop: # Sets genre tag to "Pop"
"Rick Astley": "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
"Michael Jackson": "https://www.youtube.com/playlist?list=OLAK5uy_mnY03zP6abNWH929q2XhGzWD_2uKJ_n8E"
= Rock:
# Prefixing with '+' puts the subscription into 'map-mode'.
# Music video presets in map-mode support grouping videos into different
# categories, which get set on the album field.
#
# URLs can either be strings, or maps that can overload title, year, date
"+ Guns N' Roses":
Music Videos:
- "https://www.youtube.com/playlist?list=PLOTK54q5K4INNXaHKtmXYr6J7CajWjqeJ"
Concerts:
- title: "Live at The Ritz - New York City"
year: "1988"
url: "https://www.youtube.com/watch?v=OldpIhHPsbs"
- title: "Live at The Hollywood Bowl"
date: "2023-01-11"
url: "https://www.youtube.com/watch?v=Z7hutGlvq9I"
```
The 'map-mode' (denoted by `+`) lets you specify a map of music video categories. Under each category, a URL can be specified as-is or with additional metadata (title, year/date, url).
Music videos and concerts, especially older ones, are typically uploaded by random users with inconsistencies in their titles. This syntax aims to make it easy to specify individual URLs with the ability to overwrite their title and year, and group them by category.
Playlists have always been a pain-point with ytdl-sub. If an author adds new videos to the end of a playlist, as opposed to the front, it breaks ytdl-sub's intuition of incremental scraping by breaking on the first (oldest) video. This update now makes it possible to handle this in the prebuilt TV Show presets:
- Add each URL variable (`url`, `url2`, ...) into the `download` portion of the subscription twice
- First definition is what we all know and use, simply scrapes first-to-last, then downloads last-to-first
- Second definition does the following:
- Check to see if a URL is a YouTube playlist URL, if so...
- Set the field to download, but with modifications to scrape last-to-first, then download first-to-last
- Otherwise...
- Set the field to an empty string, which means ytdl-sub will skip it
For less popular sites, ytdl-sub's custom download archive was not working. This was because we weren't using the right extraction key (though it still worked for YouTube, Bandcamp, Soundcloud, and others). This bugfix should now exactly match yt-dlp's and never be a problem again.
Huge thanks to @Svagtlys for root-causing this 🤝
Adds a new mechanism to limit subscriptions' number of videos to an explicit number (only supported date ranges prior to this).
Example usage:
```
output_options:
keep_max_files: 10
```
will only keep 10 videos at max.
In addition:
- `keep_max_files` is bundled into the `Only Recent` preset and can be used via setting the `only_recent_max_files` override variable (see examples or README for usage).
- Changed the `date_range` variable name to `only_recent_date_range` for more clarity. Usage of `date_range` will continue to work.
Provides options to make ytdl-sub look more 'human-like' to protect from throttling. For range-based values, a random number will be chosen within the range to avoid sleeps looking scripted.
Usage:
```
throttle_protection:
sleep_per_download_s:
min: 2.2
max: 10.8
sleep_per_subscription_s:
min: 9.0
max: 14.1
max_downloads_per_subscription:
min: 10
max: 36
subscription_download_probability: 1.0
```
Creates standardized prebuilt presets for music videos. Preset names are:
```
"Kodi Music Videos"
"Jellyfin Music Videos"
"Plex Music Videos"
```
Usage can be found in the `examples/` directory
For music presets, adds `track_genre_default` as an override variable (defaults to Unset)
For TV show presets, adds `tv_show_genre_default` and `tv_show_content_rating_default`, which default to 'ytdl-sub' and 'TV-14'
Music multi-tags (artists, albumartists, etc) were not writing correctly. Fix it by explicitly checking to see if the tag is multi or not in the codebase.
Adds the following prebuilt presets:
```
"Single"
"SoundCloud Discography"
"YouTube Releases"
"YouTube Full Albums"
"Bandcamp"
```
which require no config.yaml to use. Usage examples can be found in the `/examples` directory
A follow-up to the prior commit which fixes some bugs in the 'beautiful subscriptions' implementation. A proper migration guide will be written in the wiki once it's ready
With the recent push to 'beautify subscriptions' (see https://github.com/jmbannon/ytdl-sub/releases/tag/2023.10.02), we need the ability to change subscription names from their legacy form:
```
rick_a:
preset:
- "tv_show"
overrides:
tv_show_name: "Rick A"
url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
into:
```
tv_show:
"Rick A": "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
This however has implications from ytdl-sub's legacy download archive naming. By default, we write archives to `.ytdl-sub-{subscription_name}-download-archive.json`. If we change the subscription name, the archive will not be found, causing a complete redownload. This new feature gives us the ability to migrate download archives to a new naming schema.
# Migrating to Beautified Subscriptions
## Step 0
BACK UP ALL CONFIG + SUBSCRIPTION FILES!!!!! If something goes wrong, restore your backup and try again and/or ask for help.
## Step 1
Since we know we'll be changing our `subscripion_name` to the value of `tv_show_name`, we can use that in our newly migrated download archive name by setting this within the `tv_show` preset (or whatever your 'base' preset is).
```
presets:
tv_show:
output_options:
migrated_download_archive_name: ".{tv_show_name_sanitized}-download-archive.json"
```
## Step 2
Perform a download as usual, via `ytdl-sub sub ...`. This will load the old archive, and save it into the new archive. You should see `MIGRATION DETECTED` within the logs. Ensure it completes successfully.
Perform another download invocation and ensure you see the `MIGRATION SUCCESSFUL` within the logs.
## Step 3
Now we can set:
```
presets:
tv_show:
output_options:
# rename migrated_download_archive_name to just download_archive_name
download_archive_name: ".{tv_show_name_sanitized}-download-archive.json"
overrides:
tv_show_name: "{subscription_name}"
url: "{subscription_value}"
```
Our download archives now default to our new format, and we set `tv_show_name` + `url` to use subscription values by default.
## Step 4
We can now beautify our subscription.yaml file to:
```
tv_show:
"Rick A": "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
