No description
Find a file
Jesse Bannon 309f259464
[FEATURE] Ability to migrate download archive files (#751)
With the recent push to 'beautify subscriptions' (see https://github.com/jmbannon/ytdl-sub/releases/tag/2023.10.02), we need the ability to change subscription names from their legacy form:
```
rick_a:
  preset:
    - "tv_show"
  overrides:
    tv_show_name: "Rick A"
    url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
into:
```
tv_show:
  "Rick A": "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```

This however has implications from ytdl-sub's legacy download archive naming. By default, we write archives to `.ytdl-sub-{subscription_name}-download-archive.json`. If we change the subscription name, the archive will not be found, causing a complete redownload. This new feature gives us the ability to migrate download archives to a new naming schema.

# Migrating to Beautified Subscriptions
## Step 0
BACK UP ALL CONFIG + SUBSCRIPTION FILES!!!!! If something goes wrong, restore your backup and try again and/or ask for help.

## Step 1
Since we know we'll be changing our `subscripion_name` to the value of `tv_show_name`, we can use that in our newly migrated download archive name by setting this within the `tv_show` preset (or whatever your 'base' preset is).
```
presets:
  tv_show:
    output_options:
      migrated_download_archive_name: ".{tv_show_name_sanitized}-download-archive.json" 
```

## Step 2
Perform a download as usual, via `ytdl-sub sub ...`. This will load the old archive, and save it into the new archive. You should see `MIGRATION DETECTED` within the logs. Ensure it completes successfully.

Perform another download invocation and ensure you see the `MIGRATION SUCCESSFUL` within the logs.

## Step 3
Now we can set:
```
presets:
  tv_show:
    output_options:
      # rename migrated_download_archive_name to just download_archive_name
      download_archive_name: ".{tv_show_name_sanitized}-download-archive.json" 

  overrides:
    tv_show_name: "{subscription_name}"
    url: "{subscription_value}"
```
Our download archives now default to our new format, and we set `tv_show_name` + `url` to use subscription values by default.

## Step 4
We can now beautify our subscription.yaml file to:
```
tv_show:
  "Rick A": "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
2023-10-03 16:08:14 -07:00
.github [DOCKER] Tag ubuntu docker image version correctly 2023-09-21 23:31:42 -07:00
docker [REFACTOR] Support simple download (#671) 2023-07-26 09:00:00 -07:00
docs [FEATURE] Subscription nesting (#747) 2023-10-02 16:07:48 -07:00
examples [BACKEND] Perform match-filters at metadata stage, use match-filters for date_range plugin (#725) 2023-09-19 11:19:15 -07:00
src/ytdl_sub [FEATURE] Ability to migrate download archive files (#751) 2023-10-03 16:08:14 -07:00
tests [FEATURE] Ability to migrate download archive files (#751) 2023-10-03 16:08:14 -07:00
tools [DOCKER] Use alpine-3.16, update ffmpeg (attempt 2) (#332) 2022-11-15 13:34:47 -08:00
.gitignore [BACKEND] Windows support (#476) 2023-02-28 17:07:03 -08:00
.readthedocs.yaml ordering 2022-04-26 07:53:50 +00:00
codecov.yml [BACKEND] Informational codecov only (#170) 2022-08-12 11:34:22 -07:00
LICENSE Initial commit 2021-09-08 21:44:33 -07:00
Makefile [DOCKER] Add ubuntu docker image for device passthrough (#545) 2023-03-15 14:10:32 -07:00
pyproject.toml [FEATURE] Ability to change download archive name (#643) 2023-06-30 11:51:52 -07:00
README.md [DOCS] Unraid in readme (#581) 2023-04-05 12:39:35 -07:00
setup.cfg [BACKEND] Update yt-dlp to 2023.9.24 (#741) 2023-09-24 05:17:52 -07:00
ytdl-sub.spec [RELEASE] Add linux executables to release (#464) 2023-02-22 20:15:34 -08:00

ytdl-sub

Automate downloading and metadata generation with YoutubeDL.

codecov Code Qaulity Checks License

ytdl-sub is a command-line tool that downloads media via yt-dlp and prepares it for your favorite media player, including Kodi, Jellyfin, Plex, Emby, and modern music players. No additional plugins or external scrapers are needed.

We recognize that everyone stores their media differently. Our approach for file and metadata formatting is to provide maximum flexibility while maintaining simplicity.

YouTube Channels as TV Shows

Plex

unknown

Jellyfin

jellyfin

Music Videos and Concerts

Kodi

kodi

Jellyfin

jelly_mv

SoundCloud Albums and Singles

MusicBee (any file or tag-based music players)

sc_mb

Bandcamp Discography

Navidrome (any file or tag-based music servers)

bc_nav

How it Works

ytdl-sub uses YAML configs to define a layout for how you want media to look after it is downloaded. See our walk-through guide on how to get started. Ready-to-use example configurations can be found here alongside our readthedocs for detailed information on config fields.

Config

The config.yaml defines how our downloads will look. For this example, let us download YouTube channels and generate metadata to look like TV shows using ytdl-sub's prebuilt presets. No additional plugins or programs are needed for Kodi, Jellyfin, Plex, or Emby to recognize your downloads. This can also be used to download any yt-dlp supported URL, including YouTube playlists, Bitchute channels, etc.

# Set the working directory which will be used to stage downloads
# before placing them in your desired output directory.
configuration:
  working_directory: '.ytdl-sub-downloads'

# Presets are where you create 'sub-configs' that can can be
# merged together to dictate what is downloaded, how to format it,
# and what metadata to generate.
presets:

  # Let us create a preset called `only_recent_videos` that will
  # only download recent videos in the last 2 months.
  only_recent_videos:

    # Use the `date_range` plugin to specify ytdl-sub to only
    # download videos after today MINUS {download_range}, which
    # is an override variable that we can alter per channel.
    date_range:
      after: "today-{download_range}"

    # Any yt-dlp argument can be passed via ytdl-sub. Let us set
    # yt-dlp's `break_on_reject` to True to stop downloading after
    # any video is rejected. Videos will be rejected if they are
    # uploaded after our {download_range}.
    ytdl_options:
      break_on_reject: True

    # Deletes any videos uploaded after {download_range}.
    output_options:
      keep_files_after: "today-{download_range}"

    # Set the override variable {download_range} to 2months.
    # This will serve as our default value. We can override
    # this per channel or in a child preset.
    overrides:
      download_range: "2months"

  ####################################################################

  # Now let us create a preset that downloads videos and formats
  # as TV shows.
  tv_show:

    # Presets can inherit all attributes from other presets. Our
    # `tv_show` preset will inherit these presets built into ytdl-sub.
    preset:
      # Let us specify all the TV show by date presets to support all
      # players. You only need to specify one, but this ensures
      # compatibility with all players.
      - "kodi_tv_show_by_date"
      - "jellyfin_tv_show_by_date"
      - "plex_tv_show_by_date"
      # Now we choose a preset that defines how our seasons and
      # episode numbers look.
      - "season_by_year__episode_by_month_day"

    # Set override variables that will be applicable to all downloads
    # in main presets.
    overrides:
      tv_show_directory: "/tv_shows"  # Replace with desired directory

Subscriptions

The subscriptions.yaml file is where we define content to download using presets in the config.yaml. Each subscription can overwrite any field used in a preset.

# The name of our subscription. Let us create one to download
# ALL of Rick A's videos
rick_a:
  # Inherit our `tv_show` preset we made above
  preset:
    - "tv_show"
  
  # Set override variables to set the channel URL and the
  # name we want to give the TV show.
  overrides:
    tv_show_name: "Rick A"
    url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"

# Let us make another subscription that will only download Rick A's
# video's in the last 2 weeks.
rick_a_recent:
  # Inherit our `tv_show` AND `only_recent_videos` preset
  # Bottom-most presets take precedence.
  preset:
    - "tv_show"
    - "only_recent_videos"
  
  # Set override variables for this subscription. Modify the
  # `download_range` to only download and keep 2 weeks' worth
  # of videos.
  overrides:
    tv_show_name: "Rick A"
    url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
    download_range: "2weeks"

The download can now be performed using:

ytdl-sub sub subscriptions.yaml

To preview what your output files before doing any downloads, you can dry run using:

ytdl-sub --dry-run sub subscriptions.yaml

One-time Download

There are things we will only want to download once and never again. Anything you can define in a subscription can be defined using CLI arguments. This example is equivalent to the subscription example above:

ytdl-sub dl \
    --preset "tv_show" \
    --overrides.tv_show_name "Rick A" \
    --overrides.url "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"

Download Aliases

In the config.yaml, we can define aliases to make dl commands shorter.

configuration:
  dl_aliases:
    tv: "--preset tv_show"
    name: "--overrides.tv_show_name"
    url: "--overrides.url"

The above command can now be shortened to

ytdl-sub dl \
    --tv \
    --name "Rick A" \
    --url "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"

Output

After ytdl-sub runs, the end result will download and format the channel files into something ready to be consumed by your favorite media player or server.

/path/to/tv_shows/Rick Aß
  /Season 2021
    s2021.e0317 - Pattys Day Video-thumb.jpg
    s2021.e0317 - Pattys Day Video.mp4
    s2021.e0317 - Pattys Day Video.nfo
  /Season 2022
    s2022.e1225 - Merry Christmas-thumb.jpg
    s2022.e1225 - Merry Christmas.mp4
    s2022.e1225 - Merry Christmas.nfo
  poster.jpg
  fanart.jpg
  tvshow.nfo

Beyond TV Shows

The above example made heavy-use of ytdl-sub prebuilt presets and hides many features that are offered. ytdl-sub strives to support any use case that first requires a download via yt-dlp. Use ytdl-sub to download, format, and convert media for your media player to recognize downloads as:

  • Movies
  • TV shows
    • From a single channel or playlist
    • From multiple channels or playlists
    • From individual videos
  • Extracted audio as podcasts
  • Music videos
  • Music, including:
    • Individual songs
    • Albums
    • Discographies

Installation

ytdl-sub can be installed on the following platforms.

Contributing

There are many ways to contribute, even without coding. Please take a look in our GitHub Issues to submit a feature request, or pick up a bug.

Support

We are pretty active in our Discord channel if you have any questions. Also see our FAQ for commonly asked questions.