danvics/ytdl-sub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

update readme

Browse source
This commit is contained in:
jbannon 2022-04-01 18:56:29 +00:00
parent 329562bb0b
commit 02203e3a7b
2 changed files with 273 additions and 5 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

274
README.md
View file

@ -1,9 +1,273 @@
# YoutubeDL-Subscribe
Automate downloading and tagging files using YoutubeDL.
Automate downloading and adding metadata with YoutubeDL.
README is WIP but the package is usable. See the provided `subscriptions.yaml`, install the `requirements.txt`, and run `PYTHONPATH=. python ytdl_subscribe main -h` to get started.
The motivation behind this package is to download media via [youtube-dlp](https://github.com/yt-dlp/yt-dlp)
and prepare it for consumption in your favorite media player in the most hassle-free way possible.
Everyone stores their media differently, so it is critical to provide comprehensive customization to this process
while maintaining simplicity. In addition, we treat media sources differently from each other. We may want to...
## TODO
- Add tests with coverage
- Test output paths work before downloading
- Automatically download any and all audio/video file from a channel, user, playlist, etc.
- Download a soundcloud artist's discography, keep it up to date
- Download any new music videos from a YouTube channel playlist
- Manually download a single audio/video file
- Download one particular music video
- Only download recent audio/video, and remove stale files
- Download and only keep recent podcasts, news videos
We want to cover each of these use cases.
## Configuration
Below is the proposed way to define a `preset` for how you want to download and add metadata. We have two examples,
one for downloading Soundcloud discographies, and the other for downloading YouTube playlists, and treating them as
music videos. Both are intended to be consumed by [Kodi](https://github.com/xbmc/xbmc),
[Jellyfin](https://github.com/jellyfin/jellyfin), and any mp3 music player that supports id3v2.4 tags. For videos, this
should (hopefully) be customizable enough to support other use-cases like downloading a playlist as a TV show or
multiple movies.
#### config.yaml
```yaml
presets:
# [Required]
# Custom name that you define for your preset. This preset is intended to download
# a soundcloud artist's discography.
soundcloud_discography:
# [Required]
# This section defines the source you intend on using, with specific
# configurations for each source. Supports [soundcloud, youtube]
soundcloud:
# [Required]
# How to download media from the source, soundcloud supports [albums_and_singles]
download_strategy: "albums_and_singles"
# [Optional]
# Each source has specific options, this is one for soundcloud that skips
# premiere tracks
skip_premiere_tracks: True
# [Required, defined dynamically]
# The username of the soundcloud artist you want to download from
# username: alis_on
# [Optional]
# Defines any arguments you want to directly pass to yt_dlp. Use with caution!
ytdl_options:
# For my case, I want the best audio converted to mp3
format: 'bestaudio[ext=mp3]'
# [Required]
# This section defines where and how you want to save files
output_options:
# [Required]
# Directory for where you want all your files to land for this preset
output_directory: "{music_path}/{sanitized_artist}"
# [Required]
# How you want to name each file that gets downloaded in this preset
file_name: "[{album_year}] {sanitized_album}/{track_number_padded} - {sanitized_title}.{ext}"
# [Optional]
# If a thumbnail is available, convert it to your desired file type (webp, ugh)
convert_thumbnail: "jpg"
# [Optional]
# If you want to store a thumbnail, define how it gets named
thumbnail_name: "[{album_year}] {sanitized_album}/folder.jpg"
# [Optional]
# Define how you want to add metadata to your downloaded media
metadata_options:
# [Optional]
# If your final file is mp3, add id3 tags (TODO: support for more file types)
id3_tags:
# [Optional]
# id3 version, defaults to 2.4
id3_version: 2.4
# [Optional]
# For 2.4 multi-tags, define which character to use for the null separator
null_separator_char: ";"
# [Required]
# Define the tag names and values to add to your audio
tags:
artist: "{artist}"
albumartist: "{artist}"
title: "{title}"
album: "{album}"
tracknumber: "{track_number}"
year: "{album_year}"
genre: "{genre}"
# [Optional]
# This will lazily add/override any media variable that can be used in
# output or metadata options
overrides:
# Notice how we use music_path in the output_options.output_directory field above
music_path: "/mnt/nas/music"
######################################################################################
# Another preset, intended to download all music videos in a YouTube playlist.
youtube_music_video_playlist:
# [Required]
# Our source is YouTube, so define the youtube section.
youtube:
# [Required]
# Download strategies are different for each source. For YouTube, we want to
# download the whole playlist (also the only supported download_strategy for now).
download_strategy: "playlist"
# [Required, defined dynamically]
# The YouTube playlist id you want to download from
# playlist_id: "PLVTLbc6i-h_iuhdwUfuPDLFLXG2QQnz-x"
# [Optional]
ytdl_options:
# In my case, I want the best format possible, and ignore any errors like
# geo/age restricted videos.
format: 'best'
ignoreerrors: True
# [Required]
output_options:
# Notice in my overrides, I defined the both music_video_path and kodi_music_video_name.
output_directory: "{music_video_path}"
file_name: "{kodi_music_video_name}.{ext}"
convert_thumbnail: "jpg"
thumbnail_name: "{kodi_music_video_name}.jpg"
# [Optional]
metadata_options:
# [Optional]
# Kodi works with NFO files for videos. I want to format this to be read in as a music video
nfo:
# [Required]
# Name of the nfo file
nfo_name: "{kodi_music_video_name}.nfo"
# [Required]
# Inside the nfo XML, define the root section.
nfo_root: "musicvideo"
# [Required]
# Within the nfo root, define any tags you want set
tags:
artist: "{artist}"
title: "{title}"
album: "Music Videos"
year: "{upload_year}"
# [Optional]
overrides:
# I personally like my paths defined explicitly as variables.
music_video_path: "/mnt/nas/music"
# Kodi wants music videos all in one folder, named like 'artist - title'
# This format is used for the video file, thumbnail, and nfo file. So
# create a shared variable that each of them will use.
kodi_music_video_name: "{sanitized_artist} - {sanitized_title}"
```
## Usage
That was a lot! If you have made it this far, thanks for your interest. Now we can see how to use our presets to
actually download. There are three methods this project aims to build.
### Subscriptions
Defining a subscription is for media that you will continually want to download, whether that be performed manually
or in a cron job. We implement this by defining yet another yaml.
#### subscription.yaml
```yaml
# [Required]
# Custom name that you define for your subscription. Any fields defined here will override anything defined in the
# preset.
ALISON:
# [Required]
# Name of the preset to use
preset: "soundcloud_discography"
# We did not define soundcloud.username in the preset on purpose, so we can define it here
soundcloud:
username: alis_on
# For this artist, I want to explicitly define the artist name and genres that will get used in the metadata
# and output paths. An implicit feature is all overrides also create a sanitized variable as well. So
# `sanitized_artist` can be used, and will run the sanitize function on my artist override.
overrides:
artist: "A.L.I.S.O.N."
genre: "Synthwave;Electronic;Instrumental"
######################################################################################
# Another subscription that can use a different preset, overrides, etc.
Rammstein:
# [Required]
preset: "youtube_music_video_playlist"
# [Required]
# We did not define youtube.playlist_id in the preset on purpose, to define it here.
youtube:
playlist_id: "PLVTLbc6i-h_iuhdwUfuPDLFLXG2QQnz-x"
# [Optional]
overrides:
artist: "Rammstein"
```
You can have multiple subscription files for better organizing. Now that that is defined, we can invoke the download
using the command below. This makes it easy to use `ytdl-sub` in a cron job.
```shell
ytdl-sub sub subscription.yaml
```
### One-time Downloads [not available yet]
There are things I want to download, but not subscribe to. In this case, I can invoke `ytdl-sub dl` and add any
required/optional/override fields. This example is equivalent to using the subscription yaml above, but without
the yaml.
```shell
ytdl-sub dl \
--preset "soundcloud_discography" \
--soundcloud.username "alis_on" \
--overrides.artist "A.L.I.S.O.N." \
--overrides.genre "Synthwave;Electronic;Instrumental"
```
### REST API [not available yet]
The dream will be to create mobile apps or browser extensions to invoke downloads from the couch. My short-term
plan is to self-host a swaggerhub io instance and invoke the rest api from my browser. It will be a while before
this gets implemented, but when it does, it will look like this.
```commandline
# spin-up the server
ytdl-sub server
```
POST request to invoke the same one-time download example via REST.
```commandline
curl -X POST -H "Content-type: application/json" \
-d '{ \
"soundcloud.username": "alis_on", \
"overrides": { \
"artist": "A.L.I.S.O.N.", \
"genre": "Synthwave;Electronic;Instrumental" \
} \
}' \
'localhost:8080/soundcloud_wih_id3_tags'
```
## Contributing
I would like to get this to a point where I create a stable release before I accept PRs from the outside world. In the
meantime, any feedback in the form of a Github issue would be greatly appreciated and taken into consideration.

4
TODO.md Normal file
View file

@ -0,0 +1,4 @@
## TODO
A sloppy todo list file where I write any issues/things to address that pop in my head
- Add more tests with coverage
- Validate output paths work before downloading