danvics/ytdl-sub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

[DOCS] New README with only subscriptions (#791)

Browse source
This commit is contained in:
Jesse Bannon 2023-11-02 23:28:55 -07:00 committed by GitHub
parent f8868add5f
commit e9b2214a0a
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
7 changed files with 286 additions and 252 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

343
README.md
View file

@ -33,213 +33,163 @@ maximum flexibility while maintaining simplicity.
#### Jellyfin
![jelly_mv](https://user-images.githubusercontent.com/10107080/182677256-43aeb029-0c3f-4648-9fd2-352b9666b262.PNG)
### SoundCloud Albums and Singles
#### MusicBee (any file or tag-based music players)
### SoundCloud Discography
#### Writes proper music-tags via beets API
![sc_mb](https://user-images.githubusercontent.com/10107080/182685415-06adf477-3dd3-475d-bbcd-53b0152b9f0a.PNG)
### Bandcamp Discography
#### Navidrome (any file or tag-based music servers)
![bc_nav](https://user-images.githubusercontent.com/10107080/212503861-1d8748e6-6f6d-4043-b543-84226cd1f662.png)
## How it Works
`ytdl-sub` uses YAML configs to define a layout for how you want media to look
after it is downloaded. See our
`ytdl-sub` uses YAML files to define subscriptions. Each subscription imports _presets_ that
define how to handle and output media files. `ytdl-sub` comes packaged with many _prebuilt presets_
that do the work of config-building, so you can start downloading immediately.
```yaml
# subscriptions.yaml:
# __preset__ is a place to define global overrides for all subscriptions
__preset__:
overrides:
# Root folder of all ytdl-sub TV Shows
tv_show_directory: "/tv_shows"
# Root folder of all ytdl-sub Music
music_directory: "/music"
# Root folder of all ytdl-sub Music Videos
music_video_directory: "/music_videos"
# For 'Only Recent' preset, only keep vids uploaded in this range
date_range: "2months"
# Pass any arg directly to yt-dlp's Python API
ytdl_options:
cookiefile: "/config/cookie.txt"
###################################################################
# Subscriptions nested under this will use the
# `Plex TV Show by Date` preset.
#
# Can choose between:
# - Plex TV Show by Date:
# - Jellyfin TV Show by Date:
# - Kodi TV Show by Date:
Plex TV Show by Date:
# Sets genre tag to "Documentaries"
= Documentaries:
"NOVA PBS": "https://www.youtube.com/@novapbs"
"National Geographic": "https://www.youtube.com/@NatGeo"
"Cosmos - What If": "https://www.youtube.com/playlist?list=PLZdXRHYAVxTJno6oFF9nLGuwXNGYHmE8U"
# Sets genre tag to "Kids", "TV-Y" for content rating
= Kids | = TV-Y:
"Jake Trains": "https://www.youtube.com/@JakeTrains"
"Kids Toys Play": "https://www.youtube.com/@KidsToysPlayChannel"
= Music:
# TV show subscriptions can support multiple urls and store in the same TV Show
"Rick Beato":
- "https://www.youtube.com/@RickBeato"
- "https://www.youtube.com/@rickbeato240"
# Set genre tag to "News", use `Only Recent` preset to only store videos uploaded recently
= News | Only Recent:
"BBC News": "https://www.youtube.com/@BBCNews"
###################################################################
# Subscriptions nested under these will use the various prebuilt
# music presets
YouTube Releases:
= Jazz: # Sets genre tag to "Jazz"
"Thelonious Monk": "https://www.youtube.com/@theloniousmonk3870/releases"
YouTube Full Albums:
= Lofi:
"Game Chops": "https://www.youtube.com/playlist?list=PLBsm_SagFMmdWnCnrNtLjA9kzfrRkto4i"
SoundCloud Discography:
= Chill Hop:
"UKNOWY": "https://soundcloud.com/uknowymunich"
= Synthwave:
"Lazerdiscs Records": "https://soundcloud.com/lazerdiscsrecords"
"Earmake": "https://soundcloud.com/earmake"
Bandcamp:
= Lofi:
"Emily Hopkins": "https://emilyharpist.bandcamp.com/"
###################################################################
# Can choose between:
# - Plex Music Videos:
# - Jellyfin Music Videos:
# - Kodi Music Videos:
"Plex Music Videos":
= Pop: # Sets genre tag to "Pop"
"Rick Astley": "https://www.youtube.com/playlist?list=PLlaN88a7y2_plecYoJxvRFTLHVbIVAOoc"
"Michael Jackson": "https://www.youtube.com/playlist?list=OLAK5uy_mnY03zP6abNWH929q2XhGzWD_2uKJ_n8E"
```
All of this can be downloaded and ready to import to your favorite player
using the command
```commandline
ytdl-sub sub subscriptions.yaml
```
See our
[example subscriptions](https://github.com/jmbannon/ytdl-sub/tree/master/examples)
for more detailed examples and use-cases.
### Output
After `ytdl-sub` runs, the end result will download and format the files into something ready
to be consumed by your favorite media player/server.
```
tv_shows/
Jake Trains/
Season 2021/
s2021.e031701 - Pattys Day Video-thumb.jpg
s2021.e031701 - Pattys Day Video.mp4
s2021.e031701 - Pattys Day Video.nfo
s2021.e031702 - Second Pattys Day Video-thumb.jpg
s2021.e031702 - Second Pattys Day Video.mp4
s2021.e031702 - Second Pattys Day Video.nfo
Season 2022/
s2022.e122501 - Merry Christmas-thumb.jpg
s2022.e122501 - Merry Christmas.mp4
s2022.e122501 - Merry Christmas.nfo
poster.jpg
fanart.jpg
tvshow.nfo
music/
Artist/
[2022] Some Single/
01 - Some Single.mp3
folder.jpg
[2023] Latest Album/
01 - Track Title.mp3
02 - Another Track.mp3
folder.jpg
music_videos/
Elton John/
Elton John - Rocketman.jpg
Elton John - Rocketman.mp4
```
## Custom Configs
Any part of this process is modifiable by using custom configs. See our
[walk-through guide](https://github.com/jmbannon/ytdl-sub/wiki)
on how to get started. Ready-to-use
on how to build your first config from scratch. Ready-to-use
[example configurations](https://github.com/jmbannon/ytdl-sub/tree/master/examples)
can be found here alongside our
[readthedocs](https://ytdl-sub.readthedocs.io/en/latest/config.html#)
for detailed information on config fields.
### Config
The `config.yaml` defines how our downloads will look. For this example, let us
download YouTube channels and generate metadata to look like TV shows using
ytdl-sub's prebuilt presets. No additional plugins or programs are needed for
Kodi, Jellyfin, Plex, or Emby to recognize your downloads. This can also be
used to download any yt-dlp supported URL, including YouTube playlists, Bitchute channels, etc.
```yaml
# Set the working directory which will be used to stage downloads
# before placing them in your desired output directory.
configuration:
working_directory: '.ytdl-sub-downloads'
# Presets are where you create 'sub-configs' that can can be
# merged together to dictate what is downloaded, how to format it,
# and what metadata to generate.
presets:
# Let us create a preset called `only_recent_videos` that will
# only download recent videos in the last 2 months.
only_recent_videos:
# Use the `date_range` plugin to specify ytdl-sub to only
# download videos after today MINUS {download_range}, which
# is an override variable that we can alter per channel.
date_range:
after: "today-{download_range}"
# Any yt-dlp argument can be passed via ytdl-sub. Let us set
# yt-dlp's `break_on_reject` to True to stop downloading after
# any video is rejected. Videos will be rejected if they are
# uploaded after our {download_range}.
ytdl_options:
break_on_reject: True
# Deletes any videos uploaded after {download_range}.
output_options:
keep_files_after: "today-{download_range}"
# Set the override variable {download_range} to 2months.
# This will serve as our default value. We can override
# this per channel or in a child preset.
overrides:
download_range: "2months"
####################################################################
# Now let us create a preset that downloads videos and formats
# as TV shows.
tv_show:
# Presets can inherit all attributes from other presets. Our
# `tv_show` preset will inherit these presets built into ytdl-sub.
preset:
# Let us specify all the TV show by date presets to support all
# players. You only need to specify one, but this ensures
# compatibility with all players.
- "kodi_tv_show_by_date"
- "jellyfin_tv_show_by_date"
- "plex_tv_show_by_date"
# Now we choose a preset that defines how our seasons and
# episode numbers look.
- "season_by_year__episode_by_month_day"
# Set override variables that will be applicable to all downloads
# in main presets.
overrides:
tv_show_directory: "/tv_shows" # Replace with desired directory
```
### Subscriptions
The `subscriptions.yaml` file is where we define content to download using
presets in the `config.yaml`. Each subscription can overwrite any field used
in a preset.
```yaml
# The name of our subscription. Let us create one to download
# ALL of Rick A's videos
rick_a:
# Inherit our `tv_show` preset we made above
preset:
- "tv_show"
# Set override variables to set the channel URL and the
# name we want to give the TV show.
overrides:
tv_show_name: "Rick A"
url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
# Let us make another subscription that will only download Rick A's
# video's in the last 2 weeks.
rick_a_recent:
# Inherit our `tv_show` AND `only_recent_videos` preset
# Bottom-most presets take precedence.
preset:
- "tv_show"
- "only_recent_videos"
# Set override variables for this subscription. Modify the
# `download_range` to only download and keep 2 weeks' worth
# of videos.
overrides:
tv_show_name: "Rick A"
url: "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
download_range: "2weeks"
```
The download can now be performed using:
```shell
ytdl-sub sub subscriptions.yaml
```
To preview what your output files before doing any downloads, you can dry run using:
```shell
ytdl-sub --dry-run sub subscriptions.yaml
```
### One-time Download
There are things we will only want to download once and never again. Anything
you can define in a subscription can be defined using CLI arguments. This
example is equivalent to the subscription example above:
```shell
ytdl-sub dl \
--preset "tv_show" \
--overrides.tv_show_name "Rick A" \
--overrides.url "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
#### Download Aliases
In the `config.yaml`, we can define aliases to make `dl` commands shorter.
```yaml
configuration:
dl_aliases:
tv: "--preset tv_show"
name: "--overrides.tv_show_name"
url: "--overrides.url"
```
The above command can now be shortened to
```shell
ytdl-sub dl \
--tv \
--name "Rick A" \
--url "https://www.youtube.com/channel/UCuAXFkgsw1L7xaCfnd5JJOw"
```
### Output
After `ytdl-sub` runs, the end result will download and format the channel
files into something ready to be consumed by your favorite media player or
server.
```
/path/to/tv_shows/Rick Aß
/Season 2021
s2021.e0317 - Pattys Day Video-thumb.jpg
s2021.e0317 - Pattys Day Video.mp4
s2021.e0317 - Pattys Day Video.nfo
/Season 2022
s2022.e1225 - Merry Christmas-thumb.jpg
s2022.e1225 - Merry Christmas.mp4
s2022.e1225 - Merry Christmas.nfo
poster.jpg
fanart.jpg
tvshow.nfo
```
### Beyond TV Shows
The above example made heavy-use of `ytdl-sub` prebuilt presets and hides many
features that are offered. `ytdl-sub` strives to support _any_ use case that first requires
a download via yt-dlp. Use `ytdl-sub` to download, format, and convert media for your media
player to recognize downloads as:
- Movies
- TV shows
- From a single channel or playlist
- From multiple channels or playlists
- From individual videos
- Extracted audio as podcasts
- Music videos
- Music, including:
- Individual songs
- Albums
- Discographies
for detailed information on all config fields.
## Installation
`ytdl-sub` can be installed on the following platforms.
- [Docker Compose](https://ytdl-sub.readthedocs.io/en/latest/install.html#docker-compose_)
- [with CPU passthrough](https://ytdl-sub.readthedocs.io/en/latest/install.html#cpu-passthrough)
- [with GPU passthrough](https://ytdl-sub.readthedocs.io/en/latest/install.html#nvidia-gpu-passthrough)
- [Docker CLI](https://ytdl-sub.readthedocs.io/en/latest/install.html#docker)
- [Windows](https://ytdl-sub.readthedocs.io/en/latest/install.html#windows)
- [Unraid](https://ytdl-sub.readthedocs.io/en/latest/install.html#unraid)
@ -249,6 +199,15 @@ player to recognize downloads as:
- [Local Install](https://ytdl-sub.readthedocs.io/en/latest/install.html#local-install)
- [Local Docker Build](https://ytdl-sub.readthedocs.io/en/latest/install.html#local-docker-build)
### Docker Installation
Docker installs can be either headless or use the GUI image, which comprises
[LSIO's](https://www.linuxserver.io/)
[code-server](https://hub.docker.com/r/linuxserver/code-server)
Docker image with `ytdl-sub` preinstalled. This is the recommended way to use ``ytdl-sub``.
![image](https://github.com/jmbannon/ytdl-sub/assets/10107080/c2aac8a1-5443-4345-b438-be4b17187c80)
## Contributing
There are many ways to contribute, even without coding. Please take a look in
our [GitHub Issues](https://github.com/jmbannon/ytdl-sub/issues) to submit a feature request, or

3
docs/config.rst
View file

@ -423,8 +423,9 @@ Source Variables
Override Variables
------------------
.. autoclass:: ytdl_sub.config.preset_options.OverridesVariables()
.. autoclass:: ytdl_sub.entries.variables.override_variables.OverrideVariables()
:members:
:member-order: bysource
-------------------------------------------------------------------------------

89
docs/install.rst
View file

@ -3,22 +3,62 @@ Install
``ytdl-sub`` can be installed on the following platforms.
.. contents::
:depth: 2
:depth: 3
All installations require a 64-bit CPU. 32-bit is not supported.
Docker Compose
--------------
The ytdl-sub docker image uses
`Linux Server's <https://www.linuxserver.io/>`_
`base alpine image <https://github.com/linuxserver/docker-baseimage-alpine/>`_
It looks, feels, and operates like other LinuxServer images. This is the
recommended way to use ytdl-sub.
The ytdl-sub Docker images use
`LSIO-based images <https://www.linuxserver.io/>`_
and installs ytdl-sub on top. There are a few flavors to choose from.
The docker image is intended to be used as a console. For automating
``subscriptions.yaml`` downloads to pull new media, see
For automating ``subscriptions.yaml`` downloads to pull new media, see
`this guide <https://github.com/jmbannon/ytdl-sub/wiki/7.-Automate-Downloading-New-Content-Using-Your-Configs/>`_
on how set up a cron job in the docker container.
on how set up a cron job in any of the docker containers.
GUI
^^^^
The GUI image uses LSIO's
`code-server <https://hub.docker.com/r/linuxserver/code-server>`_
for its base image. More info on other code-server environment variables
can be found within its documentation. This is the recommended way to use ``ytdl-sub``.
After starting, code-server will be running at http://localhost:8443/
.. code-block:: yaml
services:
ytdl-sub:
image: ghcr.io/jmbannon/ytdl-sub-gui:latest
container_name: ytdl-sub
environment:
- PUID=1000
- PGID=1000
- TZ=America/Los_Angeles
volumes:
- <path/to/ytdl-sub/config>:/config
- <path/to/tv_shows>:/tv_shows # optional
- <path/to/movies>:/movies # optional
- <path/to/music_videos>:/music_videos # optional
- <path/to/music>:/music # optional
ports:
- 8443:8443
restart: unless-stopped
Headless
^^^^^^^^^^
The headless image uses LSIO's
`baseimage-alpine <https://github.com/linuxserver/docker-baseimage-alpine>`_
for its base image. With this image, ``ytdl-sub`` is meant to be ran from console
via exec'ing into the image using the command:
.. code-block:: bash
docker exec -u abc -it ytdl-sub /bin/bash
.. code-block:: yaml
@ -39,46 +79,51 @@ on how set up a cron job in the docker container.
- <path/to/music>:/music # optional
restart: unless-stopped
CPU Passthrough
^^^^^^^^^^^^^^^^^^^^^^
For CPU passthrough, you must use the ``ytdl-sub`` Ubuntu version with the following additions:
Passthrough
^^^^^^^^^^^
For CPU or GPU passthrough, you must use either the GUI image or the headless Ubuntu image
``ghcr.io/jmbannon/ytdl-sub:ubuntu-latest``.
The docker-compose examples use the GUI image.
CPU
____
.. code-block:: yaml
services:
ytdl-sub:
image: ghcr.io/jmbannon/ytdl-sub:ubuntu-latest
image: ghcr.io/jmbannon/ytdl-sub-gui:latest
container_name: ytdl-sub
environment:
- PUID=1000
- PGID=1000
- TZ=America/Los_Angeles
- DOCKER_MODS=linuxserver/mods:universal-cron
volumes:
- <path/to/ytdl-sub/config>:/config
- <path/to/tv_shows>:/tv_shows # optional
- <path/to/movies>:/movies # optional
- <path/to/music_videos>:/music_videos # optional
- <path/to/music>:/music # optional
ports:
- 8443:8443
devices:
- /dev/dri:/dev/dri # CPU passthrough
restart: unless-stopped
Nvidia GPU Passthrough
^^^^^^^^^^^^^^^^^^^^^^
For GPU passthrough, you must use the ``ytdl-sub`` Ubuntu version with the following additions:
GPU
____
.. code-block:: yaml
services:
ytdl-sub:
image: ghcr.io/jmbannon/ytdl-sub:ubuntu-latest
image: ghcr.io/jmbannon/ytdl-sub-gui:latest
container_name: ytdl-sub
environment:
- PUID=1000
- PGID=1000
- TZ=America/Los_Angeles
- DOCKER_MODS=linuxserver/mods:universal-cron
- NVIDIA_DRIVER_CAPABILITIES=all # Nvidia ENV args
- NVIDIA_VISIBLE_DEVICES=all
volumes:
@ -87,6 +132,8 @@ For GPU passthrough, you must use the ``ytdl-sub`` Ubuntu version with the follo
- <path/to/movies>:/movies # optional
- <path/to/music_videos>:/music_videos # optional
- <path/to/music>:/music # optional
ports:
- 8443:8443
deploy:
resources:
reservations:
@ -103,14 +150,14 @@ Docker
-e PUID=1000 \
-e PGID=1000 \
-e TZ=America/Los_Angeles \
-e DOCKER_MODS=linuxserver/mods:universal-cron \
-p 8443:8443 \
-v <path/to/ytdl-sub/config>:/config \
-v <OPTIONAL/path/to/tv_shows>:/tv_shows \
-v <OPTIONAL/path/to/movies>:/movies \
-v <OPTIONAL/path/to/music_videos>:/music_videos \
-v <OPTIONAL/path/to/music>:/music \
--restart unless-stopped \
ghcr.io/jmbannon/ytdl-sub:latest
ghcr.io/jmbannon/ytdl-sub-gui:latest
Windows
--------------

2
src/ytdl_sub/config/preset_options.py
View file

@ -9,7 +9,7 @@ from yt_dlp.utils import sanitize_filename
from ytdl_sub.config.defaults import DEFAULT_DOWNLOAD_ARCHIVE_NAME
from ytdl_sub.entries.entry import Entry
from ytdl_sub.subscriptions.utils import SUBSCRIPTION_NAME
from ytdl_sub.entries.variables.override_variables import SUBSCRIPTION_NAME
from ytdl_sub.utils.exceptions import ValidationException
from ytdl_sub.validators.file_path_validators import OverridesStringFormatterFilePathValidator
from ytdl_sub.validators.file_path_validators import StringFormatterFileNameValidator

57
src/ytdl_sub/entries/variables/override_variables.py Normal file
View file

@ -0,0 +1,57 @@
SUBSCRIPTION_NAME = "subscription_name"
SUBSCRIPTION_VALUE = "subscription_value"
class OverrideVariables:
@classmethod
def subscription_name(cls) -> str:
"""
Name of the subscription
"""
return SUBSCRIPTION_NAME
@classmethod
def subscription_value(cls) -> str:
"""
For subscriptions in the form of
.. code-block:: yaml
"Subscription Name": "https://..."
``subscription_value`` gets set to ``https://...``.
"""
return SUBSCRIPTION_VALUE
@classmethod
def subscription_indent_i(cls, index: int) -> str:
"""
For subscriptions in the form of
.. code-block:: yaml
Preset | = Indent Value 1:
= Indent Value 2:
"Subscription Name": "https://..."
``subscription_indent_1`` and ``subscription_indent_2`` get set to
``Indent Value 1`` and ``Indent Value 2``.
"""
return f"subscription_indent_{index + 1}"
@classmethod
def subscription_value_i(cls, index: int) -> str:
"""
For subscriptions in the form of
.. code-block:: yaml
"Subscription Name":
- "https://url1.com/..."
- "https://url2.com/..."
``subscription_value_1`` and ``subscription_value_2`` get set to ``https://url1.com/...``
and ``https://url2.com/...``. Note that ``subscription_value_1`` also gets set to
``subscription_value``.
"""
return f"subscription_value_{index + 1}"

13
src/ytdl_sub/subscriptions/subscription_validators.py
View file

@ -9,10 +9,9 @@ from typing import final
from ytdl_sub.config.config_file import ConfigFile
from ytdl_sub.config.preset_options import Overrides
from ytdl_sub.subscriptions.utils import SUBSCRIPTION_NAME
from ytdl_sub.subscriptions.utils import SUBSCRIPTION_VALUE
from ytdl_sub.subscriptions.utils import subscription_indent_variable_name
from ytdl_sub.subscriptions.utils import subscription_list_variable_name
from ytdl_sub.entries.variables.override_variables import SUBSCRIPTION_NAME
from ytdl_sub.entries.variables.override_variables import SUBSCRIPTION_VALUE
from ytdl_sub.entries.variables.override_variables import OverrideVariables
from ytdl_sub.validators.string_formatter_validators import DictFormatterValidator
from ytdl_sub.validators.validators import DictValidator
from ytdl_sub.validators.validators import StringListValidator
@ -33,7 +32,7 @@ class SubscriptionOutput(Validator, ABC):
indent overrides to merge with the preset dict's overrides
"""
return {
subscription_indent_variable_name(i): self._indent_overrides[i]
OverrideVariables.subscription_indent_i(i): self._indent_overrides[i]
for i in range(len(self._indent_overrides))
}
@ -159,7 +158,9 @@ class SubscriptionListValuesValidator(SubscriptionLeafValidator, StringListValid
if idx == 0:
self._overrides_to_add[SUBSCRIPTION_VALUE] = list_value.value
self._overrides_to_add[subscription_list_variable_name(index=idx)] = list_value.value
self._overrides_to_add[
OverrideVariables.subscription_value_i(index=idx)
] = list_value.value
class SubscriptionWithOverridesValidator(SubscriptionLeafValidator, DictFormatterValidator):

31
src/ytdl_sub/subscriptions/utils.py
View file

@ -1,33 +1,2 @@
SUBSCRIPTION_NAME = "subscription_name"
SUBSCRIPTION_VALUE = "subscription_value"
# Key used in configs, should delete at some point
SUBSCRIPTION_VALUE_CONFIG_KEY = "subscription_value"
def subscription_indent_variable_name(index: int) -> str:
"""
Parameters
----------
index
0th-based index
Returns
-------
subscription_index_i, where i is 1-based index
"""
return f"subscription_indent_{index + 1}"
def subscription_list_variable_name(index: int) -> str:
"""
Parameters
----------
index
0th-based index
Returns
-------
subscription_value_i, where i is 1-based index
"""
return f"subscription_value_{index + 1}"