danvics/ytdl-sub
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

plugins WIP

Browse source
This commit is contained in:
Jesse Bannon 2023-12-28 13:09:43 -08:00
parent 76703f7e7c
commit 724faaac9e
6 changed files with 305 additions and 204 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

316
docs/source/config_reference/plugins.rst
View file

@ -6,25 +6,29 @@ audio_extract
-------------
Extracts audio from a video file.
Usage:
:Usage:
.. code-block:: yaml
presets:
my_example_preset:
audio_extract:
codec: "mp3"
quality: 128
codec
~~~~~
The codec to output after extracting the audio. Supported codecs are aac, flac, mp3, m4a,
opus, vorbis, wav, and best to grab the best possible format at runtime.
``codec``
:expected type: String
:description:
The codec to output after extracting the audio. Supported codecs are aac, flac, mp3, m4a,
opus, vorbis, wav, and best to grab the best possible format at runtime.
``quality``
:expected type: Float
:description:
Optional. Specify ffmpeg audio quality. Insert a value between ``0`` (better) and ``9``
(worse) for variable bitrate, or a specific bitrate like ``128`` for 128k.
quality
~~~~~~~
Optional. Specify ffmpeg audio quality. Insert a value between ``0`` (better) and ``9``
(worse) for variable bitrate, or a specific bitrate like ``128`` for 128k.
----------------------------------------------------------------------------------------------------
@ -33,12 +37,10 @@ chapters
Embeds chapters to video files if they are present. Additional options to add SponsorBlock
chapters and remove specific ones. Can also remove chapters using regex.
Usage:
:usage:
.. code-block:: yaml
presets:
my_example_preset:
chapters:
# Embedded Chapter Fields
embed_chapters: True
@ -59,61 +61,92 @@ Usage:
remove_sponsorblock_categories: "all"
force_key_frames: False
allow_chapters_from_comments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Optional. If chapters do not exist in the video/description itself, attempt to scrape
comments to find the chapters. Defaults to False.
``allow_chapters_from_comments``
embed_chapters
~~~~~~~~~~~~~~
Optional. Embed chapters into the file. Defaults to True.
:expected type: Optional[Boolean]
:description:
Defaults to False. If chapters do not exist in the video/description itself, attempt to
scrape comments to find the chapters.
force_key_frames
~~~~~~~~~~~~~~~~
Optional. Force keyframes at cuts when removing sections. This is slow due to needing a
re-encode, but the resulting video may have fewer artifacts around the cuts. Defaults to
False.
remove_chapters_regex
~~~~~~~~~~~~~~~~~~~~~
Optional. List of regex patterns to match chapter titles against and remove them from the
entry.
``embed_chapters``
remove_sponsorblock_categories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Optional. List of SponsorBlock categories to remove from the output file. Can only remove
categories that are specified in ``sponsorblock_categories`` or "all", which removes
everything specified in ``sponsorblock_categories``.
:expected type: Optional[Boolean]
:description:
Defaults to True. Embed chapters into the file.
``force_key_frames``
:expected type: Optional[Boolean]
:description:
Defaults to False. Force keyframes at cuts when removing sections. This is slow due to
needing a re-encode, but the resulting video may have fewer artifacts around the cuts.
``remove_chapters_regex``
:expected type: Optional[List[RegexString]
:description:
List of regex patterns to match chapter titles against and remove them from the
entry.
``remove_sponsorblock_categories``
:expected type: Optional[List[String]]
:description:
List of SponsorBlock categories to remove from the output file. Can only remove
categories that are specified in ``sponsorblock_categories`` or "all", which removes
everything specified in ``sponsorblock_categories``.
``sponsorblock_categories``
:expected type: Optional[List[String]]
:description:
List of SponsorBlock categories to embed as chapters. Supports "sponsor",
"intro", "outro", "selfpromo", "preview", "filler", "interaction", "music_offtopic",
"poi_highlight", or "all" to include all categories.
sponsorblock_categories
~~~~~~~~~~~~~~~~~~~~~~~
Optional. List of SponsorBlock categories to embed as chapters. Supports "sponsor",
"intro", "outro", "selfpromo", "preview", "filler", "interaction", "music_offtopic",
"poi_highlight", or "all" to include all categories.
----------------------------------------------------------------------------------------------------
date_range
----------
Only download files uploaded within the specified date range.
Dates must adhere to a yt-dlp datetime. From their docs:
Usage:
.. code-block:: Markdown
A string in the format YYYYMMDD or
(now|today|yesterday|date)[+-][0-9](microsecond|second|minute|hour|day|week|month|year)(s)
Valid examples are ``now-2weeks`` or ``20200101``. Can use override variables in this.
Note that yt-dlp will round times to the closest day, meaning that `day` is the lowest
granularity possible.
:usage:
.. code-block:: yaml
presets:
my_example_preset:
date_range:
before: "now"
after: "today-2weeks"
after
~~~~~
Optional. Only download videos after this datetime.
``after``
:expected type: Optional[OverridesFormatter]
:description:
Only download videos before this datetime.
``before``
:expected type: Optional[OverridesFormatter]
:description:
Only download videos before this datetime.
before
~~~~~~
Optional. Only download videos before this datetime.
----------------------------------------------------------------------------------------------------
@ -159,21 +192,23 @@ Supports custom ffmpeg conversions:
-acodec copy
-scodec mov_text
convert_to
~~~~~~~~~~
``convert_to``
Convert to a desired file type. Supports:
* Video: avi, flv, mkv, mov, mp4, webm
* Audio: aac, flac, mp3, m4a, opus, vorbis, wav
convert_with
~~~~~~~~~~~~
``convert_with``
Optional. Supports ``yt-dlp`` and ``ffmpeg``. ``yt-dlp`` will convert files within
yt-dlp whereas ``ffmpeg`` specifies it will be converted using a custom command specified
with ``ffmpeg_post_process_args``. Defaults to ``yt-dlp``.
ffmpeg_post_process_args
~~~~~~~~~~~~~~~~~~~~~~~~
``ffmpeg_post_process_args``
Optional. ffmpeg args to post-process an entry file with. The args will be inserted in the
form of:
@ -184,6 +219,7 @@ form of:
The output file will use the extension specified in ``convert_to``. Post-processing args
can still be set with ``convert_with`` set to ``yt-dlp``.
----------------------------------------------------------------------------------------------------
filter_exclude
@ -245,10 +281,11 @@ Usage:
my_example_preset:
format: "(bv*[height<=1080]+bestaudio/best[height<=1080])"
format
~~~~~~
``format``
yt-dlp format, uses same syntax as yt-dlp.
----------------------------------------------------------------------------------------------------
match_filters
@ -280,12 +317,13 @@ Supports one or multiple filters:
# - "age_limit<?18"
# - "availability=?public"
filters
~~~~~~~
``filters``
The filters themselves. If used multiple times, the filter matches if at least one of the
conditions are met. For logical AND's between match filters, use the ``&`` operator in
a single match filter. These are applied when gathering metadata.
----------------------------------------------------------------------------------------------------
music_tags
@ -316,15 +354,17 @@ Usage:
- "{artist}"
- "ytdl-sub"
embed_thumbnail
~~~~~~~~~~~~~~~
``embed_thumbnail``
Optional. Whether to embed the thumbnail into the audio file.
tags
~~~~
``tags``
Key, values of tag names, tag values. Supports source and override variables.
Supports lists which will get written to MP3s as id3v2.4 multi-tags.
----------------------------------------------------------------------------------------------------
nfo_tags
@ -349,18 +389,20 @@ Usage:
# optional
kodi_safe: False
kodi_safe
~~~~~~~~~
``kodi_safe``
Optional. Kodi does not support > 3-byte unicode characters, which include emojis and some
foreign language characters. Setting this to True will replace those characters with '□'.
Defaults to False.
nfo_name
~~~~~~~~
``nfo_name``
The NFO file name.
nfo_root
~~~~~~~~
``nfo_root``
The root tag of the NFO's XML. In the usage above, it would look like
.. code-block:: xml
@ -369,8 +411,9 @@ The root tag of the NFO's XML. In the usage above, it would look like
<episodedetails>
</episodedetails>
tags
~~~~
``tags``
Tags within the nfo_root tag. In the usage above, it would look like
.. code-block:: xml
@ -403,6 +446,7 @@ Which translates to
<genre>Comedy</genre>
<genre>Drama</genre>
----------------------------------------------------------------------------------------------------
output_directory_nfo_tags
@ -426,18 +470,20 @@ Usage:
# optional
kodi_safe: False
kodi_safe
~~~~~~~~~
``kodi_safe``
Optional. Kodi does not support > 3-byte unicode characters, which include emojis and some
foreign language characters. Setting this to True will replace those characters with '□'.
Defaults to False.
nfo_name
~~~~~~~~
``nfo_name``
The NFO file name.
nfo_root
~~~~~~~~
``nfo_root``
The root tag of the NFO's XML. In the usage above, it would look like
.. code-block:: xml
@ -446,8 +492,9 @@ The root tag of the NFO's XML. In the usage above, it would look like
<tvshow>
</tvshow>
tags
~~~~
``tags``
Tags within the nfo_root tag. In the usage above, it would look like
.. code-block:: xml
@ -478,6 +525,7 @@ Which translates to
<genre>Comedy</genre>
<genre>Drama</genre>
----------------------------------------------------------------------------------------------------
output_options
@ -503,47 +551,53 @@ Usage:
keep_files_before: now
keep_files_after: 19000101
download_archive_name
~~~~~~~~~~~~~~~~~~~~~
``download_archive_name``
Optional. The file name to store a subscriptions download archive placed relative to
the output directory. Defaults to ``.ytdl-sub-{subscription_name}-download-archive.json``
file_name
~~~~~~~~~
``file_name``
Required. The file name for the media file. This can include directories such as
``"Season {upload_year}/{title}.{ext}"``, and will be placed in the output directory.
info_json_name
~~~~~~~~~~~~~~
``info_json_name``
Optional. The file name for the media's info json file. This can include directories such
as ``"Season {upload_year}/{title}.{info_json_ext}"``, and will be placed in the output
directory. Can be set to empty string or `null` to disable info json writes.
keep_files_after
~~~~~~~~~~~~~~~~
``keep_files_after``
Optional. Requires ``maintain_download_archive`` set to True.
Only keeps files that are uploaded after this datetime. By default, ytdl-sub will keep
files after ``19000101``, which implies all files. Can be used in conjunction with
``keep_max_files``.
keep_files_before
~~~~~~~~~~~~~~~~~
``keep_files_before``
Optional. Requires ``maintain_download_archive`` set to True.
Only keeps files that are uploaded before this datetime. By default, ytdl-sub will keep
files before ``now``, which implies all files. Can be used in conjunction with
``keep_max_files``.
keep_max_files
~~~~~~~~~~~~~~
``keep_max_files``
Optional. Requires ``maintain_download_archive`` set to True.
Only keeps N most recently uploaded videos. If set to <= 0, ``keep_max_files`` will not be
applied. Can be used in conjunction with ``keep_files_before`` and ``keep_files_after``.
maintain_download_archive
~~~~~~~~~~~~~~~~~~~~~~~~~
``maintain_download_archive``
Optional. Maintains a download archive file in the output directory for a subscription.
It is named ``.ytdl-sub-{subscription_name}-download-archive.json``, stored in the
output directory.
@ -554,23 +608,27 @@ prevent ytdl from redownloading media already downloaded.
Defaults to False.
migrated_download_archive_name
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``migrated_download_archive_name``
Optional. Intended to be used if you are migrating a subscription with either a new
subscription name or output directory. It will try to load the archive file using this name
first, and fallback to ``download_archive_name``. It will always save to this file
and remove the original ``download_archive_name``.
output_directory
~~~~~~~~~~~~~~~~
``output_directory``
Required. The output directory to store all media files downloaded.
thumbnail_name
~~~~~~~~~~~~~~
``thumbnail_name``
Optional. The file name for the media's thumbnail image. This can include directories such
as ``"Season {upload_year}/{title}.{thumbnail_ext}"``, and will be placed in the output
directory. Can be set to empty string or `null` to disable thumbnail writes.
----------------------------------------------------------------------------------------------------
overrides
@ -665,11 +723,12 @@ Usage:
- "{upload_month}"
- "{upload_day}"
skip_if_match_fails
~~~~~~~~~~~~~~~~~~~
``skip_if_match_fails``
Defaults to True. If True, when any match fails and has no defaults, the entry will be
skipped. If False, ytdl-sub will error and all downloads will not proceed.
----------------------------------------------------------------------------------------------------
split_by_chapters
@ -693,11 +752,12 @@ Usage:
split_by_chapters:
when_no_chapters: "pass"
when_no_chapters
~~~~~~~~~~~~~~~~
``when_no_chapters``
Behavior to perform when no chapters are present. Supports "pass" (continue processing),
"drop" (exclude it from output), and "error" (stop processing for everything).
----------------------------------------------------------------------------------------------------
subtitles
@ -719,31 +779,36 @@ Usage:
languages: "en" # supports list of multiple languages
allow_auto_generated_subtitles: False
allow_auto_generated_subtitles
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``allow_auto_generated_subtitles``
Optional. Whether to allow auto generated subtitles. Defaults to False.
embed_subtitles
~~~~~~~~~~~~~~~
``embed_subtitles``
Optional. Whether to embed the subtitles into the video file. Defaults to False.
NOTE: webm files can only embed "vtt" subtitle types.
languages
~~~~~~~~~
``languages``
Optional. Language code(s) to download for subtitles. Supports a single or list of multiple
language codes. Defaults to "en".
subtitles_name
~~~~~~~~~~~~~~
``subtitles_name``
Optional. The file name for the media's subtitles if they are present. This can include
directories such as ``"Season {upload_year}/{title_sanitized}.{lang}.{subtitles_ext}"``, and
will be placed in the output directory. ``lang`` is dynamic since you can download multiple
subtitles. It will set the respective language to the correct subtitle file.
subtitles_type
~~~~~~~~~~~~~~
``subtitles_type``
Optional. One of the subtitle file types "srt", "vtt", "ass", "lrc". Defaults to "srt"
----------------------------------------------------------------------------------------------------
throttle_protection
@ -770,25 +835,29 @@ Usage:
max: 36
subscription_download_probability: 1.0
max_downloads_per_subscription
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``max_downloads_per_subscription``
Number of downloads to perform per subscription.
sleep_per_download_s
~~~~~~~~~~~~~~~~~~~~
``sleep_per_download_s``
Number in seconds to sleep between each download. Does not include time it takes for
ytdl-sub to perform post-processing.
sleep_per_subscription_s
~~~~~~~~~~~~~~~~~~~~~~~~
``sleep_per_subscription_s``
Number in seconds to sleep between each subscription.
subscription_download_probability
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
``subscription_download_probability``
Probability to perform any downloads, recomputed for each subscription. This is only
recommended to set if you run ytdl-sub in a cron-job, that way you are statistically
guaranteed over time to eventually download the subscription.
----------------------------------------------------------------------------------------------------
video_tags
@ -806,10 +875,11 @@ Usage:
date: "{upload_date}"
description: "{description}"
tags
~~~~
``tags``
Key/values of tag names/values. Supports source and override variables.
----------------------------------------------------------------------------------------------------
ytdl_options

2
docs/source/prebuilt_presets/tv_shows.rst
View file

@ -18,7 +18,7 @@ Jellyfin
Kodi
~~~~
* Everything that the Jellyfin version does
* Turns on :ref:`config_reference/plugins:kodi_safe`, replacing characters that would break kodi with safer characters
* Enables ``kodi_safe`` NFOs, replacing 4-byte unicode characters that break kodi with ``□``
Plex
~~~~

8
src/ytdl_sub/plugins/audio_extract.py
View file

@ -25,12 +25,10 @@ class AudioExtractOptions(OptionsDictValidator):
"""
Extracts audio from a video file.
Usage:
:Usage:
.. code-block:: yaml
presets:
my_example_preset:
audio_extract:
codec: "mp3"
quality: 128
@ -56,6 +54,8 @@ class AudioExtractOptions(OptionsDictValidator):
@property
def codec(self) -> str:
"""
:expected type: String
:description:
The codec to output after extracting the audio. Supported codecs are aac, flac, mp3, m4a,
opus, vorbis, wav, and best to grab the best possible format at runtime.
"""
@ -64,6 +64,8 @@ class AudioExtractOptions(OptionsDictValidator):
@property
def quality(self) -> Optional[float]:
"""
:expected type: Float
:description:
Optional. Specify ffmpeg audio quality. Insert a value between ``0`` (better) and ``9``
(worse) for variable bitrate, or a specific bitrate like ``128`` for 128k.
"""

34
src/ytdl_sub/plugins/chapters.py
View file

@ -64,12 +64,10 @@ class ChaptersOptions(OptionsDictValidator):
Embeds chapters to video files if they are present. Additional options to add SponsorBlock
chapters and remove specific ones. Can also remove chapters using regex.
Usage:
:usage:
.. code-block:: yaml
presets:
my_example_preset:
chapters:
# Embedded Chapter Fields
embed_chapters: True
@ -89,7 +87,6 @@ class ChaptersOptions(OptionsDictValidator):
- "intro"
remove_sponsorblock_categories: "all"
force_key_frames: False
"""
_optional_keys = {
@ -135,22 +132,28 @@ class ChaptersOptions(OptionsDictValidator):
@property
def embed_chapters(self) -> Optional[bool]:
"""
Optional. Embed chapters into the file. Defaults to True.
:expected type: Optional[Boolean]
:description:
Defaults to True. Embed chapters into the file.
"""
return self._embed_chapters
@property
def allow_chapters_from_comments(self) -> bool:
"""
Optional. If chapters do not exist in the video/description itself, attempt to scrape
comments to find the chapters. Defaults to False.
:expected type: Optional[Boolean]
:description:
Defaults to False. If chapters do not exist in the video/description itself, attempt to
scrape comments to find the chapters.
"""
return self._allow_chapters_from_comments
@property
def remove_chapters_regex(self) -> Optional[List[re.Pattern]]:
"""
Optional. List of regex patterns to match chapter titles against and remove them from the
:expected type: Optional[List[RegexString]
:description:
List of regex patterns to match chapter titles against and remove them from the
entry.
"""
if self._remove_chapters_regex:
@ -160,7 +163,9 @@ class ChaptersOptions(OptionsDictValidator):
@property
def sponsorblock_categories(self) -> Optional[List[str]]:
"""
Optional. List of SponsorBlock categories to embed as chapters. Supports "sponsor",
:expected type: Optional[List[String]]
:description:
List of SponsorBlock categories to embed as chapters. Supports "sponsor",
"intro", "outro", "selfpromo", "preview", "filler", "interaction", "music_offtopic",
"poi_highlight", or "all" to include all categories.
"""
@ -174,7 +179,9 @@ class ChaptersOptions(OptionsDictValidator):
@property
def remove_sponsorblock_categories(self) -> Optional[List[str]]:
"""
Optional. List of SponsorBlock categories to remove from the output file. Can only remove
:expected type: Optional[List[String]]
:description:
List of SponsorBlock categories to remove from the output file. Can only remove
categories that are specified in ``sponsorblock_categories`` or "all", which removes
everything specified in ``sponsorblock_categories``.
"""
@ -190,9 +197,10 @@ class ChaptersOptions(OptionsDictValidator):
@property
def force_key_frames(self) -> bool:
"""
Optional. Force keyframes at cuts when removing sections. This is slow due to needing a
re-encode, but the resulting video may have fewer artifacts around the cuts. Defaults to
False.
:expected type: Optional[Boolean]
:description:
Defaults to False. Force keyframes at cuts when removing sections. This is slow due to
needing a re-encode, but the resulting video may have fewer artifacts around the cuts.
"""
return self._force_key_frames

22
src/ytdl_sub/plugins/date_range.py
View file

@ -11,13 +11,21 @@ from ytdl_sub.validators.string_datetime import StringDatetimeValidator
class DateRangeOptions(OptionsDictValidator):
"""
Only download files uploaded within the specified date range.
Dates must adhere to a yt-dlp datetime. From their docs:
Usage:
.. code-block:: Markdown
A string in the format YYYYMMDD or
(now|today|yesterday|date)[+-][0-9](microsecond|second|minute|hour|day|week|month|year)(s)
Valid examples are ``now-2weeks`` or ``20200101``. Can use override variables in this.
Note that yt-dlp will round times to the closest day, meaning that `day` is the lowest
granularity possible.
:usage:
.. code-block:: yaml
presets:
my_example_preset:
date_range:
before: "now"
after: "today-2weeks"
@ -33,14 +41,18 @@ class DateRangeOptions(OptionsDictValidator):
@property
def before(self) -> Optional[StringDatetimeValidator]:
"""
Optional. Only download videos before this datetime.
:expected type: Optional[OverridesFormatter]
:description:
Only download videos before this datetime.
"""
return self._before
@property
def after(self) -> Optional[StringDatetimeValidator]:
"""
Optional. Only download videos after this datetime.
:expected type: Optional[OverridesFormatter]
:description:
Only download videos before this datetime.
"""
return self._after

13
tools/docgen/plugins.py
View file

@ -1,10 +1,10 @@
import inspect
from pathlib import Path
from typing import Dict
from typing import Dict, Any, Optional
from typing import Type
from tools.docgen.docgen import DocGen
from tools.docgen.utils import get_function_docs, line_section
from tools.docgen.utils import line_section
from tools.docgen.utils import properties
from tools.docgen.utils import section
from ytdl_sub.config.overrides import Overrides
@ -26,6 +26,15 @@ def should_filter_property(property_name: str) -> bool:
)
def get_function_docs(
function_name: str, obj: Any, level: int
) -> str:
docs = f"\n``{function_name}``\n\n"
docs += inspect.cleandoc(getattr(obj, function_name).__doc__)
docs += "\n\n"
return docs
def generate_plugin_docs(name: str, options: Type[OptionsValidator], offset: int) -> str:
docs = ""
docs += section(name, level=offset + 0)