From 31e4da9ff78e1f3519ae11f5bb2f64c8a7f49666 Mon Sep 17 00:00:00 2001 From: Jesse Bannon Date: Wed, 23 Oct 2024 21:34:38 -0700 Subject: [PATCH] [DOCS] Music presets + more (#1095) --- .../config_reference/subscriptions_yaml.rst | 43 ++----------- docs/source/guides/development/index.rst | 63 ++++++++++++++++++- docs/source/prebuilt_presets/helpers.rst | 22 +++++++ docs/source/prebuilt_presets/index.rst | 3 +- docs/source/prebuilt_presets/music.rst | 63 ++++++++++++++++++- docs/source/prebuilt_presets/music_videos.rst | 5 ++ 6 files changed, 157 insertions(+), 42 deletions(-) create mode 100644 docs/source/prebuilt_presets/music_videos.rst diff --git a/docs/source/config_reference/subscriptions_yaml.rst b/docs/source/config_reference/subscriptions_yaml.rst index d97b65f9..508fbc82 100644 --- a/docs/source/config_reference/subscriptions_yaml.rst +++ b/docs/source/config_reference/subscriptions_yaml.rst @@ -1,16 +1,13 @@ ================== Subscriptions File ================== ------------------- -subscriptions.yaml ------------------- -The ``subscriptions.yaml`` file is where we use our :ref:`config_reference/config_yaml:presets` in the :ref:`config_reference/config_yaml:config.yaml` -to define a ``subscription``: something we want to recurrently download such as a specific +The ``subscriptions.yaml`` file is where we use :ref:`config_reference/config_yaml:presets` +to define a ``subscription``: something we want to recurrently download, such as a specific channel or playlist. The only difference between a ``subscription`` and ``preset`` is that the subscription -must have all required fields and ``{override_variables}`` defined so it can perform a download. +must have all required variables defined to perform a download. Below is an example that downloads a YouTube playlist: @@ -41,7 +38,7 @@ the ``parent preset`` to ``playlist_preset_ex``, and must define the variables ` and ``{url}`` since the preset did not. Beautifying Subscriptions -~~~~~~~~~~~~~~~~~~~~~~~~~ +------------------------- Subscriptions support using presets as keys, and using keys to set override variables as values. For example: @@ -96,10 +93,8 @@ parent preset: genre: "{subscription_indent_1}" content_rating: "{subscription_indent_2}" -.. _subscription value: - File Preset -~~~~~~~~~~~ +----------- You can apply a preset to all subscriptions in the ``subscription.yaml`` file by using the file-wide ``__preset__``: @@ -117,31 +112,3 @@ by using the file-wide ``__preset__``: This ``subscription.yaml`` is equivalent to the one above it because all subscriptions automatically set ``__preset__`` as a ``parent preset``. - - -Subscription Value -~~~~~~~~~~~~~~~~~~~ -NOTE: This is deprecated in favor of using the method in :ref:`config_reference/subscriptions_yaml:beautifying subscriptions`. - -With a clever config and use of ``__preset__``, your subscriptions can typically boil -down to a name and url. You can set ``__value__`` to the name of an override variable, -and use the override variable ``subscription_name`` to achieve one-liner subscriptions. -Using the example above, we can do: - -.. code-block:: yaml - :caption: subscription.yaml - - __preset__: - preset: - - "tv_show" - overrides: - tv_show_name: "{subscription_name}" - - __value__: "url" - - # single-line subscription, sets "Brandon Acker" and the subscription value - # to the override variables tv_show_name and url - "Brandon Acker": "https://www.youtube.com/@brandonacker" - -Traditional subscriptions that can override presets will still work when using ``__value__``. -``__value__`` can also be set within a :ref:`config_reference/config_yaml:config.yaml`. \ No newline at end of file diff --git a/docs/source/guides/development/index.rst b/docs/source/guides/development/index.rst index e606005d..bf6b2c12 100644 --- a/docs/source/guides/development/index.rst +++ b/docs/source/guides/development/index.rst @@ -1,5 +1,64 @@ Development and Contributing ============================ -.. toctree:: - \ No newline at end of file +Requirements +------------ +- python >= 3.10 +- ffmpeg/ffprobe 4.4.5 (test checksums rely on this version) +- make + + +Local Install +------------- +.. tab-set-code:: + + .. code-block:: shell + + pip install -e .[test,lint,docs] + + .. code-block:: zsh + + pip install -e .\[test,lint,docs\] + +Linter +------ +All source code contributed must be formatted to our linter specification. +Run the following to auto-format and check for any issues with your code: + +.. code-block:: shell + + make lint + +Adding Documentation +-------------------- + +Docs can be found in ``ytdl-sub/docs/source/``, and are built using the command: + + +.. code-block:: shell + :caption: Viewable at http://localhost:63342/ytdl-sub/docs/build/html/index.html once built + + make docs + +Some of the documentation is built using doc-strings from the python source code. The above +command will rebuild those as well. + +Testing +------- +Tests are written using pytest. Many of them evaluate checksums of output files to ensure no unintended +changes are introduced to the way ``ytdl-sub`` produces files. This checksum can be inaccurate for +end-to-end tests, but are reliable for integration tests. + +If integration tests are failing, ensure... + +- you're using the correct ffmpeg version +- you are developing on Linux or Mac (have not tested windows yet) +- your local ``ytdl-sub`` dependencies are up-to-date + +IDE Setup +--------- +PyCharm is our preferred IDE. The codebase is simple enough to where it's not required, but +is highly recommended. + +TODO: screenshots of configuration + diff --git a/docs/source/prebuilt_presets/helpers.rst b/docs/source/prebuilt_presets/helpers.rst index 749c4c02..a4b141c6 100644 --- a/docs/source/prebuilt_presets/helpers.rst +++ b/docs/source/prebuilt_presets/helpers.rst @@ -20,6 +20,28 @@ Helper Presets Common presets are not usable by themselves- setting one of these as the sole preset of your subscription and attempting to download will not work. But you can add these presets to quickly modify an existing preset to better suit your needs. +Only Recent +----------- + +To only download a recent number of videos, apply the ``Only Recent`` preset. Once a video's +upload date is outside of the range, or you hit max files, older videos will be deleted automatically. + +.. code-block:: yaml + + __preset__: + overrides: + # Set to a non-zero value to only keep this many files at once per sub + only_recent_max_files: 0 + only_recent_date_range: "7days" + + Plex TV Show by Date | Only Recent: + + = Documentaries: + "NOVA PBS": "https://www.youtube.com/@novapbs" + +To prevent deletion of files, use the preset ``Only Recent Archive`` instead. + + Best A/V Quality ---------------- diff --git a/docs/source/prebuilt_presets/index.rst b/docs/source/prebuilt_presets/index.rst index 00efc8ba..744428ce 100644 --- a/docs/source/prebuilt_presets/index.rst +++ b/docs/source/prebuilt_presets/index.rst @@ -11,4 +11,5 @@ definitions :doc:`here `. helpers tv_shows - music \ No newline at end of file + music + music_videos \ No newline at end of file diff --git a/docs/source/prebuilt_presets/music.rst b/docs/source/prebuilt_presets/music.rst index 60552eea..23323c6b 100644 --- a/docs/source/prebuilt_presets/music.rst +++ b/docs/source/prebuilt_presets/music.rst @@ -1,3 +1,64 @@ ============= Music Presets -============= \ No newline at end of file +============= + +Music downloadable by yt-dlp comes in many flavors. ``ytdl-sub`` offers a suite +of various presets for handling some of the most popular forms of uploaded music +content. + +YouTube Releases +---------------- +Many artists, especially those auto-uploaded as ``Topics`` in YouTube have a section on +their channel named "Releases", or "Albums and Singles". The ``YouTube Releases`` preset aims to +scrape this *playlist of playlists*. + +Playlists are recognized as the album, and videos within it are tracks. + +.. code-block:: yaml + + YouTube Releases: + = Jazz: # Sets genre tag to "Jazz" + "Thelonious Monk": "https://www.youtube.com/@theloniousmonk3870/releases" + +.. hint:: + + The subscription *value* (denoted by =) will set the genre tag for all music scraped under its key. + +YouTube Full Albums +------------------- +In many cases, albums are uploaded to YouTube as a single video, where each track as separated by either +chapters or timestamps in a description. The ``YouTube Full Albums`` preset will take each video and split +it by the chapters to form an album. + +Videos are recognized as the album, and chapters within it are tracks. + +.. code-block:: yaml + + YouTube Full Albums: + = Lofi: + "Game Chops": "https://www.youtube.com/playlist?list=PLBsm_SagFMmdWnCnrNtLjA9kzfrRkto4i" + +Soundcloud Discography +---------------------- +SoundCloud tracks can be uploaded as either a single, part of an album, or a collaboration +with another artist. At this time, ``SoundCloud Discography`` only scrapes singles and albums. +It will attempt to group tracks into albums before falling back to single format. + +.. code-block:: yaml + + SoundCloud Discography: + = Chill Hop: + "UKNOWY": "https://soundcloud.com/uknowymunich" + = Synthwave: + "Lazerdiscs Records": "https://soundcloud.com/lazerdiscsrecords" + "Earmake": "https://soundcloud.com/earmake" + +Bandcamp +-------- +Bandcamp albums and singles can be scraped using the ``Bandcamp`` preset. + +.. code-block:: yaml + + Bandcamp: + = Lofi: + "Emily Hopkins": "https://emilyharpist.bandcamp.com/" diff --git a/docs/source/prebuilt_presets/music_videos.rst b/docs/source/prebuilt_presets/music_videos.rst new file mode 100644 index 00000000..46e6fcf9 --- /dev/null +++ b/docs/source/prebuilt_presets/music_videos.rst @@ -0,0 +1,5 @@ +=================== +Music Video Presets +=================== + +WIP \ No newline at end of file