[DOCS] How to test subscription downloads (#1318)
Revise the "Getting Started" page for testing subscription downloads.
This commit is contained in:
parent
a7cf361821
commit
72419a3e98
6 changed files with 78 additions and 84 deletions
|
|
@ -172,7 +172,7 @@ need to be removed:
|
|||
necessary>`. To re-download, those options must be disabled or modified. Disable
|
||||
:ref:`the 'break_on_existing' option <config_reference/plugins:ytdl_options>`, set
|
||||
:ref:`the 'date_range:' plugin <config_reference/plugins:date_range>`, and :ref:`limit
|
||||
the subscriptions <guides/getting_started/first_download:faster iteration cycle>` to
|
||||
the subscriptions <guides/getting_started/downloading:preview>` to
|
||||
download only the files that you've renamed in the steps above.
|
||||
|
||||
Set the appropriate dates, :ref:`including a sufficient margin
|
||||
|
|
@ -182,11 +182,11 @@ need to be removed:
|
|||
|
||||
.. code-block:: shell
|
||||
|
||||
ytdl-sub sub -o "\
|
||||
ytdl-sub --match="NOVA PBS" sub -o "\
|
||||
--ytdl_options.break_on_existing False \
|
||||
--date_range.after 20240101 \
|
||||
--date_range.before 20250101 \
|
||||
" --match="NOVA PBS"
|
||||
"
|
||||
|
||||
...download a file missing from the archive
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
|
|
|||
69
docs/source/guides/getting_started/downloading.rst
Normal file
69
docs/source/guides/getting_started/downloading.rst
Normal file
|
|
@ -0,0 +1,69 @@
|
|||
Downloading
|
||||
===========
|
||||
|
||||
Once you've :doc:`defined your subscriptions <./subscriptions>`, it's time to test your
|
||||
configuration and try your first download. As a web scraping tool, :ref:`it's important
|
||||
to minimize the requests sent to external services
|
||||
<guides/getting_started/index:minimize the work to only what's necessary>` to avoid
|
||||
triggering throttling or bans. Further, a full download of even one subscription can
|
||||
take significant time. Test each change to your subscriptions carefully and quickly as
|
||||
follows.
|
||||
|
||||
|
||||
Preview
|
||||
-------
|
||||
|
||||
Preview what ``ytdl-sub`` would do for this subscription. Run the :ref:`'sub'
|
||||
sub-command <usage:subscriptions options>` with CLI options to restrict requests as much
|
||||
as possible:
|
||||
|
||||
- Pull metadata and *simulate* a download without actually downloading any media files
|
||||
using the ``--dry-run`` CLI option.
|
||||
|
||||
- Limit requests by narrowing the run to one subscription by giving a
|
||||
subscription name to the ``--match`` CLI option.
|
||||
|
||||
- Stop after just a few downloads to further minimize requests and make testing faster
|
||||
using the ``max_downloads`` setting from ``yt-dlp``.
|
||||
|
||||
Change to the directory containing your ``./subscriptions.yaml`` file and run with those
|
||||
options:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
cd "/config/ytdl-sub-configs/"
|
||||
ytdl-sub --dry-run --match="NOVA PBS" sub -o '--ytdl_options.max_downloads 3'
|
||||
|
||||
Examine the output carefully, investigate anything that doesn't look right and repeat
|
||||
this step until everything looks right.
|
||||
|
||||
|
||||
Review
|
||||
------
|
||||
|
||||
Review the results of real downloads. Run it again without the ``--dry-run`` option to
|
||||
actually download media and place the files in your library:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
ytdl-sub --match="NOVA PBS" sub -o '--ytdl_options.max_downloads 3'
|
||||
|
||||
Examine the output carefully again. Then examine how the resulting downloads work in
|
||||
your library. Repeat with a larger value for ``max_downloads`` and examine the output
|
||||
and downloads again.
|
||||
|
||||
|
||||
Next Steps
|
||||
----------
|
||||
|
||||
Once you're `previewed <preview_>`_ and `reviewed <review_>`_ successful downloads of
|
||||
each of your subscriptions, you're ready to run a full download of all your
|
||||
subscriptions. Run the sub-command without the CLI options you used to limit what
|
||||
``ytdl-sub`` does while testing:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
ytdl-sub sub
|
||||
|
||||
If you're ready to let ``ytdl-sub`` run unattended, it's time to :doc:`automate
|
||||
downloads <./automating_downloads>`.
|
||||
|
|
@ -1,56 +0,0 @@
|
|||
Initial Download
|
||||
================
|
||||
|
||||
Once you've created :doc:`a subscriptions file <./subscriptions>`, you can perform your
|
||||
first download. Access ``ytdl-sub``, navigate to the directory containing your
|
||||
``subscriptions.yaml`` file.
|
||||
|
||||
|
||||
Dry Run
|
||||
-------
|
||||
|
||||
Performing a dry run is important when applying any change to your subscriptions to
|
||||
ensure output looks as expected. Dry runs will pull metadata to *simulate* a download
|
||||
without actually downloading the media file.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
ytdl-sub --dry-run sub subscriptions.yaml
|
||||
|
||||
|
||||
Faster Iteration Cycle
|
||||
----------------------
|
||||
|
||||
Testing subscriptions can take quite some time to perform a full download. This can be
|
||||
speed up by applying an override via command-line to set max number of downloads.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
ytdl-sub --dry-run sub subscriptions.yaml -o '--ytdl_options.max_downloads 3'
|
||||
|
||||
Having many subscriptions could still make this dry run take a while. A subset of
|
||||
subscriptions can be dry ran using a match.
|
||||
|
||||
.. code-block:: shell
|
||||
:caption: Only run subscriptions that have PBS in their names
|
||||
|
||||
ytdl-sub --dry-run sub subscriptions.yaml -o '--ytdl_options.max_downloads 3' --match PBS
|
||||
|
||||
|
||||
Downloading
|
||||
-----------
|
||||
|
||||
Once the subscriptions file is validated, a download can be performed by omitting the
|
||||
dry run argument.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
ytdl-sub sub subscriptions.yaml
|
||||
|
||||
Multiple subscription file names can be provided to perform a download on all of them. A
|
||||
single file named ``subscriptions.yaml`` does not require a file name specification
|
||||
since it will look for that file name by default, making the following command valid.
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
ytdl-sub sub
|
||||
|
|
@ -151,7 +151,7 @@ adding subscriptions <./subscriptions>`.
|
|||
:hidden:
|
||||
|
||||
subscriptions
|
||||
first_download
|
||||
downloading
|
||||
automating_downloads
|
||||
first_config
|
||||
quick_start
|
||||
|
|
|
|||
|
|
@ -37,31 +37,12 @@ instructions to the letter.
|
|||
``"https://www.youtube.com/@novapbs"`` value to the URL of the channel or playlist
|
||||
for this subscription.
|
||||
|
||||
#. Preview what ``ytdl-sub`` would do for this subscription:
|
||||
|
||||
Run the :ref:`'sub' sub-command <usage:subscriptions options>` but with the ``max_downloads``
|
||||
setting from ``yt-dlp`` along with the ``--dry-run`` and ``--match`` options from
|
||||
``ytdl-sub`` to minimize requests and prevent actual downloads. Be sure to update the
|
||||
``--match="..."`` value with the subscription name::
|
||||
|
||||
$ ytdl-sub --dry-run sub -o '--ytdl_options.max_downloads 3' --match="NOVA PBS"
|
||||
|
||||
Examine the output carefully.
|
||||
|
||||
#. Review the results of real downloads:
|
||||
|
||||
Run it again without the ``--dry-run`` option to actually download media and place
|
||||
the files in your library::
|
||||
|
||||
$ ytdl-sub sub -o '--ytdl_options.max_downloads 3' --match="NOVA PBS"
|
||||
|
||||
Examine the output carefully, then examine how the downloads work in your
|
||||
library. Repeat with a larger value for ``max_downloads`` and examine the output and
|
||||
downloads again.
|
||||
#. :ref:`Preview <guides/getting_started/downloading:preview>` and :ref:`Review
|
||||
<guides/getting_started/downloading:review>` the subscription.
|
||||
|
||||
#. Add the rest of your subscriptions:
|
||||
|
||||
Repeat steps #4-7 for each of your subscriptions. Be sure to repeat the preview and
|
||||
Repeat steps #3-6 for each of your subscriptions. Be sure to repeat the preview and
|
||||
review steps for each subscription. In general, move slowly and carefully review
|
||||
everything. It's best to catch issues early :ref:`to avoid repeating downloads and to
|
||||
minimize requests <guides/getting_started/index:minimize the work to only what's
|
||||
|
|
|
|||
|
|
@ -163,5 +163,5 @@ specific to only one subscription and will never be shared with another:
|
|||
Next Steps
|
||||
----------
|
||||
|
||||
Once you've defined your subscriptions, it's time to :doc:`test the configuration and
|
||||
try your first download <./first_download>`.
|
||||
Once you've defined your subscriptions, it's time to :doc:`test your configuration and
|
||||
try your first download <./downloading>`.
|
||||
|
|
|
|||
Loading…
Reference in a new issue