docs(faq): Misunderstood re-download CLI options

https://discord.com/channels/994270357957648404/1102703969266049174/1407956781316964372
This commit is contained in:
Ross Patterson 2025-08-21 00:07:22 -07:00
parent 9939cf9bad
commit d801d0c5fc
No known key found for this signature in database
GPG key ID: 2EFF7CCE6828E359
3 changed files with 35 additions and 16 deletions

View file

@ -182,27 +182,27 @@ need to be removed:
the downloaded files in this file and delete that whole entry, from the YouTube ID
string to the closing curly braces. Be ware of JSON traling commas.
- Run ``$ ytdl-sub sub`` again with the appropriate CLI overrides:
- Run ``$ ytdl-sub sub`` again with the appropriate CLI plugin options:
Depending on the presets and overrides of the subscriptions whose downloaded files
you've renamed in the steps above, you need to pass some overrides from the command
line to tell ``ytdl-sub`` to change its behavior and re-download those files.
In normal operation, :ref:`yt-dlp minimizes requests and the files considered for
download <guides/getting_started/index:minimize the work to only what's
necessary>`. To re-download, those options must be disabled or modified. Disable
:ref:`the 'break_on_existing' option <config_reference/plugins:ytdl_options>`, set
:ref:`the 'date_range:' plugin <config_reference/plugins:date_range>`, and :ref:`limit
the subscriptions <guides/getting_started/first_download:faster iteration cycle>` to
download only the files that you've renamed in the steps above.
Most presets tell ``yt-dlp`` not to bother even looking at videos before the most
recently downloaded video :ref:`using 'break_on_existing'
<config_reference/plugins:ytdl_options>`. Other presets, such as :ref:`the Only Recent
presets <prebuilt_presets/helpers:only recent>`, restrict the videos considered for
download by date range.
Determine the necessary CLI overrides and re-run, for example:
Set the appropriate dates and subscription name to include only the files you've
renamed, and re-run. For example, if you've renamed all the files from 2024 in the
``NOVA PBS`` subscription:
.. code-block:: shell
ytdl-sub sub -o "\
--ytdl_options.break_on_existing False \
--date_range.after 19700101 \
--date_range.before 20380119 \
"
--date_range.after 20240101 \
--date_range.before 20250101 \
" --match="NOVA PBS"
There is a bug where...

View file

@ -104,6 +104,24 @@ Users define additional presets in :doc:`their configuration file <./first_confi
they then use in most of their subscriptions. Most user-defines presets extend the
:doc:`../../prebuilt_presets/index` provided by ``ytdl-sub``.
Minimize the work to only what's necessary
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Throttling and bans are a core problem for any web scraping tool, perhaps even more so
for ``yt-dlp``, and no good actor *wants* to be an onerous burden on a
service. Similarly, many web scraping use cases involve very large sets of data that are
too big to process as a whole for performance. It's important to narrow the amount of
data considered and minimize requests.
To these ends, most presets tell ``yt-dlp`` not to consider files before the most
recently downloaded file using :ref:`the 'break_on_existing' option
<config_reference/plugins:ytdl_options>`. Similarly, and particularly for huge channels
or playlists, most users should use either :ref:`an 'Only Recent' preset
<prebuilt_presets/helpers:only recent>` and/or :ref:`the 'Chunk Downloads' preset
<prebuilt_presets/helpers:chunk downloads>` to restrict the number of downloads
considered.
Caveats
~~~~~~~

View file

@ -63,8 +63,9 @@ instructions to the letter.
Repeat steps #4-7 for each of your subscriptions. Be sure to repeat the preview and
review steps for each subscription. In general, move slowly and carefully review
everything. It's best to catch issues early to avoid repeating downloads and to
minimize requests to avoid being throttled or banned by servers.
everything. It's best to catch issues early :ref:`to avoid repeating downloads and to
minimize requests <guides/getting_started/index:minimize the work to only what's
necessary>`.
#. Automate downloads: