Completely rewrite how subscription validation is performed. Optimizes it quite a bit while also adding backend support for the upcoming `dissect` sub-command, where you can resolve any subscription into its 'raw' form for easier debugging when making scripting changes.
Disallow override variable names conflicting with plugin names.
May cause backward incompatible issues from `date_range` being an old override variable name for the `Only Recent` preset. Update this variable name to `only_recent_date_range` to resolve. More info in https://ytdl-sub.readthedocs.io/en/latest/deprecation_notices.html
Closes https://github.com/jmbannon/ytdl-sub/issues/386
Adds the ability to preserve mtime via Output Options. Usage:
```
output_options:
preserve_mtime: True
```
Thanks @e1ven for the contribution!
Fixes `UPDATE_YT_DLP_ON_START` when set to 'nightly' as the pip command is missing --break-system-packages. Simplifies dockerfiles by setting this globally via pip config file.
Thanks @Snuffy2 for the contribution!
Closes https://github.com/jmbannon/ytdl-sub/issues/1269
Adds the new paramter `languages_required`, which forces an entry to have this set of subtitle files, otherwise it will error.
In addition, better file checking has been added to ensure they exist before attempting to move.
Closes https://github.com/jmbannon/ytdl-sub/issues/1154
For docker images, add the ability to set the variable `UPDATE_YT_DLP_ON_START`, which will update yt-dlp to the specified version if a new one exists.
Supports ``stable``, ``nightly``, ``master``.
Closes https://github.com/jmbannon/ytdl-sub/issues/1313
Adds `resolution_assert_ignore_titles` variable to be able to ignore specific videos in a subscription where 360p is suspected to be the only option.
Ideally, it would be nice if ytdl-sub could determine whether 360p is actually the only resolution or not automatically, and download if it's true. We need to do more research to determine if this is possible.
In the meantime, the above variable should help manually get around this. Usage:
```
# use tilda mode to set override variables to the subscription
"~My Subscription":
url: "https://youtube.com/@channel"
resolution_assert_ignore_titles:
- "This 360p Video Title"
```
Changes documentation and default configs to use the `/tv_shows` default path shown in the docker compose setup.
Also changes working directory and file lock directory to not be in /tmp, which often causes permission issues.
```
Resolution assert is enabled, will fail on low-quality video downloads and presume throttle. Disable using the override variable `enable_resolution_assert: False
```
Spams during the initialization phase. Ideally, we should only print this once the subscription has begun downloading. Perhaps via a `print` plugin? Either way, set this to debug log level.
* ci(docs): RTD Sphinx warnings false successes
Use the same options on readthedocs.com that we use locally to fail on all warnings,
mostly to better catch stale cross references.
* docs(usage): Yet another stale Sphinx cross ref
I confirmed the previous RTD CI fix in the previous commit by pushing it to a draft PR
before pushing this change and the RTD action failed with the warning this commit
fixes. After pushing this commit, the RTD action succeeded.
* build(docs): Sphinx stale cross refs false success
I figured out why I kept getting warnings for broken Sphinx cross-refs *after* the
changes that caused them have already been merged, changes I know I ran `$ make docs`
for before pushing. The issue is that by default Sphinx only builds changed files for
faster iterations while editing, but it only catches broken cross-refs when it builds
files. So if changing, for example, a section name in one page that is referenced from
another page that you do *not* change, then the warning will be missed until something
changes that other page, such as a pull or rebase.
I considered adding a separate `./Makefile` target for incremental builds
in the inner loop of making changes. But I opted to just remove the `--write-all` CLI
option locally while editing in the inner loop, because the uncommitted change will
remind me to revert it and run a full rebuild before pushing.
Adds support for setting throttle_protection range values using static override variables. `sleep_per_download_s` has additional support for entry variables since it's used per entry.
We can now experiment with scaling this value based on entry attributes, such as duration. Example:
```
throttle_protection:
sleep_per_download_s:
min: >-
{
%mul(5.5, %pow( duration, 0.4 ))
}
max: >-
{
%mul(6.5, %pow( duration, 0.6 ))
}
```
* docs(presets): Stale copied prebuilt comment
* docs(start): How to configure subscriptions
Revise the subscriptions file Getting Started tutorial from the perspective of a new
user. The idea is that the ref docs describe a component as the code sees it, whereas
the how-to/tutorial/getting-started docs provide a narrative description of the
component from the external perspective of a typical new user.
* docs(start): Document subscription override mode
It's debatable whether this belongs here. Having moved onto my own custom presets, the
only use cases for override mode in my subscriptions file are for those series that also
have TVDB metadata. Those ytdl-sub downloads I integrate into my Sonarr library need
per-subscription overrides such as:
- Sonarr manages years 2020-2023 for one series and ytdl-sub fills in
from there, so set a `sonarr_series_after: "20240101"` override
- each series needs a directory named per Sonarr's configuration with the TVDB ID, the subscription key,
but also a series name prefix without the TVDB ID for each episode file, so set a
`sonarr_series_prefix: "Foo Series (2020)"` override
So are there other use cases for override mode that are more common and less niche than
mine?
* docs(docker): Revert example bind mount volumes
Per [PR feedback](https://github.com/jmbannon/ytdl-sub/pull/1310#discussion_r2306067263).
Adds better support to run commands via `docker exec ...` by setting the working directory to be ytdl-sub's working directory.
Thanks @rpatterson for the contribution 😊
Reconcile the various debugging hints throughout the documentation and integrate into a
central debugging page. Includes clarification of the roles of ytdl-sub, yt-dlp, and
external services. Also adds cross references to the relevant bits across the docs.
* docs(logs): Mimick argparse option default format
I find it more readable to use punctuation to separate technical information, such as
required vs optional or default values, from narrative description. Follow argparse's
lead, and put such information in parens following the narrative description.
* docs(usage): Consistent structure, clarifications
* docs(config): Clarify persist_logs behavior/opts
Clarify the `persist_logs:` options per [Discord discussion](https://discord.com/channels/994270357957648404/1409161361853780060/1409602529460879431).
This is another change that touches the generated docs, so after finishing my writing, I
copied the change to `./docs/source/config_reference/scripting/static_variables.rst`
into the corresponding docstring in
`./src/ytdl_sub/entries/variables/override_variables.py` and formatted it as a
docstring. Then I ran:
$ REGENERATE_DOCS=1 tox exec -e "py" -- pytest tests/unit/docgen/test_docgen.py
But all that did was revert the changes to
`./docs/source/config_reference/scripting/static_variables.rst`. IOW, it did *not*
update the rST from the changed docstring.
