Subscription file syntax is designed to minimize boiler-plate when authoring new subscriptions.
You can now unpack any subscription using the ``inspect`` sub-command to see its boiler-plate *preset format*.
```
ytdl-sub inspect --config /path/to/config.yaml --match "BBC News" /path/to/subscriptions.yaml
```
This can be utilized for numerous purposes including:
* Ensuring your custom preset is getting applied correctly.
* Figuring out which variables set things like file names, metadata, etc.
* Understanding how subscription syntax translates to preset representation.
The default ``--level`` of inspect will fill in defined variables. Using ``--level original`` will present the subscription's raw layout with no fill.
* docs(docker): Recommended image DEFAULT_WORKSPACE
* fix(docker): Add default crontab download command
Clarify the `/config/ytdl-sub-configs/cron` script with explanatory comments and a
default `--dry-run` command. Also change the wrapper script to echo commands for easier
debugging. To update an existing script, move the old script aside, restart the
container to regenerate it, and edit the new script.
Also clarify the Automating page in the Getting Started guide docs.
* fix(docker): Unintentional unattended dry runs
[PR feedback](https://github.com/jmbannon/ytdl-sub/pull/1321#discussion_r2315215600)
prompted me to reconsider having a default command at all. We should assume,
unfortunately, that many new users will just skim the docs enough to enable the image's
cron integration but not actually incrementally test their configuration. In those
cases, they'd end up sending dry-run non-download requests for all their subscriptions
every 6 hours for no good reason. There's just no way to provide a default command that
isn't also providing a footgun.
* docs(docker): More open cron schedule generator
From [PR
feedback](https://github.com/jmbannon/ytdl-sub/pull/1321#discussion_r2320521024), this
seems like less of an ad than the previous and the source for the page is itself open source.
* docs(docker): Document image environment footgun
From [PR
feedback](https://github.com/jmbannon/ytdl-sub/pull/1321#discussion_r2320515419).
* docs(automate): Avoid external link 404 responses
* docs(automate): False simultaneous run warning
* docs(automate): Clarify Docker daemon restarts
* docs(automate): Revert run start non-recommended
* docs(automate): Remove footgun manual run command
Addressing this underlying issue requires more thought and should be a separate PR.
* docs(automate): Restore env var footgun in example
Closes https://github.com/jmbannon/ytdl-sub/issues/1154
For docker images, add the ability to set the variable `UPDATE_YT_DLP_ON_START`, which will update yt-dlp to the specified version if a new one exists.
Supports ``stable``, ``nightly``, ``master``.
Changes documentation and default configs to use the `/tv_shows` default path shown in the docker compose setup.
Also changes working directory and file lock directory to not be in /tmp, which often causes permission issues.
* docs(presets): Stale copied prebuilt comment
* docs(start): How to configure subscriptions
Revise the subscriptions file Getting Started tutorial from the perspective of a new
user. The idea is that the ref docs describe a component as the code sees it, whereas
the how-to/tutorial/getting-started docs provide a narrative description of the
component from the external perspective of a typical new user.
* docs(start): Document subscription override mode
It's debatable whether this belongs here. Having moved onto my own custom presets, the
only use cases for override mode in my subscriptions file are for those series that also
have TVDB metadata. Those ytdl-sub downloads I integrate into my Sonarr library need
per-subscription overrides such as:
- Sonarr manages years 2020-2023 for one series and ytdl-sub fills in
from there, so set a `sonarr_series_after: "20240101"` override
- each series needs a directory named per Sonarr's configuration with the TVDB ID, the subscription key,
but also a series name prefix without the TVDB ID for each episode file, so set a
`sonarr_series_prefix: "Foo Series (2020)"` override
So are there other use cases for override mode that are more common and less niche than
mine?
* docs(docker): Revert example bind mount volumes
Per [PR feedback](https://github.com/jmbannon/ytdl-sub/pull/1310#discussion_r2306067263).
Adds better support to run commands via `docker exec ...` by setting the working directory to be ytdl-sub's working directory.
Thanks @rpatterson for the contribution 😊
Reconcile the various debugging hints throughout the documentation and integrate into a
central debugging page. Includes clarification of the roles of ytdl-sub, yt-dlp, and
external services. Also adds cross references to the relevant bits across the docs.
This is another change that touches the generated docs, so after finishing my writing, I
copied the change to `./docs/source/config_reference/scripting/static_variables.rst`
into the corresponding docstring in
`./src/ytdl_sub/entries/variables/override_variables.py` and formatted it as a
docstring. Then I ran:
$ REGENERATE_DOCS=1 tox exec -e "py" -- pytest tests/unit/docgen/test_docgen.py
But all that did was revert the changes to
`./docs/source/config_reference/scripting/static_variables.rst`. IOW, it did *not*
update the rST from the changed docstring.
* docs(preset): Config file top presets vs base
* docs(throttle): resolution/quality throttle asset
I also added the "wait a few hours and try again" hint, sorry I didn't think of this
when reviewing your PR. While doing that I also reformatted the `%concat(...)` arguments
to be more readable to me. Particularly to keep clear what punctuation is about argument
separation as opposed to punctuation meant to be included in the resulting string. Not a
strong preference if you don't like it.
* docs(various): Minor typo and formatting fixes
* docs(preset): Fix presets vs preset in config file
From [feedback](https://github.com/jmbannon/ytdl-sub/pull/1289#discussion_r2289960678).
* test(throttle): Match modified assertion text
If it were my test, I'd only make assertions on the spirit of error message text
necessary to confirm it's the right error message and not another.
* docs(throttle): More likely override value
From [feedback](https://github.com/jmbannon/ytdl-sub/pull/1289#discussion_r2289976156).
I've been running into variations in style in the docs I've edited so far. I've been
including changes for consistency in with other commits but in many cases that makes one
part of one doc more consistent with the rest but less consistent with itself. It seems
like a bit of cleanup may be in order.
This change applies the following conventions:
- Section heading heirarchy:
#. ``=`` with overline for document title, IOW first heading
#. ``-``
#. ``~``
#. ``"``
- Two newlines before ``-`` sections for readability
- A newline between every section of any level and the first line of text
- Wrap paragraph lines at 88 characters to match Python's Black
I've only applied these changes to those ``*.rst`` files that aren't generated.
In the future, I might suggest another bulk change to match [the Sphinx conventions for
section
headings](https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html#sections).
* docs(config): Clarify preset ordering priority
Refs #1276
* docs(architecture): WIP Narrative how it works
Much of this is repetition of the reference docs and some of the changes here should
also be applied here. The goal is to repeat here the parts of the reference docs that
may hang up a user when reading "Getting Started" without sending them off to get lost
in the weeds of the reference docs.
Note that this references a `Quick Start` document which is just an empty placeholder
and is to be written in a subsequent change.
Fixes#1279
* docs(terms): Glossary redundant with architecture
Now that the introduction includes an architectural overview, that section introduces
new users to necessary concepts and associated terminology and makes the terminology
section redundant in that part of the docs.
* docs(quick): Add a rote quick start guide
This is what I came up with when I tried to write instructions requiring as little
understanding as possible. Doing so really did reinforce my impression that this just
shouldn't be done, that significant understanding is required for *any* use of ytdl-sub
and even offering a quick start may be irresponsible. I'm still on the fence, thoughts?
That said, I also think I got some good explanations out of this and a better
understanding of what the next bits of the Getting Started should be. This Quick Start
is currently redundant with other parts of the Getting Started pages. I suspect that
I'll end up repeating and/or reorganizing parts of this Quick Start into the next bits
of the Getting Started. To that end I might argue that this change is a WIP and that
merging should wait. But I could also argue that this is an incremental improvement and
can be merged before that other work. Your call.
* docs(start): Various rST/Sphinx markup issues
* docs(install): Redundant platform-specific mention
I'm guessing this is a remnant from before it got it's own page.
* docs(docker): Clarify container image install
Capturing my changes thus far while I go off to educate myself about rST/Sphinx refs vs
links. As such, this duplicates a reference link in a way I don't like:
:ref:`Automating Downloads <guides/getting_started/automating_downloads:docker and
unraid>`
If you want to fix that or are OK with the duplication, this should be ready to
go. Otherwise, I'll fix and force push once I've learned how to do this correctly.
A matter of opinion in this change, I did away with the tabbed GUI vs headless code
blocks and the separate CPU/GPU passthrough code blocks because I find comments in
example code to be more clear, more readable, and more approachable. This also has the
benefit of putting informative comments in the resulting user's configuration. For
example, imagine a user that learns only later that they require GPU passthrough, their
configuration is already ready to get them started. This comes at the cost some repeated
comments (`environment:` and `deploy:` for GPU passthrough) and the fancy Sphinx
highlighting of relevant lines. I think the trade-off is a net benefit for users.
* docs(sphinx): Fix extlinks LSIO reference usage
* docs(sphinx): Address inline literal warning
Addresses:
./docs/source/config_reference/plugins.rst:848: WARNING: Inline literal start-string
without end-string. [docutils]
* docs(start): Clarify prerequisite tech knowledge
* docs(docker): Clarify where the configuration is
The docs move from installation with Docker straight to modifying the configuration
without detailing where those files are.
* docs(start): Defining comments is discracting
This one is squarely me with my editor's red pen, do what you will. I whole heartedly
agree with this statement about comments and the intention (I assume) to encourage new
users to comment their configurations early and often. I just think this isn't the right
place for it and makes this document less focused and effective.
* docs(start): Less technical YAML key description
* docs(intro): Update and clarify goals/motivation
To help calibrate our collaboration, I went uninhibited with these changes including:
- assumptions about history I don't actually know:
"this project was one of the early entrants"
- description of the yt-dlp/ytdl-sub division of labor I'm not actually certain about
- max opinion, both about this project and the domain in general
- max cheeky/fun writing tone, with the intention of being personable
- max editor's red pen, cut mercilessly
IOW, review carefully and reject early and often to help me submit future changes that
require less review and revision. To that end, give me more rather than less feedback on
this change in particular.
I include an endorsement of Pinchflat because [they do the same for
ytdl-sub](https://github.com/kieraneglin/pinchflat?tab=readme-ov-file#what-it-does). It
would be my next choice if ytdl-sub couldn't meet my needs or vanquished me. More
than that link, in other Reddit comments and the like, the Pinchflat author clearly
expresses preference for ytdl-sub as their own "next option".
- Update Ubuntu-based images to use Python 3.12
- Move hidden files from GUI image's main dir
- Update github workflows to use python 3.12
- Update fixtures
- Minor doc fixes
Drastically simplifies setting up cron within ytdl-sub. Below is the documentation which is available in readthedocs, and a section on how to migrate from the old way to this.
## New Cron Documentation
Cron is preconfigured in every ytdl-sub docker container. Enable by adding the following ENV variables to your docker setup.
```
services:
ytdl-sub:
environment:
- CRON_SCHEDULE="0 */6 * * *"
- CRON_RUN_ON_START=false
```
- CRON_SCHEDULE follows the standard [cron scheduling syntax](https://crontab.guru/#0_*/6_*_*_*). The above value will run the script once every 6 hours.
- CRON_RUN_ON_START toggles whether to run your cron script on container start.
The cron script will reside in the main directory with the file name `cron`. Cron logs should show when viewing the Docker logs.
## Migrating Existing Cron Setup
Containers' cron files will remain intact until you add the `CRON_SCHEDULE` variable. Once you add that with your preferred schedule, it will write that value to /config/crontabs/abc and create a new file `cron` within your working directory. Simply copy everything from your existing `run_cron` script into `cron`.
If you have the ENV variable `DOCKER_MODS` in your docker-compose, remove it. It is now included by default. Not removing this will prevent cron logs from showing in your docker container logs.
Readthedocs for ytdl-sub is getting a massive overhaul to both look and read like a modernized app. It is still very-much work-in-progress, stay tuned for more!
Huge thanks to @Svagtlys (aka Momo) for driving this
