fix(docker): Add default crontab download command

Clarify the `/config/ytdl-sub-configs/cron` script with explanatory comments and a
default `--dry-run` command. Also change the wrapper script to echo commands for easier
debugging. To update an existing script, move the old script aside, restart the
container to regenerate it, and edit the new script.

Also clarify the Automating page in the Getting Started guide docs.
This commit is contained in:
Ross Patterson 2025-08-29 15:45:29 -07:00
parent 281988a370
commit c1541ab904
No known key found for this signature in database
GPG key ID: 2EFF7CCE6828E359
7 changed files with 138 additions and 80 deletions

View file

@ -31,6 +31,8 @@ if [ "$CRON_SCHEDULE" != "" ] ; then
# create cron script wrapper # create cron script wrapper
echo '#!/bin/bash' > "$CRON_WRAPPER_SCRIPT" echo '#!/bin/bash' > "$CRON_WRAPPER_SCRIPT"
# Echo commands for easier user debugging:
echo "set -x" >> "$CRON_WRAPPER_SCRIPT"
echo "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" >> "$CRON_WRAPPER_SCRIPT" echo "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" >> "$CRON_WRAPPER_SCRIPT"
echo "cd \"$DEFAULT_WORKSPACE\"" >> "$CRON_WRAPPER_SCRIPT" echo "cd \"$DEFAULT_WORKSPACE\"" >> "$CRON_WRAPPER_SCRIPT"
echo ". \"$CRON_SCRIPT\" | tee -a \"$LOGS_TO_STDOUT\"" >> "$CRON_WRAPPER_SCRIPT" echo ". \"$CRON_SCRIPT\" | tee -a \"$LOGS_TO_STDOUT\"" >> "$CRON_WRAPPER_SCRIPT"

View file

@ -1,4 +1,12 @@
echo "Beginning cron job..."
# Place your ytdl-sub command(s) here. # Place your ytdl-sub command(s) here.
# This script is executed in the same relative path as this file. #
# This script is executed in the same directory as this file which also contains the
# default `./config.yaml` and `./subscriptions.yaml`, so you don't need to use the
# `--config` CLI option or pass a `SUBPATH` to the `$ ytdl-sub sub` sub-command.
#
# To prevent users accidentally triggering throttles or bans or downloading before
# testing their configuration, these default options only simulate a few
# downloads. Remove the `--dry-run` and `-o ...` CLI options when you've tested your
# configuration and you're ready to download entries unattended:
ytdl-sub --dry-run sub -o '--ytdl_options.max_downloads 3'

View file

@ -41,8 +41,8 @@ download your YouTube cookie, then add it to your :ref:`ytdl options
...automate my downloads? ...automate my downloads?
~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~
:doc:`This page </guides/getting_started/automating_downloads>` shows how to set up :doc:`This page </guides/getting_started/automating>` shows how to set up ``ytdl-sub``
``ytdl-sub`` to run automatically on various platforms. to run automatically on various platforms.
...download large channels? ...download large channels?
~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~

View file

@ -0,0 +1,117 @@
Automating
==========
Automate downloading your subscriptions by running the :ref:`'sub' sub-command
<usage:subscriptions options>` periodically. There are various tools that can run
commands on a schedule you may use any of them that work with your installation
method. Most users use `cron`_ in `Docker containers <docker and unraid_>`_.
Docker and Unraid
-----------------
:doc:`The 'ytdl-sub' Docker container images <../install/docker>` provide optional cron
support. Enable cron support by setting `a cron schedule`_ in the ``CRON_SCHEDULE``
environment variable:
.. code-block:: yaml
:caption: ./compose.yaml
:emphasize-lines: 4
services:
ytdl-sub:
environment:
CRON_SCHEDULE: "0 */6 * * *"
Then recreate the container to apply the change and start it to generate the default
``/config/ytdl-sub-configs/cron`` script. Read the comments in that script and edit as
appropriate.
The container cron wrapper script will write output from the cron job to
``/config/ytdl-sub-configs/.cron.log``. The default image ``ENTRYPOINT`` will ``$ tail
...`` that file so you can monitor the cron job in the container's output and thus also
in the Docker logs.
.. _linux-setup:
Linux, Mac OS X, BSD, or other UNIX's
-------------------------------------
For installations on systems already running ``# crond``, you can also use cron to run
``ytdl-sub`` periodically. Write a script to run ``ytdl-sub`` in the cron job. Be sure
the script changes to the same directory as your configuration and uses the full path to
``ytdl-sub``:
.. code-block:: shell
:caption: ~/.local/bin/ytdl-sub-cron
:emphasize-lines: 2,3
#!/bin/bash
cd "~/.config/ytdl-sub/"
~/.local/bin/ytdl-sub --dry-run sub -o '--ytdl_options.max_downloads 3' |&
tee -a "~/.local/state/ytdl-sub/.cron.log"
Then tell ``# crond`` when to run the script:
.. code-block:: console
echo "0 */6 * * * ${HOME}/.local/bin/ytdl-sub-cron" | crontab "-"
Remove the ``--dry-run`` and ``-o ...`` CLI options from your cron script when you've
tested your configuration and you're ready to download entries unattended.
.. _windows-setup:
Windows
-------
For most Windows users, the best way to run commands periodically is `the Task
Scheduler`_:
.. attention::
These instructions are untested. Use at your own risk. If you use them, whether they
work or not, please let us know how it went in `a support post in Discord`_ or `a new
GitHub issue`_.
#. Open the Task Scheduler app.
#. Click ``Create Basic Task`` at the top of the right sidebar.
#. Set all the fields as appropriate until you get to the ``Action``...
#. For the ``Action``, select ``Start a program``...
#. Click ``Browse...`` to the installed ``ytdl-sub.exe`` executable...
#. Add CLI arguments to ``Add arguments (optional):``, for example ``--dry-run sub -o
'--ytdl_options.max_downloads 3'``...
#. Set ``Start in (optional):`` to the directory containing your configuration.
#. Finish the rest of the ``Create Basic Task`` wizard.
Next Steps
----------
At this point, ``ytdl-sub`` should run periodically and keep your subscriptions current
in your media library without your intervention. As your :doc:`subscriptions file
<./subscriptions>` grows or you discover new use cases, it becomes worth while to
simplify things by :doc:`defining your own custom presets <./first_config>`.
.. _`cron`:
https://en.wikipedia.org/wiki/Cron
.. _`a cron schedule`:
https://en.wikipedia.org/wiki/Cron#Overview
.. _`the Task Scheduler`:
https://learn.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page
.. _`a support post in Discord`:
https://discord.com/channels/994270357957648404/1084886228266127460
.. _`a new GitHub issue`:
https://github.com/jmbannon/ytdl-sub/issues/new

View file

@ -1,69 +0,0 @@
Automating Downloads
====================
:ref:`Guide for Docker and Unraid Containers <guides/getting_started/automating_downloads:docker and unraid>`
:ref:`Guide for Linux <guides/getting_started/automating_downloads:linux>`
:ref:`Guide for Windows <guides/getting_started/automating_downloads:windows>`
.. _cron scheduling syntax: https://crontab.guru/#0_*/6_*_*_*
.. _docker-unraid-setup:
Docker and Unraid
-----------------
Cron is preconfigured in every ytdl-sub docker container. Enable by adding the following
ENV variables to your docker setup.
.. code-block:: yaml
services:
ytdl-sub:
environment:
- CRON_SCHEDULE="0 */6 * * *"
- CRON_RUN_ON_START=false
- ``CRON_SCHEDULE`` follows the standard `cron scheduling syntax`_. The above value will
run the script once every 6 hours.
- ``CRON_RUN_ON_START`` toggles whether to run your cron script on container start in
addition to the cron schedule.
The cron script will reside in the main directory with the file name ``cron``. Cron
logs should show when viewing the Docker logs.
.. _linux-setup:
Linux
-----
Must configure crontab manually, like so:
.. code-block:: shell
crontab -e
0 */6 * * * /config/run_cron
.. _windows-setup:
Windows
-------
To be tested (please contact code owner or join the discord server if you can test this
out for us)
.. code-block:: powershell
ytdl-sub.exe --config \path\to\config\config.yaml sub \path\to\config\subscriptions.yaml
Next Steps
----------
Once you have a significant quantity of subscriptions or have use cases not served using
:doc:`YAML keys and the special characters <./subscriptions>`, it's time to start
:doc:`defining your own custom presets <./first_config>`.

View file

@ -50,5 +50,5 @@ instructions to the letter.
#. Automate downloads: #. Automate downloads:
:ref:`Set up ytdl-sub to run periodically :ref:`Set up ytdl-sub to run periodically <guides/getting_started/automating:docker
<guides/getting_started/automating_downloads:docker and unraid>`. and unraid>`.

View file

@ -7,8 +7,8 @@ on top. There are two flavors or variants to choose from. For a more user-friend
experience editing the `configuration`_, we recommend the `GUI image`_ experience editing the `configuration`_, we recommend the `GUI image`_
variant. :ref:`Docker Compose <guides/install/docker:install with docker compose>` is variant. :ref:`Docker Compose <guides/install/docker:install with docker compose>` is
the recommended way of managing a ``ytdl-sub`` docker container. See :ref:`Automating the recommended way of managing a ``ytdl-sub`` docker container. See :ref:`Automating
Downloads <guides/getting_started/automating_downloads:docker and unraid>` for how to Downloads <guides/getting_started/automating:docker and unraid>` for how to automate
automate running ``ytdl-sub`` in a container running either variant. running ``ytdl-sub`` in a container running either variant.
GUI Image GUI Image
@ -28,8 +28,8 @@ Headless Image
The headless image is based on LSIO's :lsio-gh:`docker-baseimage-alpine`. Once running, The headless image is based on LSIO's :lsio-gh:`docker-baseimage-alpine`. Once running,
the default command just starts services including cron for :ref:`Automating Downloads the default command just starts services including cron for :ref:`Automating Downloads
<guides/getting_started/automating_downloads:docker and unraid>` but otherwise doesn't <guides/getting_started/automating:docker and unraid>` but otherwise doesn't run
run ``ytdl-sub``. You may run arbitrary ``ytdl-sub`` commands using the ``ytdl-sub``. You may run arbitrary ``ytdl-sub`` commands using the
``--rm --user="${PUID}:${PGID}" --entrypoint="ytdl-sub"`` options to either ``$ docker ``--rm --user="${PUID}:${PGID}" --entrypoint="ytdl-sub"`` options to either ``$ docker
run`` or ``$ docker compose run``. Overriding the image's ``ENTRYPOINT`` is important so run`` or ``$ docker compose run``. Overriding the image's ``ENTRYPOINT`` is important so
that cron doesn't run ``ytdl-sub`` while you're running it manually. that cron doesn't run ``ytdl-sub`` while you're running it manually.