From c1541ab904e1a4b5983bcc093ad073181ec7d036 Mon Sep 17 00:00:00 2001 From: Ross Patterson Date: Fri, 29 Aug 2025 15:45:29 -0700 Subject: [PATCH] fix(docker): Add default crontab download command Clarify the `/config/ytdl-sub-configs/cron` script with explanatory comments and a default `--dry-run` command. Also change the wrapper script to echo commands for easier debugging. To update an existing script, move the old script aside, restart the container to regenerate it, and edit the new script. Also clarify the Automating page in the Getting Started guide docs. --- docker/root/custom-cont-init.d/defaults | 2 + docker/root/defaults/cron | 14 ++- docs/source/faq/index.rst | 4 +- .../guides/getting_started/automating.rst | 117 ++++++++++++++++++ .../getting_started/automating_downloads.rst | 69 ----------- .../guides/getting_started/quick_start.rst | 4 +- docs/source/guides/install/docker.rst | 8 +- 7 files changed, 138 insertions(+), 80 deletions(-) create mode 100644 docs/source/guides/getting_started/automating.rst delete mode 100644 docs/source/guides/getting_started/automating_downloads.rst diff --git a/docker/root/custom-cont-init.d/defaults b/docker/root/custom-cont-init.d/defaults index cfbc0776..d8d94994 100644 --- a/docker/root/custom-cont-init.d/defaults +++ b/docker/root/custom-cont-init.d/defaults @@ -31,6 +31,8 @@ if [ "$CRON_SCHEDULE" != "" ] ; then # create cron script wrapper echo '#!/bin/bash' > "$CRON_WRAPPER_SCRIPT" + # Echo commands for easier user debugging: + echo "set -x" >> "$CRON_WRAPPER_SCRIPT" echo "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" >> "$CRON_WRAPPER_SCRIPT" echo "cd \"$DEFAULT_WORKSPACE\"" >> "$CRON_WRAPPER_SCRIPT" echo ". \"$CRON_SCRIPT\" | tee -a \"$LOGS_TO_STDOUT\"" >> "$CRON_WRAPPER_SCRIPT" diff --git a/docker/root/defaults/cron b/docker/root/defaults/cron index 6fdbea71..358a981f 100644 --- a/docker/root/defaults/cron +++ b/docker/root/defaults/cron @@ -1,4 +1,12 @@ -echo "Beginning cron job..." - # Place your ytdl-sub command(s) here. -# This script is executed in the same relative path as this file. \ No newline at end of file +# +# This script is executed in the same directory as this file which also contains the +# default `./config.yaml` and `./subscriptions.yaml`, so you don't need to use the +# `--config` CLI option or pass a `SUBPATH` to the `$ ytdl-sub sub` sub-command. +# +# To prevent users accidentally triggering throttles or bans or downloading before +# testing their configuration, these default options only simulate a few +# downloads. Remove the `--dry-run` and `-o ...` CLI options when you've tested your +# configuration and you're ready to download entries unattended: + +ytdl-sub --dry-run sub -o '--ytdl_options.max_downloads 3' diff --git a/docs/source/faq/index.rst b/docs/source/faq/index.rst index bb5f4fe0..8b85f21a 100644 --- a/docs/source/faq/index.rst +++ b/docs/source/faq/index.rst @@ -41,8 +41,8 @@ download your YouTube cookie, then add it to your :ref:`ytdl options ...automate my downloads? ~~~~~~~~~~~~~~~~~~~~~~~~~ -:doc:`This page ` shows how to set up -``ytdl-sub`` to run automatically on various platforms. +:doc:`This page ` shows how to set up ``ytdl-sub`` +to run automatically on various platforms. ...download large channels? ~~~~~~~~~~~~~~~~~~~~~~~~~~~ diff --git a/docs/source/guides/getting_started/automating.rst b/docs/source/guides/getting_started/automating.rst new file mode 100644 index 00000000..7c74945f --- /dev/null +++ b/docs/source/guides/getting_started/automating.rst @@ -0,0 +1,117 @@ +Automating +========== + +Automate downloading your subscriptions by running the :ref:`'sub' sub-command +` periodically. There are various tools that can run +commands on a schedule you may use any of them that work with your installation +method. Most users use `cron`_ in `Docker containers `_. + + +Docker and Unraid +----------------- + +:doc:`The 'ytdl-sub' Docker container images <../install/docker>` provide optional cron +support. Enable cron support by setting `a cron schedule`_ in the ``CRON_SCHEDULE`` +environment variable: + +.. code-block:: yaml + :caption: ./compose.yaml + :emphasize-lines: 4 + + services: + ytdl-sub: + environment: + CRON_SCHEDULE: "0 */6 * * *" + +Then recreate the container to apply the change and start it to generate the default +``/config/ytdl-sub-configs/cron`` script. Read the comments in that script and edit as +appropriate. + +The container cron wrapper script will write output from the cron job to +``/config/ytdl-sub-configs/.cron.log``. The default image ``ENTRYPOINT`` will ``$ tail +...`` that file so you can monitor the cron job in the container's output and thus also +in the Docker logs. + + +.. _linux-setup: + +Linux, Mac OS X, BSD, or other UNIX's +------------------------------------- + +For installations on systems already running ``# crond``, you can also use cron to run +``ytdl-sub`` periodically. Write a script to run ``ytdl-sub`` in the cron job. Be sure +the script changes to the same directory as your configuration and uses the full path to +``ytdl-sub``: + +.. code-block:: shell + :caption: ~/.local/bin/ytdl-sub-cron + :emphasize-lines: 2,3 + + #!/bin/bash + cd "~/.config/ytdl-sub/" + ~/.local/bin/ytdl-sub --dry-run sub -o '--ytdl_options.max_downloads 3' |& + tee -a "~/.local/state/ytdl-sub/.cron.log" + +Then tell ``# crond`` when to run the script: + +.. code-block:: console + + echo "0 */6 * * * ${HOME}/.local/bin/ytdl-sub-cron" | crontab "-" + +Remove the ``--dry-run`` and ``-o ...`` CLI options from your cron script when you've +tested your configuration and you're ready to download entries unattended. + + +.. _windows-setup: + +Windows +------- + +For most Windows users, the best way to run commands periodically is `the Task +Scheduler`_: + +.. attention:: + + These instructions are untested. Use at your own risk. If you use them, whether they + work or not, please let us know how it went in `a support post in Discord`_ or `a new + GitHub issue`_. + +#. Open the Task Scheduler app. + +#. Click ``Create Basic Task`` at the top of the right sidebar. + +#. Set all the fields as appropriate until you get to the ``Action``... + +#. For the ``Action``, select ``Start a program``... + +#. Click ``Browse...`` to the installed ``ytdl-sub.exe`` executable... + +#. Add CLI arguments to ``Add arguments (optional):``, for example ``--dry-run sub -o + '--ytdl_options.max_downloads 3'``... + +#. Set ``Start in (optional):`` to the directory containing your configuration. + +#. Finish the rest of the ``Create Basic Task`` wizard. + + +Next Steps +---------- + +At this point, ``ytdl-sub`` should run periodically and keep your subscriptions current +in your media library without your intervention. As your :doc:`subscriptions file +<./subscriptions>` grows or you discover new use cases, it becomes worth while to +simplify things by :doc:`defining your own custom presets <./first_config>`. + + + +.. _`cron`: + https://en.wikipedia.org/wiki/Cron +.. _`a cron schedule`: + https://en.wikipedia.org/wiki/Cron#Overview + +.. _`the Task Scheduler`: + https://learn.microsoft.com/en-us/windows/win32/taskschd/task-scheduler-start-page +.. _`a support post in Discord`: + https://discord.com/channels/994270357957648404/1084886228266127460 +.. _`a new GitHub issue`: + https://github.com/jmbannon/ytdl-sub/issues/new diff --git a/docs/source/guides/getting_started/automating_downloads.rst b/docs/source/guides/getting_started/automating_downloads.rst deleted file mode 100644 index 80e86902..00000000 --- a/docs/source/guides/getting_started/automating_downloads.rst +++ /dev/null @@ -1,69 +0,0 @@ -Automating Downloads -==================== - -:ref:`Guide for Docker and Unraid Containers ` - -:ref:`Guide for Linux ` - -:ref:`Guide for Windows ` - -.. _cron scheduling syntax: https://crontab.guru/#0_*/6_*_*_* - - -.. _docker-unraid-setup: - -Docker and Unraid ------------------ - -Cron is preconfigured in every ytdl-sub docker container. Enable by adding the following -ENV variables to your docker setup. - -.. code-block:: yaml - - services: - ytdl-sub: - environment: - - CRON_SCHEDULE="0 */6 * * *" - - CRON_RUN_ON_START=false - - -- ``CRON_SCHEDULE`` follows the standard `cron scheduling syntax`_. The above value will - run the script once every 6 hours. -- ``CRON_RUN_ON_START`` toggles whether to run your cron script on container start in - addition to the cron schedule. - -The cron script will reside in the main directory with the file name ``cron``. Cron -logs should show when viewing the Docker logs. - - -.. _linux-setup: - -Linux ------ -Must configure crontab manually, like so: - -.. code-block:: shell - - crontab -e - 0 */6 * * * /config/run_cron - - -.. _windows-setup: - -Windows -------- - -To be tested (please contact code owner or join the discord server if you can test this -out for us) - -.. code-block:: powershell - - ytdl-sub.exe --config \path\to\config\config.yaml sub \path\to\config\subscriptions.yaml - - -Next Steps ----------- - -Once you have a significant quantity of subscriptions or have use cases not served using -:doc:`YAML keys and the special characters <./subscriptions>`, it's time to start -:doc:`defining your own custom presets <./first_config>`. diff --git a/docs/source/guides/getting_started/quick_start.rst b/docs/source/guides/getting_started/quick_start.rst index 7647f774..8c92986b 100644 --- a/docs/source/guides/getting_started/quick_start.rst +++ b/docs/source/guides/getting_started/quick_start.rst @@ -50,5 +50,5 @@ instructions to the letter. #. Automate downloads: - :ref:`Set up ytdl-sub to run periodically - `. + :ref:`Set up ytdl-sub to run periodically `. diff --git a/docs/source/guides/install/docker.rst b/docs/source/guides/install/docker.rst index 319bdefe..f8162832 100644 --- a/docs/source/guides/install/docker.rst +++ b/docs/source/guides/install/docker.rst @@ -7,8 +7,8 @@ on top. There are two flavors or variants to choose from. For a more user-friend experience editing the `configuration`_, we recommend the `GUI image`_ variant. :ref:`Docker Compose ` is the recommended way of managing a ``ytdl-sub`` docker container. See :ref:`Automating -Downloads ` for how to -automate running ``ytdl-sub`` in a container running either variant. +Downloads ` for how to automate +running ``ytdl-sub`` in a container running either variant. GUI Image @@ -28,8 +28,8 @@ Headless Image The headless image is based on LSIO's :lsio-gh:`docker-baseimage-alpine`. Once running, the default command just starts services including cron for :ref:`Automating Downloads -` but otherwise doesn't -run ``ytdl-sub``. You may run arbitrary ``ytdl-sub`` commands using the +` but otherwise doesn't run +``ytdl-sub``. You may run arbitrary ``ytdl-sub`` commands using the ``--rm --user="${PUID}:${PGID}" --entrypoint="ytdl-sub"`` options to either ``$ docker run`` or ``$ docker compose run``. Overriding the image's ``ENTRYPOINT`` is important so that cron doesn't run ``ytdl-sub`` while you're running it manually.