danvics/ytptube
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Merge pull request #186 from arabcoders/dev

Browse source 
Major code changes.
This commit is contained in:
Abdulmohsen 2025-01-28 00:18:41 +03:00 committed by GitHub
commit f554871b9f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
29 changed files with 2838 additions and 1237 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
.vscode/settings.json vendored
View file

@ -33,4 +33,15 @@
"writedescription", "writedescription",
"xerror" "xerror"
], ],
"css.styleSheets": [
"ui/assets/css/*.css"
],
"spellright.language": [
"en-US-10-1."
],
"spellright.documentTypes": [
"markdown",
"latex",
"plaintext"
],
} }

671
API.md Normal file
View file

@ -0,0 +1,671 @@
# HTTP API Documentation
This document describes the available endpoints and their usage. All endpoints return JSON responses (unless otherwise specified) and may require certain parameters (query, body, or path). Some endpoints serve static or streaming content (e.g., `.ts`, `.m3u8`, `.vtt` files).
> **Note**: If Basic Authentication is configured (via `auth_username` and `auth_password` in your configuration), you must include an `Authorization: Basic <base64-encoded-credentials>` header or use `?apikey=<base64-encoded-credentials>` query parameter (fallback) in every request.
- All responses use standard HTTP status codes to indicate success or error conditions.
- Endpoints support an `OPTIONS` request for CORS.
---
## Table of Contents
- [HTTP API Documentation](#http-api-documentation)
- [Table of Contents](#table-of-contents)
- [Authentication](#authentication)
- [Global Notes](#global-notes)
- [Endpoints](#endpoints)
- [GET /api/ping](#get-apiping)
- [POST /api/yt-dlp/convert](#post-apiyt-dlpconvert)
- [GET /api/yt-dlp/url/info](#get-apiyt-dlpurlinfo)
- [GET /api/history/add](#get-apihistoryadd)
- [POST /api/history](#post-apihistory)
- [DELETE /api/history](#delete-apihistory)
- [POST /api/history/{id}](#post-apihistoryid)
- [GET /api/history](#get-apihistory)
- [GET /api/tasks](#get-apitasks)
- [PUT /api/tasks](#put-apitasks)
- [GET /api/workers](#get-apiworkers)
- [POST /api/workers](#post-apiworkers)
- [PATCH /api/workers/{id}](#patch-apiworkersid)
- [DELETE /api/workers/{id}](#delete-apiworkersid)
- [GET /api/player/playlist/{file:.\*}.m3u8](#get-apiplayerplaylistfilem3u8)
- [GET /api/player/m3u8/{mode}/{file:.\*}.m3u8](#get-apiplayerm3u8modefilem3u8)
- [GET /api/player/segments/{segment}/{file:.\*}.ts](#get-apiplayersegmentssegmentfilets)
- [GET /api/player/subtitle/{file:.\*}.vtt](#get-apiplayersubtitlefilevtt)
- [GET /api/thumbnail](#get-apithumbnail)
- [GET /api/file/ffprobe/{file:.\*}](#get-apifileffprobefile)
- [GET /api/youtube/auth](#get-apiyoutubeauth)
- [GET /api/notifications](#get-apinotifications)
- [PUT /api/notifications](#put-apinotifications)
- [POST /api/notifications/test](#post-apinotificationstest)
- [Error Responses](#error-responses)
---
## Authentication
If `Config.auth_username` and `Config.auth_password` are set, all API requests must include a valid credential using one of the following:
1. HTTP Basic Auth header:
```
Authorization: Basic base64("<username>:<password>")
```
2. Query Parameter fallback:
```
?apikey=<base64("<username>:<password>")>
```
If you fail to provide valid credentials, a `401 Unauthorized` response is returned.
---
## Global Notes
- **Content-Type**
- Requests expecting a JSON body should include `Content-Type: application/json`.
- Responses typically include `Content-Type: application/json`, unless returning a file or streaming resource.
- **Status Codes**
- `200 OK` on success.
- `4xx` on client errors (e.g., missing parameters).
- `5xx` on server errors (e.g., unexpected failures).
- **Error Responses**
When an error occurs, responses follow a structure similar to:
```json
{
"error": "Description of the error",
"message": "More details if available"
}
```
with an appropriate HTTP status code.
---
## Endpoints
### GET /api/ping
**Purpose**: Health-check endpoint.
**Response**:
```json
{
"status": "pong"
}
```
---
### POST /api/yt-dlp/convert
**Purpose**: Convert a string of yt-dlp or youtube-dl CLI arguments into a JSON-friendly structure.
**Body**:
```json
{
"args": "<string>"
}
```
**Example**:
```json
{
"args": "--write-sub --embed-subs"
}
```
**Response**:
```json
{
"postprocessors": [
{
"already_have_subtitle": true,
"key": "FFmpegEmbedSubtitle"
}
],
"writesubtitles": true
}
```
or an error:
```json
{
"error": "Failed to convert args. '<reason>'."
}
```
---
### GET /api/yt-dlp/url/info
**Purpose**: Retrieves metadata (info) for a provided URL without adding it to the download queue.
**Query Parameters**:
- `?url=<video-url>`
**Response** (example):
```json
{
"title": "...",
"duration": 123.4,
"extractor": "youtube",
"_cached": {
"key": "<hash>",
"ttl": 300,
"ttl_left": 299.82,
"expires": 1690430096.429
},
...
}
```
or an error:
```json
{
"error": "text"
}
```
- If the URL is invalid or missing, returns `400 Bad Request`.
---
### GET /api/history/add
**Purpose**: **(Quick Add)** Add a single URL to the download queue via GET.
**Query Parameters**:
- `?url=<video-url>`
**Response**:
```json
{
"status": "(ok|error)",
}
```
or an error:
```json
{
"error": "text"
}
```
- If `url` is missing, returns `400 Bad Request`.
---
### POST /api/history
**Purpose**: Add one or multiple items (URLs) to the download queue via JSON body.
**Body**:
```json
// Single item
{
"url": "https://youtube.com/watch?v=...",
"preset": "default",
"folder": "folder relative to download_path",
"cookies": "...",
"template": "...",
"config": {
/* advanced config options for yt-dlp */
}
}
// Or multiple items (array of objects)
[
{
"url": "...",
"preset": "default",
...
},
{
"url": "...",
...
}
]
```
**Response**:
```json
[
{
"status": "queued",
},
...
]
```
- If any item is invalid (e.g., missing `url`), returns `400 Bad Request`.
---
### DELETE /api/history
**Purpose**: Delete items from either the "queue" or the "done" history.
**Body**:
```json
{
"ids": ["<id1>", "<id2>"],
"where": "queue" | "done",
"remove_file": true | false // optional, defaults to false, whether to delete the file from disk.
}
```
**Response**:
```json
{
"<id1>": "status",
"<id2>": "status",
...
}
```
A list or object indicating which items were removed.
---
### POST /api/history/{id}
**Purpose**: Update an item in the `completed` history.
**Path Parameter**:
- `id` = Unique item ID.
**Body** (example):
```json
{
"title": "My Custom Title",
"someOtherField": "new-value"
}
```
**Response**:
```json
{
"title": "My Custom Title",
....
}
```
or an error:
```json
{
"error": "text"
}
```
- `200 OK` with updated item if successful.
- `304 Not Modified` if nothing changed.
- `404 Not Found` if the item doesn’t exist.
- `400 Bad Request` if id is missing or the body is empty.
---
### GET /api/history
**Purpose**: Returns the download queue and the download history.
**Response**:
```json
{
"queue": [
{ ... },
...
],
"history": [
{ ... },
...
]
}
```
---
### GET /api/tasks
**Purpose**: Retrieves the scheduled tasks from the internal `Tasks` manager.
**Response**:
```json
[
{
"id": "<uuid>",
"name": "...",
"url": "...",
"folder": "...",
"preset": "...",
"timer": "<cron-expression>",
"template": "...",
"cookies": "...",
"config": { ... },
},
...
]
```
---
### PUT /api/tasks
**Purpose**: Overwrites the entire scheduled tasks list (Cron tasks).
**Body**: An array of task objects. Example:
```json
[
{
"id": "a2ae3f18-4428-4e32-9d4c-0cc45af8bb48",
"name": "My Task",
"url": "https://youtube.com/...",
"timer": "5 */2 * * *",
"cookies": "",
"config": {},
"template": "...",
"folder": "..."
},
{
"url": "https://youtube.com/...",
"timer": "*/15 * * * *"
}
]
```
If `id` or other fields are missing, they may be auto-generated or defaulted (e.g., a random ID, a default cron, etc.).
**Response**:
```json
[
{
"id": "<uuid>",
"name": "...",
"url": "...",
"timer": "...",
"cookies": "...",
"config": { ... },
"template": "...",
"folder": "..."
}
...
]
```
or on error
```json
{
"error": "text"
}
```
---
### GET /api/workers
**Purpose**: Returns the status of the worker pool and all workers.
**Response**:
```json
{
"open": true|false,
"count": 4,
"workers": [
{
"id": "worker-1",
"data": { "status": "downloading", ... }
},
{
"id": "worker-2",
"data": { "status": "Waiting for download." }
},
...
]
}
```
- `open`: Indicates if there are any available workers.
- `count`: Total number of available workers.
---
### POST /api/workers
**Purpose**: Restart the entire worker pool.
**Response**:
```json
{
"message": "Workers pool being restarted."
}
```
---
### PATCH /api/workers/{id}
**Purpose**: Restart a single worker by ID.
**Path Parameter**:
- `id` = The worker ID.
**Response**:
```json
{
"status": "restarted"
}
```
or
```json
{
"status": "in_error_state"
}
```
---
### DELETE /api/workers/{id}
**Purpose**: Stop a single worker by ID.
**Path Parameter**:
- `id` = The worker ID.
**Response**:
```json
{
"status": "stopped"
}
```
or
```json
{
"status": "in_error_state"
}
```
---
### GET /api/player/playlist/{file:.*}.m3u8
**Purpose**: Generate a playlist for a given local media file.
**Path Parameter**:
- `file` = Relative path of the media file within the `download_path`.
**Response**:
An `.m3u8` playlist.
---
### GET /api/player/m3u8/{mode}/{file:.*}.m3u8
**Purpose**: Dynamically generate an M3U8 playlist for video or subtitles.
**Path Parameters**:
- `mode`: either `video` or `subtitle`.
- `file`: relative path of the file.
**Query Parameters (when `mode=subtitle`)**:
- `duration`: The total duration of the subtitle track
**Response**:
- `Content-Type: application/x-mpegURL` containing the `.m3u8` text.
---
### GET /api/player/segments/{segment}/{file:.*}.ts
**Purpose**: Streams a single TS segment for adaptive HLS playback.
**Path Parameters**:
- `segment` = Numeric segment index.
- `file` = Relative file path.
**Query Parameters**:
- `sd` = The segment duration (float).
- `vc` = `1` or `0` (whether to convert video).
- `ac` = `1` or `0` (whether to convert audio).
**Response**:
Binary TS data (`Content-Type: video/mpegts`).
---
### GET /api/player/subtitle/{file:.*}.vtt
**Purpose**: Provides a `.vtt` (WebVTT) subtitle file for playback.
**Path Parameter**:
- `file` = Relative path of the subtitle file.
**Response**:
`Content-Type: text/vtt; charset=UTF-8`.
---
### GET /api/thumbnail
**Purpose**: Proxy/fetch a remote thumbnail image.
**Query Parameter**:
- `?url=<remote-thumbnail-url>`
**Response**:
Binary image data with the appropriate `Content-Type`.
---
### GET /api/file/ffprobe/{file:.*}
**Purpose**: Return the `ffprobe` data for a local file.
**Path Parameter**:
- `file` = Relative path to the file inside `download_path`.
**Response**:
```json
{
"streams": [...],
"format": { ... },
...
}
```
---
### GET /api/youtube/auth
**Purpose**: Checks if a valid YouTube session cookie is set (to confirm authentication).
**Response**:
```json
{
"message": "Authenticated."
}
```
Returns `200 OK` if cookies are valid, otherwise `401` with `{ "message": "Not authenticated." }`.
---
### GET /api/notifications
**Purpose**: Retrieve the configured notification targets and which event types are allowed.
**Response** (example):
```json
{
"notifications": [
{
"id": "uuid",
"name":"...",
"on": ["completed", "error",...], // empty array means all events.
"request":{
"type":"json|form",
"method":"POST|PUT",
"url":"https://...",
"headers":[
{"key":"...", "value":"..."},
...
]
}
}
}
],
"allowedTypes": ["added", "completed", "error", "cancelled", "cleared", "log_info", "log_success", ...]
}
```
---
### PUT /api/notifications
**Purpose**: Overwrites the entire list of notification targets.
**Body**: An array of notification target configurations. Example:
```json
[
{
"id": "uuid",
"name": "My Webhook",
"on": ["completed", "error"],
"request": {
"type": "json",
"method": "POST",
"url": "https://...",
"headers": [
{ "key": "Authorization", "value": "Bearer ..." }
]
}
},
{
"name": "Another Webhook",
"on": ["completed"],
"request": {
"type": "form",
"method": "PUT",
"url": "https://...",
"headers": []
}
}
...
]
```
- If `id` is not provided or is not a valid UUIDv4, it will be auto-generated.
- If the payload list is empty, all existing notifications are removed.
**Response**:
```json
{
"notifications": [
{
"id": "uuid",
"name": "...",
"on": ["completed", "error", ...],
"request": { ... }
},
...
],
"allowedTypes": ["added", "completed", "error", "cancelled", "cleared", "log_info", "log_success", ...]
}
```
---
### POST /api/notifications/test
**Purpose**: Triggers a test notification event to all configured targets.
**Response**:
```json
{
"type": "test",
"message": "This is a test notification."
}
```
---
## Error Responses
Most endpoints return standard error codes (`400`, `403`, `404`, `500`, etc.) and a JSON body on failure. For example:
```json
{
"error": "url param is required."
}
```
with `400 Bad Request`, or:
```json
{
"error": "Item not found."
}
```
with `404 Not Found`.
---

12
Pipfile.lock generated
View file

@ -142,11 +142,11 @@
}, },
"attrs": { "attrs": {
"hashes": [ "hashes": [
"sha256:8f5c07333d543103541ba7be0e2ce16eeee8130cb0b3f9238ab904ce1e85baff", "sha256:1c97078a80c814273a76b2a298a932eb681c87415c11dee0a6921de7f1b02c3e",
"sha256:ac96cd038792094f438ad1f6ff80837353805ac950cd2aa0e0625ef19850c308" "sha256:c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a"
], ],
"markers": "python_version >= '3.8'", "markers": "python_version >= '3.8'",
"version": "==24.3.0" "version": "==25.1.0"
}, },
"bidict": { "bidict": {
"hashes": [ "hashes": [
@ -1247,12 +1247,12 @@
}, },
"yt-dlp": { "yt-dlp": {
"hashes": [ "hashes": [
"sha256:b8666b88e23c3fa5ee1e80920f4a9dfac7c405504a447214c0cf3d0c386edcfc", "sha256:1c9738266921ad43c568ad01ac3362fb7c7af549276fbec92bd72f140da16240",
"sha256:e8ec515d49bb62704915d13a22ee6fe03a5658d651e4e64574e3a17ee01f6e3b" "sha256:3e76bd896b9f96601021ca192ca0fbdd195e3c3dcc28302a3a34c9bc4979da7b"
], ],
"index": "pypi", "index": "pypi",
"markers": "python_version >= '3.9'", "markers": "python_version >= '3.9'",
"version": "==2025.1.15" "version": "==2025.1.26"
} }
}, },
"develop": {} "develop": {}

5
README.md
View file

@ -14,12 +14,15 @@ YTPTube started as a fork of [meTube](https://github.com/alexta69/metube), Since
* Support per link `yt-dlp JSON config or cli options`, `cookies` & `output format`. * Support per link `yt-dlp JSON config or cli options`, `cookies` & `output format`.
* Queue multiple URLs separated by comma. * Queue multiple URLs separated by comma.
* A built in video player that can play any video file regardless of the format. **With support for sidecar external subtitles**. * A built in video player that can play any video file regardless of the format. **With support for sidecar external subtitles**.
* New `/api/add_batch` endpoint that allow multiple links to be sent. * New `POST /api/history` endpoint that allow one or multiple links to be sent at the same time.
* New `GET /api/history/add?url=http://..` endpoint that allow to add single item via GET request.
* Completely redesigned the frontend UI. * Completely redesigned the frontend UI.
* Switched out of binary file storage in favor of SQLite. * Switched out of binary file storage in favor of SQLite.
* Basic Authentication support. * Basic Authentication support.
* Support for curl_cffi, see [yt-dlp documentation](https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#impersonation) * Support for curl_cffi, see [yt-dlp documentation](https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#impersonation)
* Support for both advanced and basic mode for WebUI. * Support for both advanced and basic mode for WebUI.
For more API endpoints, please refer to the [API documentation](API.md).
### Tips ### Tips
Your `yt-dlp` config should include the following options for optimal working conditions. Your `yt-dlp` config should include the following options for optimal working conditions.

2
app/library/DataStore.py
View file

@ -106,7 +106,7 @@ class DataStore:
def getNextDownload(self) -> Download: def getNextDownload(self) -> Download:
for key in self.dict: for key in self.dict:
if self.dict[key].started() is False and self.dict[key].is_canceled() is False: if self.dict[key].started() is False and self.dict[key].is_cancelled() is False:
return self.dict[key] return self.dict[key]
return None return None

26
app/library/Download.py
View file

@ -28,15 +28,15 @@ class Download:
id: str = None id: str = None
download_dir: str = None download_dir: str = None
temp_dir: str = None temp_dir: str = None
output_template: str = None template: str = None
output_template_chapter: str = None template_chapter: str = None
ytdl_opts: dict = None ytdl_opts: dict = None
info: ItemDTO = None info: ItemDTO = None
default_ytdl_opts: dict = None default_ytdl_opts: dict = None
debug: bool = False debug: bool = False
tempPath: str = None tempPath: str = None
emitter: Emitter = None emitter: Emitter = None
canceled: bool = False cancelled: bool = False
is_live: bool = False is_live: bool = False
info_dict: dict = None info_dict: dict = None
"yt-dlp metadata dict." "yt-dlp metadata dict."
@ -72,15 +72,15 @@ class Download:
self.download_dir = info.download_dir self.download_dir = info.download_dir
self.temp_dir = info.temp_dir self.temp_dir = info.temp_dir
self.output_template_chapter = info.output_template_chapter self.template = info.template
self.output_template = info.output_template self.template_chapter = info.template_chapter
self.preset = info.preset self.preset = info.preset
self.ytdl_opts = info.ytdlp_config if info.ytdlp_config else {} self.ytdl_opts = info.config if info.config else {}
self.info = info self.info = info
self.id = info._id self.id = info._id
self.default_ytdl_opts = config.ytdl_options self.default_ytdl_opts = config.ytdl_options
self.debug = debug self.debug = debug
self.canceled = False self.cancelled = False
self.tmpfilename = None self.tmpfilename = None
self.status_queue = None self.status_queue = None
self.proc = None self.proc = None
@ -120,7 +120,7 @@ class Download:
{ {
"color": "no_color", "color": "no_color",
"paths": {"home": self.download_dir, "temp": self.tempPath}, "paths": {"home": self.download_dir, "temp": self.tempPath},
"outtmpl": {"default": self.output_template, "chapter": self.output_template_chapter}, "outtmpl": {"default": self.template, "chapter": self.template_chapter},
"noprogress": True, "noprogress": True,
"break_on_existing": True, "break_on_existing": True,
"progress_hooks": [self._progress_hook], "progress_hooks": [self._progress_hook],
@ -136,9 +136,9 @@ class Download:
params["verbose"] = True params["verbose"] = True
params["noprogress"] = False params["noprogress"] = False
if self.info.ytdlp_cookies: if self.info.cookies:
try: try:
data = jsonCookie(json.loads(self.info.ytdlp_cookies)) data = jsonCookie(json.loads(self.info.cookies))
if not data: if not data:
LOG.warning( LOG.warning(
f"The cookie string that was provided for {self.info.title} is empty or not in expected spec." f"The cookie string that was provided for {self.info.title} is empty or not in expected spec."
@ -212,7 +212,7 @@ class Download:
if not self.started(): if not self.started():
return False return False
self.canceled = True self.cancelled = True
return True return True
@ -271,8 +271,8 @@ class Download:
except ValueError: except ValueError:
return False return False
def is_canceled(self) -> bool: def is_cancelled(self) -> bool:
return self.canceled return self.cancelled
def kill(self) -> bool: def kill(self) -> bool:
if not self.running(): if not self.running():

194
app/library/DownloadQueue.py
View file

@ -15,50 +15,116 @@ from .DataStore import DataStore
from .Download import Download from .Download import Download
from .Emitter import Emitter from .Emitter import Emitter
from .ItemDTO import ItemDTO from .ItemDTO import ItemDTO
from .Utils import ExtractInfo, get_opts, calcDownloadPath, isDownloaded, mergeConfig from .Singleton import Singleton
from .EventsSubscriber import Events
from .Utils import ExtractInfo, calcDownloadPath, get_opts, isDownloaded, mergeConfig
LOG = logging.getLogger("DownloadQueue") LOG = logging.getLogger("DownloadQueue")
TYPE_DONE: str = "done"
TYPE_QUEUE: str = "queue"
class DownloadQueue: class DownloadQueue(metaclass=Singleton):
"""
DownloadQueue class is a singleton class that manages the download queue and the download history.
"""
TYPE_DONE: str = "done"
"""Queue type for completed downloads."""
TYPE_QUEUE: str = "queue"
"""Queue type for pending downloads."""
paused: asyncio.Event paused: asyncio.Event
event: asyncio.Event | None = None """Event to pause the download queue."""
pool: AsyncPool | None = None
def __init__(self, emitter: Emitter, connection: Connection): event: asyncio.Event
self.config = Config.get_instance() """Event to signal the download queue to start downloading."""
self.emitter = emitter
self.done = DataStore(type=TYPE_DONE, connection=connection) pool: AsyncPool | None = None
self.queue = DataStore(type=TYPE_QUEUE, connection=connection) """Pool of workers to download the files."""
_instance = None
"""Instance of the DownloadQueue."""
def __init__(self, connection: Connection, emitter: Emitter | None = None, config: Config | None = None):
DownloadQueue._instance = self
self.config = config or Config.get_instance()
self.emitter = emitter or Emitter.get_instance()
self.done = DataStore(type=DownloadQueue.TYPE_DONE, connection=connection)
self.queue = DataStore(type=DownloadQueue.TYPE_QUEUE, connection=connection)
self.done.load() self.done.load()
self.queue.load() self.queue.load()
self.paused = asyncio.Event() self.paused = asyncio.Event()
self.paused.set() self.paused.set()
self.event = asyncio.Event()
@staticmethod
def get_instance():
"""
Get the instance of the DownloadQueue.
Returns:
DownloadQueue: The instance of the DownloadQueue
"""
if not DownloadQueue._instance:
DownloadQueue._instance = DownloadQueue()
return DownloadQueue._instance
async def test(self) -> bool: async def test(self) -> bool:
"""
Test the datastore connection to the database.
Returns:
bool: True if the test is successful, False otherwise.
"""
await self.done.test() await self.done.test()
return True return True
async def initialize(self): async def initialize(self):
self.event = asyncio.Event() """
Initialize the download queue.
"""
LOG.info( LOG.info(
f"Using '{self.config.max_workers}' worker/s for downloading. Can be configured via `YTP_MAX_WORKERS` environment variable." f"Using '{self.config.max_workers}' worker/s for downloading. Can be configured via `YTP_MAX_WORKERS` environment variable."
) )
asyncio.create_task(self.__download_pool(), name="download_pool") asyncio.create_task(self.__download_pool(), name="download_pool")
def pause(self): def pause(self) -> bool:
"""
Pause the download queue.
Returns:
bool: True if the download is paused, False otherwise
"""
if self.paused.is_set(): if self.paused.is_set():
self.paused.clear() self.paused.clear()
LOG.warning(f"Download paused at. {datetime.datetime.now().isoformat()}") LOG.warning(f"Download paused at. {datetime.datetime.now().isoformat()}")
return True
def resume(self): return False
def resume(self) -> bool:
"""
Resume the download queue.
Returns:
bool: True if the download is resumed, False otherwise
"""
if not self.paused.is_set(): if not self.paused.is_set():
self.paused.set() self.paused.set()
LOG.warning(f"Downloading resumed at. {datetime.datetime.now().isoformat()}") LOG.warning(f"Downloading resumed at. {datetime.datetime.now().isoformat()}")
return True
return False
def isPaused(self) -> bool: def isPaused(self) -> bool:
"""
Check if the download queue is paused.
Returns:
bool: True if the download queue is paused, False otherwise
"""
return False if self.paused.is_set() else True return False if self.paused.is_set() else True
async def __add_entry( async def __add_entry(
@ -66,11 +132,26 @@ class DownloadQueue:
entry: dict, entry: dict,
preset: str, preset: str,
folder: str, folder: str,
ytdlp_config: dict = {}, config: dict = {},
ytdlp_cookies: str = "", cookies: str = "",
output_template: str = "", template: str = "",
already=None, already=None,
): ):
"""
Add an entry to the download queue.
Args:
entry (dict): The entry to add to the download queue.
preset (str): The preset to use for the download.
folder (str): The folder to save the download to.
config (dict): The yt-dlp configuration to use for the download.
cookies (str): The cookies to use for the download.
template (str): The output template to use for the download.
already (set): The set of already downloaded items.
Returns:
dict: The status of the operation.
"""
if not entry: if not entry:
return {"status": "error", "msg": "Invalid/empty data was given."} return {"status": "error", "msg": "Invalid/empty data was given."}
@ -99,9 +180,9 @@ class DownloadQueue:
entry=etr, entry=etr,
preset=preset, preset=preset,
folder=folder, folder=folder,
ytdlp_config=ytdlp_config, config=config,
ytdlp_cookies=ytdlp_cookies, cookies=cookies,
output_template=output_template, template=template,
already=already, already=already,
) )
) )
@ -168,10 +249,10 @@ class DownloadQueue:
folder=folder, folder=folder,
download_dir=download_dir, download_dir=download_dir,
temp_dir=self.config.temp_path, temp_dir=self.config.temp_path,
ytdlp_cookies=ytdlp_cookies, cookies=cookies,
ytdlp_config=ytdlp_config, config=config,
output_template=output_template if output_template else self.config.output_template, template=template if template else self.config.output_template,
output_template_chapter=self.config.output_template_chapter, template_chapter=self.config.output_template_chapter,
datetime=formatdate(time.time()), datetime=formatdate(time.time()),
error=error, error=error,
is_live=is_live, is_live=is_live,
@ -182,24 +263,23 @@ class DownloadQueue:
for property, value in entry.items(): for property, value in entry.items():
if property.startswith("playlist"): if property.startswith("playlist"):
dl.output_template = str(dl.output_template).replace(f"%({property})s", str(value)) dl.template = str(dl.template).replace(f"%({property})s", str(value))
dlInfo: Download = Download(info=dl, info_dict=entry, debug=bool(self.config.ytdl_debug)) dlInfo: Download = Download(info=dl, info_dict=entry, debug=bool(self.config.ytdl_debug))
if dlInfo.info.live_in or "is_upcoming" == entry.get("live_status", None): if dlInfo.info.live_in or "is_upcoming" == entry.get("live_status", None):
dlInfo.info.status = "not_live" dlInfo.info.status = "not_live"
itemDownload = self.done.put(dlInfo) itemDownload = self.done.put(dlInfo)
NotifyEvent = "completed" NotifyEvent = Events.COMPLETED
elif self.config.allow_manifestless is False and is_manifestless is True: elif self.config.allow_manifestless is False and is_manifestless is True:
dlInfo.info.status = "error" dlInfo.info.status = "error"
dlInfo.info.error = "Video is in post-live manifestless mode." dlInfo.info.error = "Video is in post-live manifestless mode."
itemDownload = self.done.put(dlInfo) itemDownload = self.done.put(dlInfo)
NotifyEvent = "completed" NotifyEvent = Events.COMPLETED
else: else:
NotifyEvent = "added" NotifyEvent = Events.ADDED
itemDownload = self.queue.put(dlInfo) itemDownload = self.queue.put(dlInfo)
if self.event: self.event.set()
self.event.set()
asyncio.create_task( asyncio.create_task(
self.emitter.emit(NotifyEvent, itemDownload.info), name=f"notifier-{NotifyEvent}-{itemDownload.info.id}" self.emitter.emit(NotifyEvent, itemDownload.info), name=f"notifier-{NotifyEvent}-{itemDownload.info.id}"
@ -211,9 +291,9 @@ class DownloadQueue:
url=str(entry.get("url")), url=str(entry.get("url")),
preset=preset, preset=preset,
folder=folder, folder=folder,
ytdlp_config=ytdlp_config, config=config,
ytdlp_cookies=ytdlp_cookies, cookies=cookies,
output_template=output_template, template=template,
already=already, already=already,
) )
@ -224,25 +304,25 @@ class DownloadQueue:
url: str, url: str,
preset: str, preset: str,
folder: str, folder: str,
ytdlp_config: dict = {}, config: dict = {},
ytdlp_cookies: str = "", cookies: str = "",
output_template: str = "", template: str = "",
already=None, already=None,
): ):
ytdlp_config = ytdlp_config if ytdlp_config else {} config = config if config else {}
folder = str(folder) if folder else "" folder = str(folder) if folder else ""
filePath = calcDownloadPath(basePath=self.config.download_path, folder=folder) filePath = calcDownloadPath(basePath=self.config.download_path, folder=folder)
LOG.info( LOG.info(
f"Adding 'URL: {url}' to 'Folder: {filePath}' with 'Preset: {preset}' 'Naming: {output_template}', 'Cookies: {ytdlp_cookies}' 'YTConfig: {ytdlp_config}'." f"Adding 'URL: {url}' to 'Folder: {filePath}' with 'Preset: {preset}' 'Naming: {template}', 'Cookies: {cookies}' 'YTConfig: {config}'."
) )
if isinstance(ytdlp_config, str): if isinstance(config, str):
try: try:
ytdlp_config = json.loads(ytdlp_config) config = json.loads(config)
except Exception as e: except Exception as e:
LOG.error(f"Unable to load '{ytdlp_config=}'. {str(e)}") LOG.error(f"Unable to load '{config=}'. {str(e)}")
return {"status": "error", "msg": f"Failed to parse json yt-dlp config. {str(e)}"} return {"status": "error", "msg": f"Failed to parse json yt-dlp config. {str(e)}"}
already = set() if already is None else already already = set() if already is None else already
@ -267,7 +347,7 @@ class DownloadQueue:
fut=asyncio.get_running_loop().run_in_executor( fut=asyncio.get_running_loop().run_in_executor(
None, None,
ExtractInfo, ExtractInfo,
get_opts(preset, mergeConfig(self.config.ytdl_options, ytdlp_config)), get_opts(preset, mergeConfig(self.config.ytdl_options, config)),
url, url,
bool(self.config.ytdl_debug), bool(self.config.ytdl_debug),
), ),
@ -297,9 +377,9 @@ class DownloadQueue:
entry=entry, entry=entry,
preset=preset, preset=preset,
folder=folder, folder=folder,
ytdlp_config=ytdlp_config, config=config,
ytdlp_cookies=ytdlp_cookies, cookies=cookies,
output_template=output_template, template=template,
already=already, already=already,
) )
@ -326,9 +406,9 @@ class DownloadQueue:
await item.close() await item.close()
LOG.debug(f"Deleting from queue {itemMessage}") LOG.debug(f"Deleting from queue {itemMessage}")
self.queue.delete(id) self.queue.delete(id)
asyncio.create_task(self.emitter.canceled(dl=item.info.serialize()), name=f"notifier-c-{id}") asyncio.create_task(self.emitter.cancelled(dl=item.info.serialize()), name=f"notifier-c-{id}")
item.info.status = "canceled" item.info.status = "cancelled"
item.info.error = "Canceled by user." item.info.error = "Cancelled by user."
self.done.put(item) self.done.put(item)
asyncio.create_task(self.emitter.completed(dl=item.info.serialize()), name=f"notifier-d-{id}") asyncio.create_task(self.emitter.completed(dl=item.info.serialize()), name=f"notifier-d-{id}")
LOG.info(f"Deleted from queue {itemMessage}") LOG.info(f"Deleted from queue {itemMessage}")
@ -384,6 +464,12 @@ class DownloadQueue:
return status return status
def get(self) -> dict[str, list[dict[str, ItemDTO]]]: def get(self) -> dict[str, list[dict[str, ItemDTO]]]:
"""
Get the download queue and the download history.
Returns:
dict: The download queue and the download history.
"""
items = {"queue": {}, "done": {}} items = {"queue": {}, "done": {}}
for k, v in self.queue.saved_items(): for k, v in self.queue.saved_items():
@ -444,7 +530,7 @@ class DownloadQueue:
LOG.debug(f"Pushing {entry=} to executor.") LOG.debug(f"Pushing {entry=} to executor.")
if entry.started() is False and entry.is_canceled() is False: if entry.started() is False and entry.is_cancelled() is False:
await self.pool.push(id=entry.info._id, entry=entry) await self.pool.push(id=entry.info._id, entry=entry)
LOG.debug(f"Pushed {entry=} to executor.") LOG.debug(f"Pushed {entry=} to executor.")
await asyncio.sleep(1) await asyncio.sleep(1)
@ -458,7 +544,7 @@ class DownloadQueue:
try: try:
await entry.start(self.emitter) await entry.start(self.emitter)
if entry.info.status != "finished": if "finished" != entry.info.status:
if entry.tmpfilename and os.path.isfile(entry.tmpfilename): if entry.tmpfilename and os.path.isfile(entry.tmpfilename):
try: try:
os.remove(entry.tmpfilename) os.remove(entry.tmpfilename)
@ -474,10 +560,10 @@ class DownloadQueue:
LOG.debug(f"Download '{id}' is done. Removing from queue.") LOG.debug(f"Download '{id}' is done. Removing from queue.")
self.queue.delete(key=id) self.queue.delete(key=id)
if entry.is_canceled() is True: if entry.is_cancelled() is True:
asyncio.create_task(self.emitter.canceled(dl=entry.info.serialize()), name=f"notifier-c-{id}") asyncio.create_task(self.emitter.cancelled(dl=entry.info.serialize()), name=f"notifier-c-{id}")
entry.info.status = "canceled" entry.info.status = "cancelled"
entry.info.error = "Canceled by user." entry.info.error = "Cancelled by user."
self.done.put(value=entry) self.done.put(value=entry)
asyncio.create_task(self.emitter.completed(dl=entry.info.serialize()), name=f"notifier-d-{id}") asyncio.create_task(self.emitter.completed(dl=entry.info.serialize()), name=f"notifier-d-{id}")

101
app/library/Emitter.py
View file

@ -1,76 +1,129 @@
import asyncio import asyncio
import logging import logging
from typing import Awaitable
from .Singleton import Singleton
from .EventsSubscriber import Events
LOG = logging.getLogger("Emitter") LOG = logging.getLogger("Emitter")
class Emitter: class Emitter(metaclass=Singleton):
""" """
This class is used to emit events to the registered emitters. This class is used to emit events to the registered emitters.
""" """
emitters: list[callable] = [] # type: ignore emitters: list[(str, Awaitable)] = []
"""The emitters for the events."""
def add_emitter(self, emitter: callable): # type: ignore _instance = None
def __init__(self):
Emitter._instance = self
@staticmethod
def get_instance():
"""
Get the instance of the Emitter.
Returns:
Emitter: The instance of the Emitter
"""
if not Emitter._instance:
Emitter._instance = Emitter()
return Emitter._instance
def add_emitter(self, emitter: list[Awaitable] | Awaitable, local: bool = False) -> "Emitter":
""" """
Add an emitter to the list of emitters. Add an emitter to the list of emitters.
Args: Args:
emitter (function): The emitter function. The function must return a coroutine or None. emitter (Awaitable|list[Awaitable]): The emitter function. The function must return a coroutine or None.
local (bool): Mark the emitter as target for local events.
Returns:
Emitter: The instance of the Emitter
""" """
self.emitters.append(emitter) if not isinstance(emitter, list):
emitter = [emitter]
async def added(self, dl: dict, **kwargs): for e in emitter:
await self.emit("added", dl, **kwargs) self.emitters.append((local, e))
async def updated(self, dl: dict, **kwargs): return self
await self.emit("updated", dl, **kwargs)
async def completed(self, dl: dict, **kwargs): async def added(self, dl: dict, local: bool = False, **kwargs):
await self.emit("completed", dl, **kwargs) await self.emit(Events.ADDED, data=dl, local=local, **kwargs)
async def canceled(self, dl: dict, **kwargs): async def updated(self, dl: dict, local: bool = False, **kwargs):
await self.emit("canceled", dl, **kwargs) await self.emit(Events.UPDATED, data=dl, local=local, **kwargs)
async def cleared(self, dl: dict | None = None, **kwargs): async def completed(self, dl: dict, local: bool = False, **kwargs):
await self.emit("cleared", dl, **kwargs) await self.emit(Events.COMPLETED, data=dl, local=local, **kwargs)
async def error(self, message: str, data: dict = {}, **kwargs): async def cancelled(self, dl: dict, local: bool = False, **kwargs):
await self.emit(Events.CANCELLED, data=dl, local=local, **kwargs)
async def cleared(self, dl: dict | None = None, local: bool = False, **kwargs):
await self.emit(Events.CLEARED, data=dl, local=local, **kwargs)
async def error(self, message: str, data: dict = {}, local: bool = False, **kwargs):
msg = {"type": "error", "message": message, "data": {}} msg = {"type": "error", "message": message, "data": {}}
if data: if data:
msg.update({"data": data}) msg.update({"data": data})
await self.emit("error", msg, **kwargs) await self.emit(Events.ERROR, data=msg, local=local, **kwargs)
async def info(self, message: str, data: dict = {}, **kwargs): async def info(self, message: str, data: dict = {}, local: bool = False, **kwargs):
msg = {"type": "info", "message": message, "data": {}} msg = {"type": "info", "message": message, "data": {}}
if data: if data:
msg.update({"data": data}) msg.update({"data": data})
await self.emit("log_info", msg, **kwargs) await self.emit(Events.LOG_INFO, data=msg, local=local, **kwargs)
async def success(self, message: str, data: dict = {}, **kwargs): async def success(self, message: str, data: dict = {}, local: bool = False, **kwargs):
msg = {"type": "success", "message": message, "data": {}} msg = {"type": "success", "message": message, "data": {}}
if data: if data:
msg.update({"data": data}) msg.update({"data": data})
await self.emit("log_success", msg, **kwargs) await self.emit(Events.LOG_SUCCESS, data=msg, local=local, **kwargs)
async def emit(self, event: str, data, **kwargs): async def emit(self, event: str, data, local: bool = False, **kwargs):
"""
Emit an event.
Args:
event (str): The event to emit.
data (dict): The data to send with the event.
local (bool): If the event should be sent to the local emitters only.
kwargs: Additional arguments to pass to the emitters.
Returns:
None
"""
tasks = [] tasks = []
for emitter in self.emitters: for emitter in self.emitters:
try: try:
_ret = emitter(event, data, **kwargs) isLocal, callback = emitter
if local and not isLocal:
continue
_ret = callback(event, data, **kwargs)
if _ret: if _ret:
if isinstance(_ret, list): if isinstance(_ret, list):
tasks.extend(_ret) tasks.extend(_ret)
else: else:
tasks.append(_ret) tasks.append(_ret)
except Exception as e: except Exception as e:
LOG.error(f"Emitter '{emitter}' failed with error '{e}'.") LOG.error(f"Emitter '{callback}' failed with error '{e}'.")
if len(tasks) < 1: if len(tasks) < 1:
return return
try: try:
await asyncio.wait_for(asyncio.gather(*tasks), timeout=60) await asyncio.wait_for(asyncio.gather(*tasks), timeout=60)
except asyncio.CancelledError:
LOG.error(f"Cancelled sending event '{event}'.")
except asyncio.TimeoutError: except asyncio.TimeoutError:
LOG.error(f"Timed out sending event '{event}'.") LOG.error(f"Timed out sending event '{event}'.")
except Exception as e:
LOG.exception(e)
LOG.error(f"Failed to send event '{event}'. '{str(e)}'.")

187
app/library/EventsSubscriber.py Normal file
View file

@ -0,0 +1,187 @@
import asyncio
import logging
from typing import Awaitable
from dataclasses import dataclass
from .Singleton import Singleton
LOG = logging.getLogger("EventsSubscriber")
class Events:
"""
The events that can be emitted.
"""
ADDED = "added"
UPDATED = "updated"
COMPLETED = "completed"
CANCELLED = "cancelled"
CLEARED = "cleared"
ERROR = "error"
LOG_INFO = "log_info"
LOG_SUCCESS = "log_success"
INITIAL_DATA = "initial_data"
YTDLP_CONVERT = "ytdlp_convert"
ITEM_DELETE = "item_delete"
ITEM_CANCEL = "item_cancel"
STATUS = "status"
CLI_CLOSE = "cli_close"
CLI_OUTPUT = "cli_output"
UPDATE = "update"
TEST = "test"
UPDATED = "updated"
ADD_URL = "add_url"
CLI_POST = "cli_post"
PAUSED = "paused"
TEST = "test"
TASKS_ADD = "task_add"
TASK_DISPATCHED = "task_dispatched"
TASK_FINISHED = "task_finished"
TASK_ERROR = "task_error"
@dataclass(kw_only=True)
class Event:
id: str
data: dict
class EventsSubscriber(metaclass=Singleton):
"""
This class is used to subscribe to and emit events to the registered listeners.
"""
_instance = None
"""the instance of the EventsSubscriber"""
_listeners: dict[str, list[str, Awaitable]] = {}
"""The listeners for the events."""
def __init__(self):
EventsSubscriber._instance = self
@staticmethod
def get_instance() -> "EventsSubscriber":
"""
Get the instance of the EventsSubscriber.
Returns:
EventsSubscriber: The instance of the EventsSubscriber
"""
if not EventsSubscriber._instance:
EventsSubscriber._instance = EventsSubscriber()
return EventsSubscriber._instance
def subscribe(self, event: str | list | tuple, id: str, callback: Awaitable) -> "EventsSubscriber":
"""
Subscribe to an event.
Args:
event (str): The event to subscribe to.
id (str|None): The id of the subscriber, if None a random uuid will be generated.
callback (Awaitable): The function to call. Must be a coroutine.
Returns:
EventsSubscriber: The instance of the EventsSubscriber
"""
if isinstance(event, str):
event = [event]
for e in event:
if e not in self._listeners:
self._listeners[e] = {}
self._listeners[e][id] = callback
return self
def unsubscribe(self, event: str | list | tuple, id: str) -> "EventsSubscriber":
"""
Unsubscribe from an event.
Args:
event (str): The event to unsubscribe from.
id (str): The id of the subscriber.
Returns:
EventsSubscriber: The instance of the EventsSubscriber
"""
if isinstance(event, str):
event = [event]
for e in event:
if e in self._listeners and id in self._listeners[e]:
del self._listeners[e][id]
return self
def emit_sync(self, event: str, *args, **kwargs):
"""
Emit an event synchronously.
Args:
event (str): The event to emit.
*args: The arguments to pass to the event.
**kwargs: The keyword arguments to pass to the event.
Returns:
list: The results are the return values of the coroutines. If the coroutine raises an exception,
the exception is caught and logged. If event does not exist, an empty list is returned.
"""
if event not in self._listeners:
return []
results = []
for id, callback in self._listeners[event].items():
try:
if "data" not in kwargs or not isinstance(kwargs["data"], Event):
data = Event(id=id, data={"args": args if args else [], **kwargs})
else:
data = kwargs["data"]
results.append(asyncio.get_event_loop().run_until_complete(callback(event, data)))
except Exception as e:
LOG.error(f"Failed to emit event '{event}' to '{id}'. Error message '{str(e)}'.")
LOG.exception(e)
return results
def emit(self, event: str, *args, **kwargs):
"""
Emit an event.
Args:
event (str): The event to emit.
*args: The arguments to pass to the event.
**kwargs: The keyword arguments to pass to the event.
Returns:
Awaitable: The task that was created to run the event.
"""
if event not in self._listeners:
return
tasks = []
for id, callback in self._listeners[event].items():
try:
if args and isinstance(args[0], Event):
data = args[0]
elif "data" in kwargs and isinstance(kwargs["data"], Event):
data = kwargs["data"]
else:
data = Event(id=id, data={"args": args if args else [], **kwargs})
tasks.append(asyncio.create_task(callback(event, data)))
except Exception as e:
LOG.error(f"Failed to emit event '{event}' to '{id}'. Error message '{str(e)}'.")
LOG.exception(e)
return asyncio.gather(*tasks)

657
app/library/HttpAPI.py
View file

@ -1,3 +1,4 @@
import asyncio
import base64 import base64
import functools import functools
import hmac import hmac
@ -6,9 +7,10 @@ import logging
import os import os
import random import random
import time import time
import uuid
from datetime import datetime from datetime import datetime
from pathlib import Path from pathlib import Path
import uuid from typing import Awaitable
import httpx import httpx
import magic import magic
@ -21,13 +23,15 @@ from .config import Config
from .DownloadQueue import DownloadQueue from .DownloadQueue import DownloadQueue
from .Emitter import Emitter from .Emitter import Emitter
from .encoder import Encoder from .encoder import Encoder
from .EventsSubscriber import Events
from .ffprobe import ffprobe from .ffprobe import ffprobe
from .M3u8 import M3u8 from .M3u8 import M3u8
from .Notifications import Notification, NotificationEvents
from .Playlist import Playlist from .Playlist import Playlist
from .Segments import Segments from .Segments import Segments
from .Subtitle import Subtitle from .Subtitle import Subtitle
from .Utils import StreamingError, arg_converter, calcDownloadPath, getVideoInfo, validate_url, validateUUID from .Tasks import Task, Tasks
from .Notifications import Notification from .Utils import StreamingError, arg_converter, calcDownloadPath, getVideoInfo, validate_url, validate_uuid
LOG = logging.getLogger("http_api") LOG = logging.getLogger("http_api")
MIME = magic.Magic(mime=True) MIME = magic.Magic(mime=True)
@ -43,23 +47,34 @@ class HttpAPI(common):
".ico": "image/x-icon", ".ico": "image/x-icon",
} }
def __init__(self, queue: DownloadQueue, emitter: Emitter, encoder: Encoder, load_tasks: callable): def __init__(
super().__init__(queue=queue, encoder=encoder) self,
queue: DownloadQueue | None = None,
emitter: Emitter | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
):
self.queue = queue or DownloadQueue.get_instance()
self.emitter = emitter or Emitter.get_instance()
self.encoder = encoder or Encoder()
self.config = config or Config.get_instance()
self.rootPath = str(Path(__file__).parent.parent.parent) self.rootPath = str(Path(__file__).parent.parent.parent)
self.config = Config.get_instance()
self.routes = web.RouteTableDef() self.routes = web.RouteTableDef()
self.encoder = encoder
self.emitter = emitter
self.queue = queue
self.cache = Cache() self.cache = Cache()
self.load_tasks = load_tasks
def route(method: str, path: str): super().__init__(queue=self.queue, encoder=self.encoder, config=self.config)
def route(method: str, path: str) -> Awaitable:
""" """
Decorator to mark a method as an HTTP route handler. Decorator to mark a method as an HTTP route handler.
Args:
method (str): The HTTP method.
path (str): The path to the route.
Returns:
Awaitable: The decorated function.
""" """
def decorator(func): def decorator(func):
@ -73,11 +88,49 @@ class HttpAPI(common):
return decorator return decorator
def attach(self, app: web.Application) -> "HttpAPI":
"""
Attach the routes to the application.
Args:
app (web.Application): The application to attach the routes to.
Returns:
HttpAPI: The instance of the HttpAPI.
"""
if self.config.auth_username and self.config.auth_password:
app.middlewares.append(HttpAPI.basic_auth(self.config.auth_username, self.config.auth_password))
self.add_routes(app)
async def on_prepare(request: Request, response: Response):
if "Server" in response.headers:
del response.headers["Server"]
if "Origin" in request.headers:
response.headers["Access-Control-Allow-Origin"] = request.headers["Origin"]
response.headers["Access-Control-Allow-Headers"] = "Content-Type, Authorization"
response.headers["Access-Control-Allow-Methods"] = "GET, PATCH, PUT, POST, DELETE"
try:
app.on_response_prepare.append(on_prepare)
except Exception as e:
LOG.exception(e)
return self
async def staticFile(self, req: Request) -> Response: async def staticFile(self, req: Request) -> Response:
""" """
Preload static files from the ui/exported folder. Preload static files from the ui/exported folder.
Args:
req (Request): The request object.
Returns:
Response: The response object.
""" """
path = req.path path = req.path
if req.path not in self.staticHolder: if req.path not in self.staticHolder:
return web.json_response({"error": "File not found.", "file": path}, status=web.HTTPNotFound.status_code) return web.json_response({"error": "File not found.", "file": path}, status=web.HTTPNotFound.status_code)
@ -94,10 +147,17 @@ class HttpAPI(common):
status=web.HTTPOk.status_code, status=web.HTTPOk.status_code,
) )
def preloadStatic(self, app: web.Application): def preloadStatic(self, app: web.Application) -> "HttpAPI":
""" """
Preload static files from the ui/exported folder. Preload static files from the ui/exported folder.
Args:
app (web.Application): The application to attach the routes to.
Returns:
HttpAPI: The instance of the HttpAPI.
""" """
staticDir = os.path.join(self.rootPath, "ui", "exported") staticDir = os.path.join(self.rootPath, "ui", "exported")
if not os.path.exists(staticDir): if not os.path.exists(staticDir):
raise ValueError(f"Could not find the frontend UI static assets. '{staticDir}'.") raise ValueError(f"Could not find the frontend UI static assets. '{staticDir}'.")
@ -130,22 +190,28 @@ class HttpAPI(common):
preloaded += 2 preloaded += 2
if preloaded < 1: if preloaded < 1:
message = f"Could not find the frontend UI static assets. '{staticDir}'." message = f"Failed to find any static files in '{staticDir}'."
if self.config.ignore_ui: if self.config.ignore_ui:
LOG.warning(message) LOG.warning(message)
return return self
raise ValueError(message) raise ValueError(message)
LOG.info(f"Preloaded '{preloaded}' static files.") LOG.info(f"Preloaded '{preloaded}' static files.")
def attach(self, app: web.Application): return self
if self.config.auth_username and self.config.auth_password:
app.middlewares.append(HttpAPI.basic_auth(self.config.auth_username, self.config.auth_password))
self.add_routes(app) def add_routes(self, app: web.Application) -> "HttpAPI":
"""
Add the routes to the application.
Args:
app (web.Application): The application to attach the routes to.
Returns:
HttpAPI: The instance of the HttpAPI.
"""
def add_routes(self, app: web.Application):
for attr_name in dir(self): for attr_name in dir(self):
method = getattr(self, attr_name) method = getattr(self, attr_name)
if hasattr(method, "_http_method") and hasattr(method, "_http_path"): if hasattr(method, "_http_method") and hasattr(method, "_http_path"):
@ -155,30 +221,33 @@ class HttpAPI(common):
self.routes.route(method._http_method, f"/{http_path}")(method) self.routes.route(method._http_method, f"/{http_path}")(method)
async def on_prepare(request: Request, response: Response):
if "Server" in response.headers:
del response.headers["Server"]
if "Origin" in request.headers:
response.headers["Access-Control-Allow-Origin"] = request.headers["Origin"]
response.headers["Access-Control-Allow-Headers"] = "Content-Type, Authorization"
response.headers["Access-Control-Allow-Methods"] = "GET, PATCH, PUT, POST, DELETE"
self.routes.static("/api/download/", self.config.download_path) self.routes.static("/api/download/", self.config.download_path)
self.preloadStatic(app) self.preloadStatic(app)
try: try:
app.add_routes(self.routes) app.add_routes(self.routes)
app.on_response_prepare.append(on_prepare)
except ValueError as e: except ValueError as e:
if "ui/exported" in str(e): if "ui/exported" in str(e):
raise RuntimeError(f"Could not find the frontend UI static assets. '{e}'.") from e raise RuntimeError(f"Could not find the frontend UI static assets. '{e}'.") from e
raise e raise e
def basic_auth(username: str, password: str): def basic_auth(username: str, password: str) -> Awaitable:
"""
Middleware to handle basic authentication.
Args:
username (str): The username.
password (str): The password.
Returns:
Awaitable: The middleware handler.
"""
@web.middleware @web.middleware
async def middleware_handler(request: Request, handler: RequestHandler) -> Response: async def middleware_handler(request: Request, handler: RequestHandler) -> Response:
auth_header = request.headers.get("Authorization") auth_header = request.headers.get("Authorization")
if auth_header is None and request.query.get("apikey") is not None:
auth_header = f"Basic {request.query.get('apikey')}"
if auth_header is None: if auth_header is None:
return web.json_response( return web.json_response(
@ -214,20 +283,47 @@ class HttpAPI(common):
@route("OPTIONS", "/{path:.*}") @route("OPTIONS", "/{path:.*}")
async def add_coors(self, _: Request) -> Response: async def add_coors(self, _: Request) -> Response:
"""
Add CORS headers to the response.
Args:
_: The request object.
Returns:
Response: The response object.
"""
return web.json_response(data={"status": "ok"}, status=web.HTTPOk.status_code) return web.json_response(data={"status": "ok"}, status=web.HTTPOk.status_code)
@route("GET", "api/ping") @route("GET", "api/ping")
async def ping(self, _) -> Response: async def ping(self, _: Request) -> Response:
"""
Ping the server.
Args:
_: The request object.
Returns:
Response: The response object.
"""
await self.queue.test() await self.queue.test()
return web.json_response(data={"status": "pong"}, status=web.HTTPOk.status_code) return web.json_response(data={"status": "pong"}, status=web.HTTPOk.status_code)
@route("POST", "api/yt-dlp/convert") @route("POST", "api/yt-dlp/convert")
async def yt_dlp_convert(self, request: Request) -> Response: async def yt_dlp_convert(self, request: Request) -> Response:
"""
Convert the yt-dlp args to a dict.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
post = await request.json() post = await request.json()
args: str | None = post.get("args") args: str | None = post.get("args")
if not args: if not args:
return web.json_response(data={"error": "args is required."}, status=web.HTTPBadRequest.status_code) return web.json_response(data={"error": "args param is required."}, status=web.HTTPBadRequest.status_code)
try: try:
return web.json_response(data=arg_converter(args), status=web.HTTPOk.status_code) return web.json_response(data=arg_converter(args), status=web.HTTPOk.status_code)
@ -239,121 +335,25 @@ class HttpAPI(common):
data={"error": f"Failed to convert args. '{err}'."}, status=web.HTTPBadRequest.status_code data={"error": f"Failed to convert args. '{err}'."}, status=web.HTTPBadRequest.status_code
) )
@route("POST", "api/add") @route("GET", "api/yt-dlp/url/info")
async def add_url(self, request: Request) -> Response:
post = await request.json()
url: str = post.get("url")
if not url:
return web.json_response(data={"error": "url is required."}, status=web.HTTPBadRequest.status_code)
preset: str = str(post.get("preset", self.config.default_preset))
folder: str = str(post.get("folder")) if post.get("folder") else ""
ytdlp_cookies: str = str(post.get("ytdlp_cookies")) if post.get("ytdlp_cookies") else ""
output_template: str = str(post.get("output_template")) if post.get("output_template") else ""
ytdlp_config = post.get("ytdlp_config")
if isinstance(ytdlp_config, str) and ytdlp_config:
try:
ytdlp_config = json.loads(ytdlp_config)
except Exception as e:
LOG.error(f"Failed to parse json yt-dlp config for '{url}'. {str(e)}")
return web.json_response(
data={"error": f"Failed to parse json yt-dlp config. {str(e)}"},
status=web.HTTPBadRequest.status_code,
)
status = await self.add(
url=url,
preset=preset,
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config if isinstance(ytdlp_config, dict) else {},
output_template=output_template,
)
return web.json_response(data=status, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("GET", "api/tasks")
async def tasks(self, _: Request) -> Response:
tasks_file: str = os.path.join(self.config.config_path, "tasks.json")
if os.path.exists(tasks_file):
try:
with open(tasks_file, "r") as f:
tasks = json.load(f)
except Exception as e:
LOG.exception(e)
return web.json_response(
{"error": "Failed to load tasks."}, status=web.HTTPInternalServerError.status_code
)
else:
tasks = []
return web.json_response(data=tasks, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("PUT", "api/tasks")
async def add_tasks(self, request: Request) -> Response:
tasks_file: str = os.path.join(self.config.config_path, "tasks.json")
post = await request.json()
if not isinstance(post, list):
return web.json_response(
{"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code,
)
tasks: list = []
for item in post:
if not isinstance(item, dict):
return web.json_response(
{"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code,
)
if not item.get("url"):
return web.json_response(
{"error": "url is required.", "data": post}, status=web.HTTPBadRequest.status_code
)
if not item.get("id", None):
item["id"] = str(uuid.uuid4())
if not item.get("timer", None):
item["timer"] = f"{random.randint(1,59)} */1 * * *"
tasks.append(item)
try:
with open(tasks_file, "w") as f:
json.dump(tasks, f, indent=4)
except Exception as e:
LOG.exception(e)
return web.json_response({"error": "Failed to save tasks."}, status=web.HTTPInternalServerError.status_code)
try:
self.load_tasks()
except Exception as e:
LOG.exception(e)
return web.json_response(
{"error": "Failed to reload tasks.", "message": str(e)},
status=web.HTTPInternalServerError.status_code,
)
return web.json_response(data=tasks, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("GET", "api/url/info")
async def get_info(self, request: Request) -> Response: async def get_info(self, request: Request) -> Response:
"""
Get the video info.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
url = request.query.get("url") url = request.query.get("url")
if not url: if not url:
return web.json_response(data={"error": "URL is required."}, status=web.HTTPForbidden.status_code) return web.json_response(data={"error": "URL is required."}, status=web.HTTPBadRequest.status_code)
try: try:
validate_url(url) validate_url(url)
except ValueError as e: except ValueError as e:
return web.json_response(data={"error": str(e)}, status=web.HTTPForbidden.status_code) return web.json_response(data={"error": str(e)}, status=web.HTTPBadRequest.status_code)
try: try:
key = self.cache.hash(url) key = self.cache.hash(url)
@ -393,52 +393,165 @@ class HttpAPI(common):
status=web.HTTPInternalServerError.status_code, status=web.HTTPInternalServerError.status_code,
) )
@route("POST", "api/add_batch") @route("GET", "api/history/add")
async def add_batch(self, request: Request) -> Response: async def quick_add(self, request: Request) -> Response:
status = {} """
Add a URL to the download queue.
post = await request.json() Args:
if not isinstance(post, list): request (Request): The request object.
Returns:
Response: The response object
"""
url: str | None = request.query.get("url")
if not url:
return web.json_response(data={"error": "url param is required."}, status=web.HTTPBadRequest.status_code)
try:
status = await self.add(**self.format_item({"url": url}))
except ValueError as e:
return web.json_response(data={"error": str(e)}, status=web.HTTPBadRequest.status_code)
return web.json_response(data=status, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("POST", "api/history")
async def history_item_add(self, request: Request) -> Response:
"""
Add a URL to the download queue.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
data = await request.json()
if isinstance(data, dict):
data = [data]
for item in data:
try:
self.format_item(item)
except ValueError as e:
return web.json_response(data={"error": str(e), "data": item}, status=web.HTTPBadRequest.status_code)
return web.json_response(
data=await asyncio.wait_for(
fut=asyncio.gather(*[self.add(**self.format_item(item)) for item in data]),
timeout=None,
),
status=web.HTTPOk.status_code,
dumps=self.encoder.encode,
)
@route("GET", "api/tasks")
async def tasks(self, _: Request) -> Response:
"""
Get the tasks.
Args:
_: The request object.
Returns:
Response: The response object.
"""
return web.json_response(
data=Tasks.get_instance().getTasks(), status=web.HTTPOk.status_code, dumps=self.encoder.encode
)
@route("PUT", "api/tasks")
async def tasks_add(self, request: Request) -> Response:
"""
Add tasks to the queue.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
data = await request.json()
if not isinstance(data, list):
return web.json_response( return web.json_response(
data={"error": "Invalid request body expecting list with dicts."}, {"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code, status=web.HTTPBadRequest.status_code,
) )
for item in post: tasks: list = []
ins = Tasks.get_instance()
for item in data:
if not isinstance(item, dict): if not isinstance(item, dict):
return web.json_response( return web.json_response(
data={"error": "Invalid request body expecting list with dicts."}, {"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code, status=web.HTTPBadRequest.status_code,
) )
if not item.get("url"): if not item.get("url"):
return web.json_response( return web.json_response(
data={"error": "url is required.", "data": post}, status=web.HTTPBadRequest.status_code {"error": "url is required.", "data": item}, status=web.HTTPBadRequest.status_code
) )
for item in post: if not item.get("id", None) or not validate_uuid(item.get("id"), version=4):
status[item.get("url")] = await self.add( item["id"] = str(uuid.uuid4())
url=item.get("url"),
preset=item.get("preset", "default"), if not item.get("timer", None) or str(item.get("timer")).strip() == "":
folder=item.get("folder"), item["timer"] = f"{random.randint(1,59)} */1 * * *"
ytdlp_cookies=item.get("ytdlp_cookies"),
ytdlp_config=item.get("ytdlp_config"), if not item.get("cookies", None):
output_template=item.get("output_template"), item["cookies"] = ""
if not item.get("config", None) or str(item.get("config")).strip() == "":
item["config"] = {}
if not item.get("template", None):
item["template"] = str()
try:
ins.validate(item)
except ValueError as e:
return web.json_response(
{"error": f"Failed to validate task '{item.get('name')}'. '{str(e)}'"},
status=web.HTTPBadRequest.status_code,
)
tasks.append(Task(**item))
try:
tasks = ins.save(tasks=tasks).load().getTasks()
except Exception as e:
LOG.exception(e)
return web.json_response(
{"error": "Failed to save tasks.", "message": str(e)},
status=web.HTTPInternalServerError.status_code,
) )
return web.json_response(data=status, status=web.HTTPOk.status_code, dumps=self.encoder.encode) return web.json_response(data=tasks, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("DELETE", "api/delete") @route("DELETE", "api/history")
async def delete(self, request: Request) -> Response: async def history_delete(self, request: Request) -> Response:
post = await request.json() """
ids = post.get("ids") Delete an item from the queue.
where = post.get("where")
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
data = await request.json()
ids = data.get("ids")
where = data.get("where")
if not ids or where not in ["queue", "done"]: if not ids or where not in ["queue", "done"]:
return web.json_response( return web.json_response(
data={"error": "ids and where are required."}, status=web.HTTPBadRequest.status_code data={"error": "ids and where are required."}, status=web.HTTPBadRequest.status_code
) )
remove_file: bool = bool(post.get("remove_file", True)) remove_file: bool = bool(data.get("remove_file", True))
return web.json_response( return web.json_response(
data=await (self.queue.cancel(ids) if where == "queue" else self.queue.clear(ids, remove_file=remove_file)), data=await (self.queue.cancel(ids) if where == "queue" else self.queue.clear(ids, remove_file=remove_file)),
@ -447,7 +560,16 @@ class HttpAPI(common):
) )
@route("POST", "api/history/{id}") @route("POST", "api/history/{id}")
async def update_item(self, request: Request) -> Response: async def history_item_update(self, request: Request) -> Response:
"""
Update an item in the history.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
id: str = request.match_info.get("id") id: str = request.match_info.get("id")
if not id: if not id:
return web.json_response(data={"error": "id is required."}, status=web.HTTPBadRequest.status_code) return web.json_response(data={"error": "id is required."}, status=web.HTTPBadRequest.status_code)
@ -475,7 +597,7 @@ class HttpAPI(common):
if updated: if updated:
self.queue.done.put(item) self.queue.done.put(item)
await self.emitter.emit("update", item.info) await self.emitter.emit(Events.UPDATE, item.info)
return web.json_response( return web.json_response(
data=item.info, data=item.info,
@ -485,6 +607,15 @@ class HttpAPI(common):
@route("GET", "api/history") @route("GET", "api/history")
async def history(self, _: Request) -> Response: async def history(self, _: Request) -> Response:
"""
Get the history.
Args:
_: The request object.
Returns:
Response: The response object.
"""
data: dict = {"queue": [], "history": []} data: dict = {"queue": [], "history": []}
for _, v in self.queue.queue.saved_items(): for _, v in self.queue.queue.saved_items():
@ -495,7 +626,16 @@ class HttpAPI(common):
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode) return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("GET", "api/workers") @route("GET", "api/workers")
async def workers(self, _) -> Response: async def pool_list(self, _) -> Response:
"""
Get the workers status.
Args:
_: The request object.
Returns:
Response: The response object.
"""
if self.queue.pool is None: if self.queue.pool is None:
return web.json_response({"error": "Worker pool not initialized."}, status=web.HTTPNotFound.status_code) return web.json_response({"error": "Worker pool not initialized."}, status=web.HTTPNotFound.status_code)
@ -523,7 +663,16 @@ class HttpAPI(common):
) )
@route("POST", "api/workers") @route("POST", "api/workers")
async def restart_pool(self, _) -> Response: async def pool_restart(self, _) -> Response:
"""
Restart the workers pool.
Args:
_: The request object.
Returns:
Response: The response object.
"""
if self.queue.pool is None: if self.queue.pool is None:
return web.json_response({"error": "Worker pool not initialized."}, status=web.HTTPNotFound.status_code) return web.json_response({"error": "Worker pool not initialized."}, status=web.HTTPNotFound.status_code)
@ -532,7 +681,16 @@ class HttpAPI(common):
return web.json_response({"message": "Workers pool being restarted."}, status=web.HTTPOk.status_code) return web.json_response({"message": "Workers pool being restarted."}, status=web.HTTPOk.status_code)
@route("PATCH", "api/workers/{id}") @route("PATCH", "api/workers/{id}")
async def restart_worker(self, request: Request) -> Response: async def worker_restart(self, request: Request) -> Response:
"""
Restart a worker.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
id: str = request.match_info.get("id") id: str = request.match_info.get("id")
if not id: if not id:
return web.json_response({"error": "worker id is required."}, status=web.HTTPBadRequest.status_code) return web.json_response({"error": "worker id is required."}, status=web.HTTPBadRequest.status_code)
@ -544,8 +702,17 @@ class HttpAPI(common):
return web.json_response({"status": "restarted" if status else "in_error_state"}, status=web.HTTPOk.status_code) return web.json_response({"status": "restarted" if status else "in_error_state"}, status=web.HTTPOk.status_code)
@route("delete", "api/workers/{id}") @route("DELETE", "api/workers/{id}")
async def stop_worker(self, request: Request) -> Response: async def worker_stop(self, request: Request) -> Response:
"""
Stop a worker.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
id: str = request.match_info.get("id") id: str = request.match_info.get("id")
if not id: if not id:
raise web.HTTPBadRequest(text="worker id is required.") raise web.HTTPBadRequest(text="worker id is required.")
@ -559,6 +726,15 @@ class HttpAPI(common):
@route("GET", "api/player/playlist/{file:.*}.m3u8") @route("GET", "api/player/playlist/{file:.*}.m3u8")
async def playlist(self, request: Request) -> Response: async def playlist(self, request: Request) -> Response:
"""
Get the playlist.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file") file: str = request.match_info.get("file")
if not file: if not file:
@ -583,6 +759,15 @@ class HttpAPI(common):
@route("GET", "api/player/m3u8/{mode}/{file:.*}.m3u8") @route("GET", "api/player/m3u8/{mode}/{file:.*}.m3u8")
async def m3u8(self, request: Request) -> Response: async def m3u8(self, request: Request) -> Response:
"""
Get the m3u8 file.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file") file: str = request.match_info.get("file")
mode: str = request.match_info.get("mode") mode: str = request.match_info.get("mode")
@ -624,6 +809,15 @@ class HttpAPI(common):
@route("GET", r"api/player/segments/{segment:\d+}/{file:.*}.ts") @route("GET", r"api/player/segments/{segment:\d+}/{file:.*}.ts")
async def segments(self, request: Request) -> Response: async def segments(self, request: Request) -> Response:
"""
Get the segments.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file") file: str = request.match_info.get("file")
segment: int = request.match_info.get("segment") segment: int = request.match_info.get("segment")
sd: int = request.query.get("sd") sd: int = request.query.get("sd")
@ -673,6 +867,15 @@ class HttpAPI(common):
@route("GET", "api/player/subtitle/{file:.*}.vtt") @route("GET", "api/player/subtitle/{file:.*}.vtt")
async def subtitles(self, request: Request) -> Response: async def subtitles(self, request: Request) -> Response:
"""
Get the subtitles.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file") file: str = request.match_info.get("file")
file_path: str = os.path.normpath(os.path.join(self.config.download_path, file)) file_path: str = os.path.normpath(os.path.join(self.config.download_path, file))
if not file_path.startswith(self.config.download_path): if not file_path.startswith(self.config.download_path):
@ -707,7 +910,16 @@ class HttpAPI(common):
) )
@route("GET", "/") @route("GET", "/")
async def index(self, _) -> Response: async def index(self, _: Request) -> Response:
"""
Get the index file.
Args:
_: The request object.
Returns:
Response: The response object.
"""
if "/index.html" not in self.staticHolder: if "/index.html" not in self.staticHolder:
LOG.error("Static frontend files not found.") LOG.error("Static frontend files not found.")
return web.json_response( return web.json_response(
@ -724,6 +936,15 @@ class HttpAPI(common):
@route("GET", "api/thumbnail") @route("GET", "api/thumbnail")
async def get_thumbnail(self, request: Request) -> Response: async def get_thumbnail(self, request: Request) -> Response:
"""
Get the thumbnail.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
url = request.query.get("url") url = request.query.get("url")
if not url: if not url:
return web.json_response(data={"error": "URL is required."}, status=web.HTTPForbidden.status_code) return web.json_response(data={"error": "URL is required."}, status=web.HTTPForbidden.status_code)
@ -759,13 +980,21 @@ class HttpAPI(common):
) )
except Exception as e: except Exception as e:
LOG.error(f"Error fetching thumbnail from '{url}'. '{e}'.") LOG.error(f"Error fetching thumbnail from '{url}'. '{e}'.")
LOG.exception(e)
return web.json_response( return web.json_response(
data={"error": "failed to retrieve the thumbnail."}, status=web.HTTPInternalServerError.status_code data={"error": "failed to retrieve the thumbnail."}, status=web.HTTPInternalServerError.status_code
) )
@route("GET", "api/ffprobe/{file:.*}") @route("GET", "api/file/ffprobe/{file:.*}")
async def get_ffprobe(self, request: Request) -> Response: async def get_ffprobe(self, request: Request) -> Response:
"""
Get the ffprobe data.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file") file: str = request.match_info.get("file")
try: try:
realFile: str = calcDownloadPath(basePath=self.config.download_path, folder=file, createPath=False) realFile: str = calcDownloadPath(basePath=self.config.download_path, folder=file, createPath=False)
@ -782,6 +1011,15 @@ class HttpAPI(common):
@route("GET", "api/youtube/auth") @route("GET", "api/youtube/auth")
async def is_authenticated(self, request: Request) -> Response: async def is_authenticated(self, request: Request) -> Response:
"""
Check if the user yt-dlp cookie is valid & authenticated.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
cookie_file = self.config.ytdl_options.get("cookiefile", None) cookie_file = self.config.ytdl_options.get("cookiefile", None)
if not cookie_file: if not cookie_file:
return web.json_response(data={"message": "No cookie file provided."}, status=web.HTTPForbidden.status_code) return web.json_response(data={"message": "No cookie file provided."}, status=web.HTTPForbidden.status_code)
@ -834,22 +1072,35 @@ class HttpAPI(common):
@route("GET", "api/notifications") @route("GET", "api/notifications")
async def notifications(self, _: Request) -> Response: async def notifications(self, _: Request) -> Response:
data = { """
"notifications": Notification.get_instance().getTargets(), Get the notification targets.
"allowedTypes": Notification.allowed_events,
}
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode) Args:
_: The request object.
@route("POST", "api/notifications/test") Returns:
async def test_notifications(self, _: Request) -> Response: Response: The response object.
data = {"type": "test", "message": "This is a test notification."} """
await self.emitter.emit("test", data) return web.json_response(
data={
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode) "notifications": Notification.get_instance().getTargets(),
"allowedTypes": list(NotificationEvents.getEvents().values()),
},
status=web.HTTPOk.status_code,
dumps=self.encoder.encode,
)
@route("PUT", "api/notifications") @route("PUT", "api/notifications")
async def add_notifications(self, request: Request) -> Response: async def notification_add(self, request: Request) -> Response:
"""
Add notification targets.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
post = await request.json() post = await request.json()
if not isinstance(post, list): if not isinstance(post, list):
return web.json_response( return web.json_response(
@ -859,6 +1110,7 @@ class HttpAPI(common):
targets: list = [] targets: list = []
ins = Notification.get_instance()
for item in post: for item in post:
if not isinstance(item, dict): if not isinstance(item, dict):
return web.json_response( return web.json_response(
@ -866,7 +1118,7 @@ class HttpAPI(common):
status=web.HTTPBadRequest.status_code, status=web.HTTPBadRequest.status_code,
) )
if not item.get("id", None) or validateUUID(item.get("id"), version=4): if not item.get("id", None) or validate_uuid(item.get("id"), version=4):
item["id"] = str(uuid.uuid4()) item["id"] = str(uuid.uuid4())
try: try:
@ -877,13 +1129,11 @@ class HttpAPI(common):
status=web.HTTPBadRequest.status_code, status=web.HTTPBadRequest.status_code,
) )
targets.append(item) targets.append(ins.makeTarget(item))
ins = Notification.get_instance()
try: try:
if len(targets) < 1: if len(targets) < 1:
ins.clearTargets() ins.clear()
ins.save(targets=targets) ins.save(targets=targets)
ins.load() ins.load()
@ -891,9 +1141,22 @@ class HttpAPI(common):
LOG.exception(e) LOG.exception(e)
return web.json_response({"error": "Failed to save tasks."}, status=web.HTTPInternalServerError.status_code) return web.json_response({"error": "Failed to save tasks."}, status=web.HTTPInternalServerError.status_code)
data = { data = {"notifications": targets, "allowedTypes": list(NotificationEvents.getEvents().values())}
"notifications": targets,
"allowedTypes": Notification.allowed_events, return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
}
@route("POST", "api/notifications/test")
async def notification_test(self, _: Request) -> Response:
"""
Test the notification.
Args:
_: The request object.
Returns:
Response: The response object.
"""
data = {"type": "test", "message": "This is a test notification."}
await self.emitter.emit(Events.TEST, data)
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode) return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)

108
app/library/HttpSocket.py
View file

@ -1,7 +1,6 @@
import asyncio import asyncio
import errno import errno
import functools import functools
import json
import logging import logging
import os import os
import pty import pty
@ -11,6 +10,7 @@ from datetime import datetime
import socketio import socketio
from aiohttp import web from aiohttp import web
from typing import Any
from .common import common from .common import common
from .config import Config from .config import Config
@ -18,6 +18,7 @@ from .DownloadQueue import DownloadQueue
from .Emitter import Emitter from .Emitter import Emitter
from .encoder import Encoder from .encoder import Encoder
from .Utils import isDownloaded, arg_converter from .Utils import isDownloaded, arg_converter
from .EventsSubscriber import EventsSubscriber, Events, Event
LOG = logging.getLogger("socket_api") LOG = logging.getLogger("socket_api")
@ -29,17 +30,29 @@ class HttpSocket(common):
config: Config config: Config
sio: socketio.AsyncServer sio: socketio.AsyncServer
queue: DownloadQueue
emitter: Emitter
def __init__(self, queue: DownloadQueue, emitter: Emitter, encoder: Encoder): def __init__(
super().__init__(queue=queue, encoder=encoder) self,
queue: DownloadQueue | None = None,
emitter: Emitter | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
sio: socketio.AsyncServer | None = None,
):
self.config = config or Config.get_instance()
self.queue = queue or DownloadQueue.get_instance()
self.emitter = emitter or Emitter.get_instance()
self.sio = sio or socketio.AsyncServer(cors_allowed_origins="*")
encoder = encoder or Encoder()
self.sio = socketio.AsyncServer(cors_allowed_origins="*") def emit(event: str, data: Any, **kwargs):
emitter.add_emitter(lambda event, data, **kwargs: self.sio.emit(event, encoder.encode(data), **kwargs)) return self.sio.emit(event=event, data=encoder.encode(data), **kwargs)
self.config = Config.get_instance() self.emitter.add_emitter([emit], local=False)
self.queue = queue super().__init__(queue=queue, encoder=encoder, config=config)
self.emitter = emitter
def ws_event(func): # type: ignore def ws_event(func): # type: ignore
""" """
@ -61,10 +74,18 @@ class HttpSocket(common):
if hasattr(method, "_ws_event") and self.sio: if hasattr(method, "_ws_event") and self.sio:
self.sio.on(method._ws_event)(method) # type: ignore self.sio.on(method._ws_event)(method) # type: ignore
# self.sio.on("*", es.emit)
async def handle_event(_: str, data: Event):
LOG.debug(f"Event received. '{data}'")
await self.add(**data.data)
EventsSubscriber.get_instance().subscribe(Events.ADD_URL, "socket_add_url", handle_event)
@ws_event # type: ignore @ws_event # type: ignore
async def cli_post(self, sid: str, data): async def cli_post(self, sid: str, data):
if not data: if not data:
await self.emitter.emit("cli_close", {"exitcode": 0}, to=sid) await self.emitter.emit(Events.CLI_CLOSE, {"exitcode": 0}, to=sid)
return return
try: try:
@ -100,7 +121,7 @@ class HttpSocket(common):
except Exception as e: except Exception as e:
LOG.error(f"Error closing PTY. '{str(e)}'.") LOG.error(f"Error closing PTY. '{str(e)}'.")
async def read_pty(): async def read_pty(sid: str):
loop = asyncio.get_running_loop() loop = asyncio.get_running_loop()
buffer = b"" buffer = b""
while True: while True:
@ -117,14 +138,18 @@ class HttpSocket(common):
if buffer: if buffer:
# Emit any remaining partial line # Emit any remaining partial line
await self.emitter.emit( await self.emitter.emit(
"cli_output", {"type": "stdout", "line": buffer.decode("utf-8", errors="replace")} Events.CLI_OUTPUT,
{"type": "stdout", "line": buffer.decode("utf-8", errors="replace")},
to=sid,
) )
break break
buffer += chunk buffer += chunk
*lines, buffer = buffer.split(b"\n") *lines, buffer = buffer.split(b"\n")
for line in lines: for line in lines:
await self.emitter.emit( await self.emitter.emit(
"cli_output", {"type": "stdout", "line": line.decode("utf-8", errors="replace")} Events.CLI_OUTPUT,
{"type": "stdout", "line": line.decode("utf-8", errors="replace")},
to=sid,
) )
try: try:
os.close(master_fd) os.close(master_fd)
@ -132,7 +157,7 @@ class HttpSocket(common):
LOG.error(f"Error closing PTY. '{str(e)}'.") LOG.error(f"Error closing PTY. '{str(e)}'.")
# Start reading output from PTY # Start reading output from PTY
read_task = asyncio.create_task(read_pty()) read_task = asyncio.create_task(read_pty(sid=sid))
# Wait until process finishes # Wait until process finishes
returncode = await proc.wait() returncode = await proc.wait()
@ -140,51 +165,29 @@ class HttpSocket(common):
# Ensure reading is done # Ensure reading is done
await read_task await read_task
await self.emitter.emit("cli_close", {"exitcode": returncode}, to=sid) await self.emitter.emit(Events.CLI_CLOSE, {"exitcode": returncode}, to=sid)
except Exception as e: except Exception as e:
LOG.error(f"CLI execute exception was thrown for client '{sid}'.") LOG.error(f"CLI execute exception was thrown for client '{sid}'.")
LOG.exception(e) LOG.exception(e)
await self.emitter.emit( await self.emitter.emit(Events.CLI_OUTPUT, {"type": "stderr", "line": str(e)}, to=sid)
"cli_out1put", await self.emitter.emit(Events.CLI_CLOSE, {"exitcode": -1}, to=sid)
{
"type": "stderr",
"line": str(e),
},
to=sid,
)
await self.emitter.emit("cli_close", {"exitcode": -1}, to=sid)
@ws_event # type: ignore @ws_event # type: ignore
async def add_url(self, sid: str, data: dict): async def add_url(self, sid: str, data: dict):
url: str | None = data.get("url") url: str | None = data.get("url")
if not url: if not url:
await self.emitter.error("No URL provided.", to=sid) await self.emitter.error("No URL provided.", data={"unlock": True}, to=sid)
return return
preset: str = str(data.get("preset", self.config.default_preset)) try:
folder: str = str(data.get("folder")) if data.get("folder") else "" item = self.format_item(data)
ytdlp_cookies: str = str(data.get("ytdlp_cookies")) if data.get("ytdlp_cookies") else "" except ValueError as e:
output_template: str = str(data.get("output_template")) if data.get("output_template") else "" return web.json_response(data={"error": str(e)}, status=web.HTTPBadRequest.status_code)
ytdlp_config = data.get("ytdlp_config") status = await self.add(**item)
if isinstance(ytdlp_config, str) and ytdlp_config:
try:
ytdlp_config = json.loads(ytdlp_config)
except Exception as e:
await self.emitter.error(f"Failed to parse json yt-dlp config. {str(e)}", to=sid)
return
status = await self.add( await self.emitter.emit(event=Events.STATUS, data=status, to=sid)
url=url,
preset=preset,
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config if isinstance(ytdlp_config, dict) else {},
output_template=output_template,
)
await self.emitter.emit("status", status, to=sid)
@ws_event # type: ignore @ws_event # type: ignore
async def item_cancel(self, sid: str, id: str): async def item_cancel(self, sid: str, id: str):
@ -196,7 +199,7 @@ class HttpSocket(common):
status = await self.queue.cancel([id]) status = await self.queue.cancel([id])
status.update({"identifier": id}) status.update({"identifier": id})
await self.emitter.emit("item_cancel", status) await self.emitter.emit(event=Events.ITEM_CANCEL, data=status)
@ws_event # type: ignore @ws_event # type: ignore
async def item_delete(self, sid: str, data: dict): async def item_delete(self, sid: str, data: dict):
@ -213,7 +216,7 @@ class HttpSocket(common):
status = await self.queue.clear([id], remove_file=bool(data.get("remove_file", False))) status = await self.queue.clear([id], remove_file=bool(data.get("remove_file", False)))
status.update({"identifier": id}) status.update({"identifier": id})
await self.emitter.emit("item_delete", status) await self.emitter.emit(event=Events.ITEM_DELETE, data=status)
@ws_event # type: ignore @ws_event # type: ignore
async def archive_item(self, sid: str, data: dict): async def archive_item(self, sid: str, data: dict):
@ -256,26 +259,25 @@ class HttpSocket(common):
data = { data = {
**self.queue.get(), **self.queue.get(),
"config": self.config.frontend(), "config": self.config.frontend(),
"tasks": self.config.tasks,
"presets": self.config.presets, "presets": self.config.presets,
"paused": self.queue.isPaused(), "paused": self.queue.isPaused(),
} }
# get download folder listing # get download folder listing.
downloadPath: str = self.config.download_path downloadPath: str = self.config.download_path
data["folders"] = [name for name in os.listdir(downloadPath) if os.path.isdir(os.path.join(downloadPath, name))] data["folders"] = [name for name in os.listdir(downloadPath) if os.path.isdir(os.path.join(downloadPath, name))]
await self.emitter.emit("initial_data", data, to=sid) await self.emitter.emit(event=Events.INITIAL_DATA, data=data, to=sid)
@ws_event # type: ignore @ws_event # type: ignore
async def pause(self, sid: str, _=None): async def pause(self, sid: str, _=None):
self.queue.pause() self.queue.pause()
await self.emitter.emit("paused", {"paused": True, "at": time.time()}) await self.emitter.emit(event=Events.PAUSED, data={"paused": True, "at": time.time()})
@ws_event # type: ignore @ws_event # type: ignore
async def resume(self, sid: str, _=None): async def resume(self, sid: str, _=None):
self.queue.resume() self.queue.resume()
await self.emitter.emit("paused", {"paused": False, "at": time.time()}) await self.emitter.emit(event=Events.PAUSED, data={"paused": False, "at": time.time()})
@ws_event @ws_event
async def ytdlp_convert(self, sid: str, data: dict): async def ytdlp_convert(self, sid: str, data: dict):
@ -290,7 +292,7 @@ class HttpSocket(common):
return return
try: try:
await self.emitter.emit("ytdlp_convert", arg_converter(args), to=sid) await self.emitter.emit(event=Events.YTDLP_CONVERT, data=arg_converter(args), to=sid)
return return
except Exception as e: except Exception as e:
err = str(e).strip() err = str(e).strip()

36
app/library/ItemDTO.py
View file

@ -5,6 +5,7 @@ from dataclasses import dataclass, field
from email.utils import formatdate from email.utils import formatdate
from typing import Any from typing import Any
@dataclass(kw_only=True) @dataclass(kw_only=True)
class ItemDTO: class ItemDTO:
""" """
@ -14,22 +15,21 @@ class ItemDTO:
_id: str = field(default_factory=lambda: str(uuid.uuid4()), init=False) _id: str = field(default_factory=lambda: str(uuid.uuid4()), init=False)
error: str|None = None error: str | None = None
id: str id: str
title: str title: str
url: str url: str
quality: str | None = None quality: str | None = None
format: str | None = None format: str | None = None
thumbnail: str | None = None
preset: str = "default" preset: str = "default"
folder: str folder: str
download_dir: str | None = None download_dir: str | None = None
temp_dir: str | None = None temp_dir: str | None = None
status: str | None = None status: str | None = None
ytdlp_cookies: str | None = None cookies: str | None = None
ytdlp_config: dict = field(default_factory=dict) config: dict = field(default_factory=dict)
output_template: str | None = None template: str | None = None
output_template_chapter: str | None = None template_chapter: str | None = None
timestamp: float = time.time_ns() timestamp: float = time.time_ns()
is_live: bool | None = None is_live: bool | None = None
datetime: str = field(default_factory=lambda: str(formatdate(time.time()))) datetime: str = field(default_factory=lambda: str(formatdate(time.time())))
@ -49,19 +49,41 @@ class ItemDTO:
speed: str | None = None speed: str | None = None
eta: str | None = None eta: str | None = None
# DEPRECATED: These fields are deprecated and will be removed in the future.
thumbnail: str | None = None
ytdlp_cookies: str | None = None
ytdlp_config: dict = field(default_factory=dict)
output_template: str | None = None
output_template_chapter: str | None = None
def serialize(self) -> dict: def serialize(self) -> dict:
deprecated: tuple = ( deprecated: tuple = (
"thumbnail", "thumbnail",
"quality", "quality",
"format", "format",
"ytdlp_cookies",
"ytdlp_config",
"output_template",
"output_template_chapter",
) )
if self.thumbnail and "thumbnail" not in self.extras: if self.thumbnail and "thumbnail" not in self.extras:
self.extras["thumbnail"] = self.thumbnail self.extras["thumbnail"] = self.thumbnail
mapper: dict = {
"cookies": "ytdlp_cookies",
"config": "ytdlp_config",
"template": "output_template",
"template_chapter": "output_template_chapter",
}
for k, v in mapper.items():
if not self.get(k) and self.get(v):
self.__dict__[k] = self.get(v)
dump = self.__dict__.copy() dump = self.__dict__.copy()
for f in deprecated: for f in deprecated:
dump.pop(f) dump.pop(f, None)
return dump return dump

268
app/library/Notifications.py
View file

@ -3,38 +3,65 @@ from datetime import datetime, timezone
import json import json
import logging import logging
import os import os
from typing import List, TypedDict from typing import List, Any
import uuid
from dataclasses import dataclass, field
import httpx import httpx
from .config import Config from .config import Config
from .ItemDTO import ItemDTO from .ItemDTO import ItemDTO
from .Singleton import Singleton from .Singleton import Singleton
from .Utils import ag, validateUUID from .Utils import validate_uuid, ag
from .version import APP_VERSION from .version import APP_VERSION
from .encoder import Encoder from .encoder import Encoder
from .EventsSubscriber import Events
LOG = logging.getLogger("notifications") LOG = logging.getLogger("notifications")
class targetRequestHeader(TypedDict): @dataclass(kw_only=True)
class targetRequestHeader:
"""Request header details for a notification target.""" """Request header details for a notification target."""
key: str key: str
value: str value: str
def serialize(self) -> dict:
return {"key": self.key, "value": self.value}
class targetRequest(TypedDict): def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return self.serialize().get(key, default)
@dataclass(kw_only=True)
class targetRequest:
"""Request details for a notification target.""" """Request details for a notification target."""
type: str type: str
method: str method: str
url: str url: str
headers: list[targetRequestHeader] = [] headers: list[targetRequestHeader] = field(default_factory=list)
def serialize(self) -> dict:
return {
"type": self.type,
"method": self.method,
"url": self.url,
"headers": [h.serialize() for h in self.headers],
}
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return ag(self.serialize(), key, default)
class Target(TypedDict): @dataclass(kw_only=True)
class Target:
"""Notification target details.""" """Notification target details."""
id: str id: str
@ -42,37 +69,76 @@ class Target(TypedDict):
on: List[str] on: List[str]
request: targetRequest request: targetRequest
def serialize(self) -> dict:
return {
"id": self.id,
"name": self.name,
"on": self.on,
"request": self.request.serialize(),
}
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return self.serialize().get(key, default)
class NotificationEvents:
ADDED = Events.ADDED
COMPLETED = Events.COMPLETED
ERROR = Events.ERROR
CANCELLED = Events.CANCELLED
CLEARED = Events.CLEARED
LOG_INFO = Events.LOG_INFO
LOG_SUCCESS = Events.LOG_SUCCESS
TEST = Events.TEST
@staticmethod
def getEvents() -> dict[str, str]:
data: dict[str, str] = {}
for k, v in vars(NotificationEvents).items():
if not k.startswith("__") and not callable(v):
data[k] = v
return data
@staticmethod
def isValid(event: str) -> bool:
return event in NotificationEvents.getEvents().values()
class Notification(metaclass=Singleton): class Notification(metaclass=Singleton):
targets: list[Target] = [] _targets: list[Target] = []
"""Notification targets to send events to.""" """Notification targets to send events to."""
allowed_events: tuple = (
"added",
"completed",
"error",
"not_live",
"canceled",
"cleared",
"log_info",
"log_success",
)
"""Events that can be sent to the targets."""
_instance = None _instance = None
"""The instance of the Notification class."""
def __init__(self): def __init__(
self,
file: str | None = None,
client: httpx.AsyncClient | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
):
Notification._instance = self Notification._instance = self
config: Config = config or Config.get_instance()
config = Config.get_instance()
self._debug = config.debug self._debug = config.debug
self.file: str = os.path.join(config.config_path, "notifications.json") self._file: str = file or os.path.join(config.config_path, "notifications.json")
self._client: httpx.AsyncClient = httpx.AsyncClient() self._client: httpx.AsyncClient = client or httpx.AsyncClient()
self._encoder: Encoder = Encoder() self._encoder: Encoder = encoder or Encoder()
if os.path.exists(self.file) and os.path.getsize(self.file) > 10: if os.path.exists(self._file):
self.load() try:
if "600" != oct(os.stat(self._file).st_mode)[-3:]:
os.chmod(self._file, 0o600)
except Exception:
pass
if os.path.getsize(self._file) > 10:
self.load()
@staticmethod @staticmethod
def get_instance() -> "Notification": def get_instance() -> "Notification":
@ -83,14 +149,14 @@ class Notification(metaclass=Singleton):
def getTargets(self) -> list[Target]: def getTargets(self) -> list[Target]:
"""Get the list of notification targets.""" """Get the list of notification targets."""
return self.targets return self._targets
def clearTargets(self) -> "Notification": def clear(self) -> "Notification":
"""Clear the list of notification targets.""" """Clear the list of notification targets."""
self.targets.clear() self._targets.clear()
return self return self
def save(self, targets: list[Target] | None = None) -> "Notification": def save(self, targets: list[Target]) -> "Notification":
""" """
Save notification targets to the file. Save notification targets to the file.
@ -100,36 +166,36 @@ class Notification(metaclass=Singleton):
Returns: Returns:
Notification: The Notification instance. Notification: The Notification instance.
""" """
LOG.info(f"Saving notification targets to '{self.file}'.")
LOG.info(f"Saving notification targets to '{self._file}'.")
try: try:
with open(self.file, "w") as f: with open(self._file, "w") as f:
f.write(json.dumps(targets or self.targets, indent=4)) json.dump([t.serialize() for t in targets], fp=f, indent=4)
except Exception as e: except Exception as e:
LOG.error(f"Error saving notification targets to '{self.file}'. '{e}'") LOG.exception(e)
LOG.error(f"Error saving notification targets to '{self._file}'. '{e}'")
pass pass
return self return self
def load(self): def load(self) -> "Notification":
"""Load or reload notification targets from the file.""" """Load or reload notification targets from the file."""
if len(self.targets) > 0: if len(self._targets) > 0:
LOG.info("Clearing existing notification targets.") LOG.info("Clearing existing notification targets.")
self.targets.clear() self.clear()
modified = False if not os.path.exists(self._file) or os.path.getsize(self._file) < 10:
return self
targets = [] targets = []
if not os.path.exists(self.file) or os.path.getsize(self.file) < 10: LOG.info(f"Loading notification targets from '{self._file}'.")
return
LOG.info(f"Loading notification targets from '{self.file}'.")
try: try:
with open(self.file, "r") as f: with open(self._file, "r") as f:
targets = json.load(f) targets = json.load(f)
except Exception as e: except Exception as e:
LOG.error(f"Error loading notification targets from '{self.file}'. '{e}'") LOG.error(f"Error loading notification targets from '{self._file}'. '{e}'")
pass pass
for target in targets: for target in targets:
@ -140,48 +206,18 @@ class Notification(metaclass=Singleton):
LOG.error(f"Invalid notification target '{target}'. '{e}'") LOG.error(f"Invalid notification target '{target}'. '{e}'")
continue continue
if "on" not in target: target = self.makeTarget(target)
target["on"] = []
modified = True
if "type" not in target["request"]: self._targets.append(target)
target["request"]["type"] = "json"
modified = True
if "method" not in target["request"]:
target["request"]["method"] = "POST"
modified = True
if "id" not in target or validateUUID(target["id"], version=4) is False:
target["id"] = str(uuid.uuid4())
modified = True
target = Target(
id=target.get("id"),
name=target.get("name"),
on=target.get("on", []),
request=targetRequest(
type=target.get("request", {}).get("type", "json"),
method=target.get("request", {}).get("method", "POST"),
url=target.get("request", {}).get("url"),
headers=[
targetRequestHeader(key=h.get("key"), value=h.get("value"))
for h in target.get("request", {}).get("headers", [])
],
),
)
self.targets.append(target)
LOG.info( LOG.info(
f"Will send '{target['on'] if len(target['on']) > 0 else 'all'}' as {target['request']['type']} notification events to '{target['name']}'." f"Will send '{target.on if len(target.on) > 0 else 'all'}' as {target.request.type} notification events to '{target.name}'."
) )
except Exception as e: except Exception as e:
LOG.error(f"Error loading notification target '{target}'. '{e}'") LOG.error(f"Error loading notification target '{target}'. '{e}'")
pass pass
if modified: return self
self.save()
@staticmethod @staticmethod
def validate(target: Target | dict) -> bool: def validate(target: Target | dict) -> bool:
@ -195,7 +231,10 @@ class Notification(metaclass=Singleton):
bool: True if the target is valid, False otherwise. bool: True if the target is valid, False otherwise.
""" """
if "id" not in target or validateUUID(target["id"], version=4) is False: if not isinstance(target, dict):
target = target.serialize()
if "id" not in target or validate_uuid(target["id"], version=4) is False:
raise ValueError("Invalid notification target. No ID found.") raise ValueError("Invalid notification target. No ID found.")
if "name" not in target: if "name" not in target:
@ -218,7 +257,7 @@ class Notification(metaclass=Singleton):
raise ValueError("Invalid notification target. Invalid 'on' event list found.") raise ValueError("Invalid notification target. Invalid 'on' event list found.")
for e in target["on"]: for e in target["on"]:
if e not in Notification.allowed_events: if e not in NotificationEvents.getEvents().values():
raise ValueError(f"Invalid notification target. Invalid event '{e}' found.") raise ValueError(f"Invalid notification target. Invalid event '{e}' found.")
if "headers" in target["request"]: if "headers" in target["request"]:
@ -234,7 +273,7 @@ class Notification(metaclass=Singleton):
return True return True
async def send(self, event: str, item: ItemDTO | dict) -> list[dict]: async def send(self, event: str, item: ItemDTO | dict) -> list[dict]:
if len(self.targets) < 1: if len(self._targets) < 1:
return [] return []
if not isinstance(item, ItemDTO) and not isinstance(item, dict): if not isinstance(item, ItemDTO) and not isinstance(item, dict):
@ -243,8 +282,8 @@ class Notification(metaclass=Singleton):
tasks = [] tasks = []
for target in self.targets: for target in self._targets:
if len(target["on"]) > 0 and event not in target["on"] and "test" != event: if len(target.on) > 0 and event not in target.on and "test" != event:
continue continue
tasks.append(self._send(event, target, item)) tasks.append(self._send(event, target, item))
@ -252,44 +291,42 @@ class Notification(metaclass=Singleton):
return await asyncio.gather(*tasks) return await asyncio.gather(*tasks)
async def _send(self, event: str, target: Target, item: ItemDTO | dict) -> dict: async def _send(self, event: str, target: Target, item: ItemDTO | dict) -> dict:
req: targetRequest = target["request"]
try: try:
itemId = ag(item, ["id", "_id"], "??") itemId = item.get("id", item.get("_id", "??"))
except Exception: except Exception:
itemId = "??" itemId = "??"
try: try:
LOG.info(f"Sending Notification event '{event}' id '{itemId}' to '{target.get('name')}'.") LOG.info(f"Sending Notification event '{event}' id '{itemId}' to '{target.name}'.")
reqBody = { reqBody = {
"method": req["method"].upper(), "method": target.request.method.upper(),
"url": req["url"], "url": target.request.url,
"headers": { "headers": {
"User-Agent": f"YTPTube/{APP_VERSION}", "User-Agent": f"YTPTube/{APP_VERSION}",
"Content-Type": "application/json" "Content-Type": "application/json"
if "json" == req["type"].lower() if "json" == target.request.type.lower()
else "application/x-www-form-urlencoded", else "application/x-www-form-urlencoded",
}, },
} }
if len(req["headers"]) > 0: if len(target.request.headers) > 0:
for h in req["headers"]: for h in target.request.headers:
reqBody["headers"][h["key"]] = h["value"] reqBody["headers"][h.key] = h.value
reqBody["json" if "json" == req["type"].lower() else "data"] = { reqBody["json" if "json" == target.request.type.lower() else "data"] = {
"event": event, "event": event,
"created_at": datetime.now(tz=timezone.utc).isoformat(), "created_at": datetime.now(tz=timezone.utc).isoformat(),
"payload": item.__dict__ if isinstance(item, ItemDTO) else item, "payload": item.__dict__ if isinstance(item, ItemDTO) else item,
} }
if "form" == req["type"].lower(): if "form" == target.request.type.lower():
reqBody["data"]["payload"] = self._encoder.encode(reqBody["data"]["payload"]) reqBody["data"]["payload"] = self._encoder.encode(reqBody["data"]["payload"])
response = await self._client.request(**reqBody) response = await self._client.request(**reqBody)
respData = {"url": req["url"], "status": response.status_code, "text": response.text} respData = {"url": target.request.url, "status": response.status_code, "text": response.text}
msg = f"Notification target '{target['name']}' Responded to event '{event}' id '{itemId}' with status '{response.status_code}'." msg = f"Notification target '{target.name}' Responded to event '{event}' id '{itemId}' with status '{response.status_code}'."
if self._debug and respData.get("text"): if self._debug and respData.get("text"):
msg += f" body '{respData.get('text','??')}'." msg += f" body '{respData.get('text','??')}'."
@ -297,14 +334,39 @@ class Notification(metaclass=Singleton):
return respData return respData
except Exception as e: except Exception as e:
LOG.error(f"Error sending Notification event '{event}' id '{itemId}' to '{target['name']}'. '{e}'.") LOG.error(f"Error sending Notification event '{event}' id '{itemId}' to '{target.name}'. '{e}'.")
return {"url": req["url"], "status": 500, "text": str(e)} return {"url": target.request.url, "status": 500, "text": str(e)}
def makeTarget(self, target: dict) -> Target:
"""
Make a notification target from a dictionary.
Args:
target (dict): The target details.
Returns:
Target: The notification target.
"""
return Target(
id=target.get("id"),
name=target.get("name"),
on=target.get("on", []),
request=targetRequest(
type=target.get("request", {}).get("type", "json"),
method=target.get("request", {}).get("method", "POST"),
url=target.get("request", {}).get("url"),
headers=[
targetRequestHeader(key=h.get("key"), value=h.get("value"))
for h in target.get("request", {}).get("headers", [])
],
),
)
def emit(self, event, data, **kwargs): def emit(self, event, data, **kwargs):
if len(self.targets) < 1: if len(self._targets) < 1:
return False return False
if event not in self.allowed_events and "test" != event: if not NotificationEvents.isValid(event):
return False return False
return self.send(event, data) return self.send(event, data)

336
app/library/Tasks.py Normal file
View file

@ -0,0 +1,336 @@
import asyncio
from datetime import datetime
import json
import logging
import os
import time
from dataclasses import dataclass, field
from typing import Any, List
import httpx
from aiocron import Cron, crontab
from aiohttp import web
from .config import Config
from .Emitter import Emitter
from .encoder import Encoder
from .EventsSubscriber import Event, Events, EventsSubscriber
from .Singleton import Singleton
LOG = logging.getLogger("notifications")
@dataclass(kw_only=True)
class Task:
id: str
name: str
url: str
folder: str = ""
preset: str = ""
timer: str = ""
template: str = ""
cookies: str = ""
config: dict[str, str] = field(default_factory=dict)
def serialize(self) -> dict:
return self.__dict__
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return self.serialize().get(key, default)
@dataclass(kw_only=True)
class Job:
id: str
name: str
task: Task
job: Cron
class Tasks(metaclass=Singleton):
"""
This class is used to manage the tasks.
"""
_jobs: List[Job] = []
"""The jobs for the tasks."""
_instance = None
"""The instance of the Tasks."""
def __init__(
self,
file: str | None = None,
emitter: Emitter | None = None,
loop: asyncio.AbstractEventLoop | None = None,
config: Config | None = None,
encoder: Encoder | None = None,
client: httpx.AsyncClient | None = None,
):
Tasks._instance = self
config = config or Config.get_instance()
self._debug = config.debug
self._default_preset = config.default_preset
self._file: str = file or os.path.join(config.config_path, "tasks.json")
self._client: httpx.AsyncClient = client or httpx.AsyncClient()
self._encoder: Encoder = encoder or Encoder()
self._loop: asyncio.AbstractEventLoop = loop or asyncio.get_event_loop()
self._emitter: Emitter = emitter or Emitter.get_instance()
if os.path.exists(self._file) and "600" != oct(os.stat(self._file).st_mode)[-3:]:
try:
os.chmod(self._file, 0o600)
except Exception:
pass
def handle_event(_, e: Event):
self.save(**e.data)
EventsSubscriber.get_instance().subscribe(Events.TASKS_ADD, f"{__class__}.save", handle_event)
@staticmethod
def get_instance() -> "Tasks":
"""
Get the instance of the Tasks.
Returns:
Tasks: The instance of the Tasks
"""
if not Tasks._instance:
Tasks._instance = Tasks()
return Tasks._instance
def attach(self, _: web.Application):
"""
Attach the tasks to the aiohttp application.
Args:
_ (web.Application): The aiohttp application.
Returns:
None
"""
self.load()
def shutdown(self):
"""
Shutdown the tasks service.
"""
self.clear()
def getTasks(self) -> List[Task]:
"""Return the tasks."""
return [job.task for job in self._jobs]
def load(self) -> "Tasks":
"""
Load the tasks.
Returns:
Tasks: The current instance.
"""
self.clear()
if not os.path.exists(self._file) or os.path.getsize(self._file) < 10:
return self
LOG.info(f"Loading tasks from '{self._file}'.")
try:
with open(self._file, "r") as f:
tasks = json.load(f)
except Exception as e:
LOG.error(f"Failed to parse tasks from '{self._file}'. '{e}'.")
return self
if not tasks or len(tasks) < 1:
LOG.info(f"No tasks were defined in '{self._file}'.")
return self
for i, task in enumerate(tasks):
try:
task = Task(**task)
except Exception as e:
LOG.error(f"Failed to parse task at list position '{i}'. '{str(e)}'.")
continue
try:
self._jobs.append(
Job(
id=task.id,
name=task.name,
task=task,
job=crontab(spec=task.timer, func=self._runner, args=(task,), start=True, loop=self._loop),
)
)
LOG.info(f"Task '{i}: {task.name}' queued to be executed every '{task.timer}'.")
except Exception as e:
LOG.exception(e)
LOG.error(f"Failed to queue task '{i}: {task['name']}'. '{str(e)}'.")
return self
def clear(self) -> "Tasks":
"""
Clear all tasks.
Returns:
Tasks: The current instance.
"""
if len(self._jobs) < 1:
return self
for task in self._jobs:
try:
LOG.info(f"Stopping job '{task.id}: {task.name}'.")
task.job.stop()
except Exception as e:
LOG.exception(e)
LOG.error(f"Failed to stop job '{task.id}: {task.name}'. '{str(e)}'.")
self._jobs.clear()
return self
def validate(self, task: Task | dict) -> bool:
"""
Validate the task.
Args:
tasks (Task|dict): The task to validate.
Returns:
bool: True if the task is valid, False otherwise.
"""
if not isinstance(task, dict):
if not isinstance(task, Task):
raise ValueError("Invalid task type.")
task = task.serialize()
if not task.get("name"):
raise ValueError("No name found.")
if not task.get("timer"):
raise ValueError("No timer found.")
if not task.get("url"):
raise ValueError("No URL found.")
if not isinstance(task.get("cookies"), str):
raise ValueError("Invalid cookies type.")
if not isinstance(task.get("config"), dict):
raise ValueError("Invalid config type.")
if not isinstance(task.get("template"), str):
raise ValueError("Invalid template type.")
return True
def save(self, tasks: List[Task | dict]) -> "Tasks":
"""
Save the tasks.
Args:
tasks (List[Task]): The tasks to save.
Returns:
Tasks: The current instance.
"""
for i, task in enumerate(tasks):
try:
if not isinstance(task, Task):
task = Task(**task)
tasks[i] = task
except Exception as e:
LOG.error(f"Failed to save task '{i}' unable to parse task. '{str(e)}'.")
continue
try:
self.validate(task)
except ValueError as e:
LOG.error(f"Failed to add task '{i}: {task.name}'. '{e}'.")
continue
try:
with open(self._file, "w") as f:
json.dump(obj=[task.serialize() for task in tasks], fp=f, indent=4)
LOG.info(f"Tasks saved to '{self._file}'.")
except Exception as e:
LOG.error(f"Failed to save tasks to '{self._file}'. '{str(e)}'.")
return self
async def _runner(self, task: Task):
"""
Run the task.
Args:
task (Task): The task to run.
Returns:
None
"""
try:
timeNow = datetime.now().isoformat()
started = time.time()
if not task.url:
LOG.error(f"Failed to dispatch task '{task.id}: {task.name}'. No URL found.")
return
preset: str = str(task.preset or self._default_preset)
folder: str = task.folder if task.folder else ""
cookies: str = str(task.cookies) if task.cookies else ""
template: str = task.template if task.template else ""
config = task.config if task.config else {}
if isinstance(config, str) and config:
try:
config = json.loads(config)
except Exception as e:
LOG.error(f"Failed to parse json yt-dlp config for '{task.name}'. {str(e)}")
return
LOG.info(f"Task '{task.id}: {task.name}' dispatched at '{timeNow}'.")
tasks = []
tasks.append(self._emitter.info(f"Task '{task.name}' dispatched at '{timeNow}'."))
tasks.append(
self._emitter.emit(
event=Events.ADD_URL,
data=Event(
id=task.id,
data={
"url": task.url,
"preset": preset,
"folder": folder,
"cookies": cookies,
"config": config,
"template": template,
},
),
local=True,
)
)
await asyncio.wait_for(asyncio.gather(*tasks), timeout=None)
timeNow = datetime.now().isoformat()
ended = time.time()
LOG.info(f"Task '{task.id}: {task.name}' completed at '{timeNow}' took '{ended - started:.2f}' seconds.")
await self._emitter.success(f"Task '{task.name}' completed in '{ended - started:.2f}' seconds.")
except Exception as e:
timeNow = datetime.now().isoformat()
LOG.error(f"Task '{task.id}: {task.name}' has failed to execute at '{timeNow}'. '{str(e)}'.")
await self._emitter.error(f"Task '{task.name}' failed to execute at '{timeNow}'. '{str(e)}'.")

2
app/library/Utils.py
View file

@ -537,7 +537,7 @@ def arg_converter(args: str) -> dict:
return json.loads(json.dumps(diff)) return json.loads(json.dumps(diff))
def validateUUID(uuid_str: str, version: int = 4) -> bool: def validate_uuid(uuid_str: str, version: int = 4) -> bool:
""" """
Validate if the UUID is valid. Validate if the UUID is valid.

90
app/library/common.py
View file

@ -1,7 +1,9 @@
import json
import logging import logging
from .DownloadQueue import DownloadQueue from .DownloadQueue import DownloadQueue
from .encoder import Encoder from .encoder import Encoder
from .config import Config
LOG = logging.getLogger("common") LOG = logging.getLogger("common")
@ -11,27 +13,87 @@ class common:
This class is used to share common methods between the socket and the API gateways. This class is used to share common methods between the socket and the API gateways.
""" """
queue: DownloadQueue def __init__(
encoder: Encoder self,
queue: DownloadQueue | None = None,
def __init__(self, queue: DownloadQueue, encoder: Encoder): encoder: Encoder | None = None,
config: Config | None = None,
):
super().__init__() super().__init__()
self.queue = queue self.queue = queue or DownloadQueue.get_instance()
self.encoder = encoder self.encoder = encoder or Encoder()
config = config or Config.get_instance()
self.default_preset = config.default_preset
async def add( async def add(
self, url: str, preset: str, folder: str, ytdlp_cookies: str, ytdlp_config: dict, output_template: str self, url: str, preset: str, folder: str, cookies: str, config: dict, template: str
) -> dict[str, str]: ) -> dict[str, str]:
if ytdlp_cookies and isinstance(ytdlp_cookies, dict): """
ytdlp_cookies = self.encoder.encode(ytdlp_cookies) Add an item to the download queue.
status = await self.queue.add( Args:
url (str): The url to be added to the queue.
preset (str): The preset to be used for the download.
folder (str): The folder to save the download to.
cookies (str): The cookies to be used for the download.
config (dict): The yt-dlp config to be used for the download.
template (str): The template to be used for the download.
Returns:
dict[str, str]: The status of the download.
{ "status": "text" }
"""
if cookies and isinstance(cookies, dict):
cookies = self.encoder.encode(cookies)
return await self.queue.add(
url=url, url=url,
preset=preset if preset else "default", preset=preset if preset else "default",
folder=folder, folder=folder,
ytdlp_cookies=ytdlp_cookies, cookies=cookies,
ytdlp_config=ytdlp_config if isinstance(ytdlp_config, dict) else {}, config=config if isinstance(config, dict) else {},
output_template=output_template, template=template,
) )
return status def format_item(self, item: dict) -> dict:
"""
Format the item to be added to the download queue.
Args:
item (dict): The item to be formatted.
Raises:
ValueError: If the url is not provided.
ValueError: If the yt-dlp config is not a valid json.
Returns:
dict: The formatted item
"""
url: str = item.get("url")
if not url:
raise ValueError("url param is required.")
preset: str = str(item.get("preset", self.config.default_preset))
folder: str = str(item.get("folder")) if item.get("folder") else ""
cookies: str = str(item.get("cookies")) if item.get("cookies") else ""
template: str = str(item.get("template")) if item.get("template") else ""
config = item.get("config")
if isinstance(config, str) and config:
try:
config = json.loads(config)
except Exception as e:
raise ValueError(f"Failed to parse json yt-dlp config for '{url}'. {str(e)}")
item = {
"url": url,
"preset": preset,
"folder": folder,
"cookies": cookies,
"config": config if isinstance(config, dict) else {},
"template": template,
}
return item

168
app/main.py
View file

@ -1,52 +1,39 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
import asyncio import asyncio
import json
import logging import logging
import os import os
import random
import sqlite3 import sqlite3
from datetime import datetime
from pathlib import Path from pathlib import Path
import time
from typing import TypedDict
import uuid
import caribou import caribou
import magic import magic
from aiocron import crontab, Cron
from aiohttp import web from aiohttp import web
from library.Utils import load_file
from library.config import Config from library.config import Config
from library.DownloadQueue import DownloadQueue from library.DownloadQueue import DownloadQueue
from library.Emitter import Emitter from library.Emitter import Emitter
from library.encoder import Encoder from library.EventsSubscriber import EventsSubscriber
from library.HttpAPI import HttpAPI, LOG as http_logger from library.HttpAPI import LOG as http_logger
from library.HttpAPI import HttpAPI
from library.HttpSocket import HttpSocket from library.HttpSocket import HttpSocket
from library.Notifications import Notification from library.Notifications import Notification
from library.PackageInstaller import PackageInstaller from library.PackageInstaller import PackageInstaller
from library.Tasks import Tasks
LOG = logging.getLogger("app") LOG = logging.getLogger("app")
MIME = magic.Magic(mime=True) MIME = magic.Magic(mime=True)
class job_item(TypedDict):
name: str
job: Cron
class Main: class Main:
config: Config config: Config
app: web.Application app: web.Application
http: HttpAPI http: HttpAPI
socket: HttpSocket socket: HttpSocket
cron: list[job_item] = []
def __init__(self): def __init__(self):
self.config = Config.get_instance() self.config = Config.get_instance()
self.rootPath = str(Path(__file__).parent.absolute()) self.rootPath = str(Path(__file__).parent.absolute())
self.app = web.Application() self.app = web.Application()
self.encoder = Encoder()
self.checkFolders() self.checkFolders()
@ -61,18 +48,27 @@ class Main:
connection = sqlite3.connect(database=self.config.db_file, isolation_level=None) connection = sqlite3.connect(database=self.config.db_file, isolation_level=None)
connection.row_factory = sqlite3.Row connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=wal") connection.execute("PRAGMA journal_mode=wal")
try:
if "600" != oct(os.stat(self.config.db_file).st_mode)[-3:]:
os.chmod(self.config.db_file, 0o600)
except Exception:
pass
self.emitter = Emitter() queue = DownloadQueue(connection=connection)
self.http = HttpAPI(queue=queue)
self.socket = HttpSocket(queue=queue)
Emitter.get_instance().add_emitter([Notification().emit], local=False).add_emitter(
[EventsSubscriber().emit], local=True
)
queue = DownloadQueue(emitter=self.emitter, connection=connection)
self.app.on_startup.append(lambda _: queue.initialize()) self.app.on_startup.append(lambda _: queue.initialize())
self.http = HttpAPI(queue=queue, emitter=self.emitter, encoder=self.encoder, load_tasks=self.load_tasks) def checkFolders(self):
self.socket = HttpSocket(queue=queue, emitter=self.emitter, encoder=self.encoder) """
self.emitter.add_emitter(Notification().emit) Check if the required folders exist and create them if they do not.
"""
def checkFolders(self) -> None:
try: try:
LOG.debug(f"Checking download folder at '{self.config.download_path}'.") LOG.debug(f"Checking download folder at '{self.config.download_path}'.")
if not os.path.exists(self.config.download_path): if not os.path.exists(self.config.download_path):
@ -110,127 +106,13 @@ class Main:
LOG.error(f"Could not create database file at '{self.config.db_file}'.") LOG.error(f"Could not create database file at '{self.config.db_file}'.")
raise e raise e
async def cron_runner(self, task: dict):
try:
taskName = task.get("name", task.get("url"))
timeNow = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
started = time.time()
url = task.get("url")
if not url:
LOG.error(f"Invalid task '{task}'. No URL found.")
return
preset: str = str(task.get("preset", self.config.default_preset))
folder: str = str(task.get("folder")) if task.get("folder") else ""
ytdlp_cookies: str = str(task.get("ytdlp_cookies")) if task.get("ytdlp_cookies") else ""
output_template: str = str(task.get("output_template")) if task.get("output_template") else ""
ytdlp_config = task.get("ytdlp_config")
if isinstance(ytdlp_config, str) and ytdlp_config:
try:
ytdlp_config = json.loads(ytdlp_config)
except Exception as e:
LOG.error(f"Failed to parse json yt-dlp config for '{taskName}'. {str(e)}")
return
await self.emitter.info(f"Started 'Task: {taskName}' at '{timeNow}'.")
LOG.info(f"Started 'Task: {taskName}' at '{timeNow}'.")
await self.socket.add(
url=url,
preset=preset,
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config,
output_template=output_template,
)
timeNow = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
ended = time.time()
LOG.info(f"Completed 'Task: {taskName}' at '{timeNow}' ")
await self.emitter.success(f"Completed 'Task: {taskName}' '{ended - started:.2f}' seconds.")
except Exception as e:
timeNow = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
LOG.error(f"Failed 'Task: {taskName}' at '{timeNow}'. Error message '{str(e)}'.")
await self.emitter.error(f"Failed 'Task: {taskName}' at '{timeNow}'. Error message '{str(e)}'.")
def load_tasks(self):
tasksFile = os.path.join(self.config.config_path, "tasks.json")
if not os.path.exists(tasksFile):
return
LOG.info(f"Loading tasks from '{tasksFile}'.")
try:
(tasks, status, error) = load_file(tasksFile, list)
if not status:
LOG.error(f"Could not load tasks file from '{tasksFile}'. '{error}'.")
return
except Exception:
pass
for job in self.cron:
try:
LOG.info(f"Stopping job '{job['name']}'.")
job["job"].stop()
except Exception as e:
LOG.error(f"Failed to stop job. Error message '{str(e)}'.")
LOG.exception(e)
self.cron.clear()
if not tasks or len(tasks) < 1:
LOG.info(f"No tasks found in '{tasksFile}'.")
return
modified = False
loop = asyncio.get_event_loop()
for task in tasks:
if not task.get("url"):
LOG.warning(f"Invalid task '{task}'. No URL found.")
continue
try:
if not task.get("timer", None):
task["timer"] = f"{random.randint(1,59)} */1 * * *"
modified = True
if not task.get("id", None):
task["id"] = str(uuid.uuid4())
modified = True
self.cron.append(
job_item(
name=task.get("name", "??"),
job=crontab(
spec=task.get("timer"),
func=self.cron_runner,
args=(task,),
start=True,
loop=loop,
),
)
)
LOG.info(f"Queued 'Task: {task.get('name','??')}' to be executed every '{task.get('timer')}'.")
except Exception as e:
LOG.error(f"Failed to add 'Task: {task.get('name', '??')}'. Error message '{str(e)}'.")
LOG.exception(e)
if modified:
try:
with open(tasksFile, "w") as f:
f.write(json.dumps(tasks, indent=4))
except Exception as e:
LOG.error(f"Failed to save tasks file '{tasksFile}'. Error message '{str(e)}'.")
LOG.exception(e)
self.config.tasks = tasks
def start(self): def start(self):
"""
Start the application.
"""
self.socket.attach(self.app) self.socket.attach(self.app)
self.http.attach(self.app) self.http.attach(self.app)
Tasks.get_instance().attach(self.app)
self.load_tasks()
def started(_): def started(_):
LOG.info("=" * 40) LOG.info("=" * 40)

7
ui/assets/css/style.css
View file

@ -160,6 +160,13 @@ hr {
background-color: #00d1b2; background-color: #00d1b2;
} }
.button.is-purple {
background-color: #5f00d1;
border-color: transparent;
color: #fff;
}
@media (prefers-color-scheme: dark) { @media (prefers-color-scheme: dark) {
.footer { .footer {
background-color: #121212; background-color: #121212;

13
ui/components/GetInfo.vue
View file

@ -29,6 +29,7 @@ import { request } from '~/utils/index'
const emitter = defineEmits(['closeModel']) const emitter = defineEmits(['closeModel'])
const isLoading = ref(false) const isLoading = ref(false)
const data = ref({}) const data = ref({})
const toast = useToast()
const props = defineProps({ const props = defineProps({
link: { link: {
@ -47,15 +48,17 @@ const eventFunc = e => {
onMounted(async () => { onMounted(async () => {
window.addEventListener('keydown', eventFunc) window.addEventListener('keydown', eventFunc)
const url = '/api/url/info?url=' + encodePath(props.link)
const url = '/api/yt-dlp/url/info?url=' + encodePath(props.link)
try { try {
isLoading.value = true isLoading.value = true
const response = await request(url, { credentials: 'include' }); const response = await request(url, { credentials: 'include' });
data.value = await response.json(); data.value = await response.json()
} catch (e) { } catch (e) {
console.log(e) console.error(e)
} toast.error(`Error: ${e.message}`)
finally { } finally {
isLoading.value = false isLoading.value = false
} }
}) })

8
ui/components/History.vue
View file

@ -421,7 +421,7 @@ const setIcon = item => {
return 'fa-solid fa-circle-xmark' return 'fa-solid fa-circle-xmark'
} }
if (item.status === 'canceled') { if (item.status === 'cancelled') {
return 'fa-solid fa-eject' return 'fa-solid fa-eject'
} }
@ -468,9 +468,9 @@ const reQueueItem = item => {
url: item.url, url: item.url,
preset: item.preset, preset: item.preset,
folder: item.folder, folder: item.folder,
ytdlp_config: item.ytdlp_config, config: item.config,
ytdlp_cookies: item.ytdlp_cookies, cookies: item.cookies,
output_template: item.output_template, template: item.template,
}) })
} }
</script> </script>

46
ui/components/NewDownload.vue
View file

@ -20,7 +20,8 @@
</div> </div>
<div class="control is-expanded"> <div class="control is-expanded">
<div class="select is-fullwidth"> <div class="select is-fullwidth">
<select id="preset" class="is-fullwidth" :disabled="!socket.isConnected" v-model="selectedPreset"> <select id="preset" class="is-fullwidth" :disabled="!socket.isConnected || addInProgress"
v-model="selectedPreset">
<option v-for="item in config.presets" :key="item.name" :value="item.name"> <option v-for="item in config.presets" :key="item.name" :value="item.name">
{{ item.name }} {{ item.name }}
</option> </option>
@ -36,7 +37,7 @@
</div> </div>
<div class="control is-expanded"> <div class="control is-expanded">
<input type="text" class="input is-fullwidth" id="path" v-model="downloadPath" placeholder="Default" <input type="text" class="input is-fullwidth" id="path" v-model="downloadPath" placeholder="Default"
:disabled="!socket.isConnected" list="folders"> :disabled="!socket.isConnected || addInProgress" list="folders">
</div> </div>
</div> </div>
</div> </div>
@ -65,6 +66,7 @@
</label> </label>
<div class="control"> <div class="control">
<input type="text" class="input" v-model="output_template" id="output_format" <input type="text" class="input" v-model="output_template" id="output_format"
:disabled="!socket.isConnected || addInProgress"
placeholder="Uses default output template naming if empty."> placeholder="Uses default output template naming if empty.">
</div> </div>
<span class="help"> <span class="help">
@ -80,12 +82,13 @@
JSON yt-dlp config or CLI options. JSON yt-dlp config or CLI options.
</label> </label>
<div class="control"> <div class="control">
<textarea class="textarea" id="ytdlpConfig" v-model="ytdlpConfig" :disabled="!socket.isConnected" <textarea class="textarea" id="ytdlpConfig" v-model="ytdlpConfig"
:disabled="!socket.isConnected || addInProgress"
placeholder="--no-embed-metadata --no-embed-thumbnail"></textarea> placeholder="--no-embed-metadata --no-embed-thumbnail"></textarea>
</div> </div>
<span class="help"> <span class="help">
Some config fields are ignored like cookiefile, path, and output_format etc. Some config fields are ignored like <code>format</code> <code>cookiefile</code>, <code>paths</code>,
Available option can be found at <NuxtLink target="_blank" and <code>outtmpl</code> etc. Available option can be found at <NuxtLink target="_blank"
to="https://github.com/yt-dlp/yt-dlp/blob/a0b19d319a6ce8b7059318fa17a34b144fde1785/yt_dlp/YoutubeDL.py#L194"> to="https://github.com/yt-dlp/yt-dlp/blob/a0b19d319a6ce8b7059318fa17a34b144fde1785/yt_dlp/YoutubeDL.py#L194">
this page</NuxtLink>. Warning: Use with caution some of those options can break yt-dlp or the this page</NuxtLink>. Warning: Use with caution some of those options can break yt-dlp or the
frontend. frontend.
@ -99,7 +102,7 @@
</label> </label>
<div class="control"> <div class="control">
<textarea class="textarea" id="ytdlpCookies" v-model="ytdlpCookies" <textarea class="textarea" id="ytdlpCookies" v-model="ytdlpCookies"
:disabled="!socket.isConnected"></textarea> :disabled="!socket.isConnected || addInProgress"></textarea>
</div> </div>
<span class="help"> <span class="help">
Use <NuxtLink target="_blank" to="https://github.com/jrie/flagCookies"> Use <NuxtLink target="_blank" to="https://github.com/jrie/flagCookies">
@ -157,7 +160,7 @@ const addInProgress = ref(false)
const addDownload = async () => { const addDownload = async () => {
// -- send request to convert cli options to JSON // -- send request to convert cli options to JSON
if (ytdlpConfig.value && ytdlpConfig.value.length > 2 && !ytdlpConfig.value.trim().startsWith('{')) { if (ytdlpConfig.value && !ytdlpConfig.value.trim().startsWith('{')) {
const response = await request('/api/yt-dlp/convert', { const response = await request('/api/yt-dlp/convert', {
method: 'POST', method: 'POST',
headers: { headers: {
@ -175,7 +178,7 @@ const addDownload = async () => {
ytdlpConfig.value = JSON.stringify(data, null, 2) ytdlpConfig.value = JSON.stringify(data, null, 2)
} }
if (ytdlpConfig.value && ytdlpConfig.value.length > 2) { if (ytdlpConfig.value) {
try { try {
JSON.parse(ytdlpConfig.value) JSON.parse(ytdlpConfig.value)
} catch (e) { } catch (e) {
@ -193,9 +196,9 @@ const addDownload = async () => {
url: url, url: url,
preset: config.app.basic_mode ? config.app.default_preset : selectedPreset.value, preset: config.app.basic_mode ? config.app.default_preset : selectedPreset.value,
folder: config.app.basic_mode ? null : downloadPath.value, folder: config.app.basic_mode ? null : downloadPath.value,
ytdlp_config: config.app.basic_mode ? '' : ytdlpConfig.value, config: config.app.basic_mode ? '' : ytdlpConfig.value,
ytdlp_cookies: config.app.basic_mode ? '' : ytdlpCookies.value, cookies: config.app.basic_mode ? '' : ytdlpCookies.value,
output_template: config.app.basic_mode ? null : output_template.value, template: config.app.basic_mode ? null : output_template.value,
}) })
}) })
} }
@ -216,8 +219,8 @@ const resetConfig = () => {
toast.success('Local configuration has been reset.') toast.success('Local configuration has been reset.')
} }
const statusHandler = async data => { const statusHandler = async stream => {
const { status, msg } = JSON.parse(data) const { status, msg } = JSON.parse(stream)
addInProgress.value = false addInProgress.value = false
@ -229,11 +232,26 @@ const statusHandler = async data => {
url.value = '' url.value = ''
} }
const unlockDownload = async stream => {
const json = JSON.parse(stream)
if (!json?.data) {
return
}
if ("unlock" in json.data && json.data.unlock === true) {
addInProgress.value = false
}
}
onMounted(() => { onMounted(() => {
socket.on('status', statusHandler) socket.on('status', statusHandler)
socket.on('error', unlockDownload)
if ('' === selectedPreset.value) { if ('' === selectedPreset.value) {
selectedPreset.value = config.app.default_preset.value selectedPreset.value = config.app.default_preset.value
} }
}) })
onUnmounted(() => socket.off('status', statusHandler))
onUnmounted(() => {
socket.off('status', statusHandler)
socket.off('error', unlockDownload)
})
</script> </script>

14
ui/components/NotificationForm.vue
View file

@ -205,11 +205,8 @@
</template> </template>
<script setup> <script setup>
import { request } from '~/utils/index'
const emitter = defineEmits(['cancel', 'submit']); const emitter = defineEmits(['cancel', 'submit']);
const toast = useToast(); const toast = useToast();
const addInProgress = ref(false);
const props = defineProps({ const props = defineProps({
reference: { reference: {
type: String, type: String,
@ -223,7 +220,12 @@ const props = defineProps({
item: { item: {
type: Object, type: Object,
required: true, required: true,
} },
addInProgress: {
type: Boolean,
required: false,
default: false,
},
}) })
const form = reactive(props.item); const form = reactive(props.item);
@ -252,16 +254,12 @@ const checkInfo = async () => {
return; return;
} }
// -- validate headers
for (const header of form.request.headers) { for (const header of form.request.headers) {
if (!header.key || !header.value) { if (!header.key || !header.value) {
form.request.headers.splice(form.request.headers.indexOf(header), 1); form.request.headers.splice(form.request.headers.indexOf(header), 1);
return;
} }
} }
addInProgress.value = true;
emitter('submit', { reference: toRaw(props.reference), item: toRaw(form) }); emitter('submit', { reference: toRaw(props.reference), item: toRaw(form) });
} }

51
ui/components/TaskForm.vue
View file

@ -113,25 +113,24 @@
</label> </label>
<div class="control has-icons-left"> <div class="control has-icons-left">
<input type="text" class="input" id="output_template" placeholder="The output template to use" <input type="text" class="input" id="output_template" placeholder="The output template to use"
v-model="form.output_template" :disabled="addInProgress"> v-model="form.template" :disabled="addInProgress">
<span class="icon is-small is-left"><i class="fa-solid fa-file" /></span> <span class="icon is-small is-left"><i class="fa-solid fa-file" /></span>
</div> </div>
<span class="help"> <span class="help">
<span class="icon"><i class="fa-solid fa-info" /></span> <span class="icon"><i class="fa-solid fa-info" /></span>
<span>The output template to use, if not set, it will defaults to <span>The output template to use, if not set, it will defaults to
<code>{{ config.app.output_template }}</code></span> <code>{{ config.app.template }}</code></span>
</span> </span>
</div> </div>
</div> </div>
<div class="column is-6-tablet is-12-mobile"> <div class="column is-6-tablet is-12-mobile">
<div class="field"> <div class="field">
<label class="label is-inline" for="ytdlp_config" <label class="label is-inline" for="config" v-tooltip="'Extends current global yt-dlp config. (JSON)'">
v-tooltip="'Extends current global yt-dlp config. (JSON)'">
JSON yt-dlp config or CLI options. JSON yt-dlp config or CLI options.
</label> </label>
<div class="control"> <div class="control">
<textarea class="textarea" id="ytdlp_config" v-model="form.ytdlp_config" :disabled="addInProgress" <textarea class="textarea" id="config" v-model="form.config" :disabled="addInProgress"
placeholder="--no-embed-metadata --no-embed-thumbnail"></textarea> placeholder="--no-embed-metadata --no-embed-thumbnail"></textarea>
</div> </div>
<span class="help"> <span class="help">
@ -144,12 +143,11 @@
<div class="column is-6-tablet is-12-mobile"> <div class="column is-6-tablet is-12-mobile">
<div class="field"> <div class="field">
<label class="label is-inline" for="ytdlp_cookies" v-tooltip="'JSON exported cookies for downloading.'"> <label class="label is-inline" for="cookies" v-tooltip="'JSON exported cookies for downloading.'">
yt-dlp Cookies JSON Cookies
</label> </label>
<div class="control"> <div class="control">
<textarea class="textarea" id="ytdlp_cookies" v-model="form.ytdlp_cookies" <textarea class="textarea" id="cookies" v-model="form.cookies" :disabled="addInProgress"></textarea>
:disabled="addInProgress"></textarea>
</div> </div>
<span class="help"> <span class="help">
<span class="icon"><i class="fa-solid fa-info" /></span> <span class="icon"><i class="fa-solid fa-info" /></span>
@ -195,7 +193,7 @@ import { request } from '~/utils/index'
const emitter = defineEmits(['cancel', 'submit']); const emitter = defineEmits(['cancel', 'submit']);
const toast = useToast(); const toast = useToast();
const config = useConfigStore(); const config = useConfigStore();
const addInProgress = ref(false);
const props = defineProps({ const props = defineProps({
reference: { reference: {
type: String, type: String,
@ -205,12 +203,22 @@ const props = defineProps({
task: { task: {
type: Object, type: Object,
required: true, required: true,
} },
addInProgress: {
type: Boolean,
required: false,
default: false,
},
}) })
const form = reactive(props.task); const form = reactive(props.task);
onMounted(() => { onMounted(() => {
if (props.task?.config && (typeof props.task.config === 'object')) {
form.config = JSON.stringify(props.task.config, null, 4);
}
if (!props.task?.preset || '' === props.task.preset) { if (!props.task?.preset || '' === props.task.preset) {
form.preset = toRaw(config.app.default_preset); form.preset = toRaw(config.app.default_preset);
} }
@ -242,13 +250,13 @@ const checkInfo = async () => {
} }
// -- send request to convert cli options to JSON // -- send request to convert cli options to JSON
if (form.ytdlp_config && form.ytdlp_config.length > 2 && !form.ytdlp_config.trim().startsWith('{')) { if (form.config && form.config.length > 2 && !form.config.trim().startsWith('{')) {
const response = await request('/api/yt-dlp/convert', { const response = await request('/api/yt-dlp/convert', {
method: 'POST', method: 'POST',
headers: { headers: {
'Content-Type': 'application/json', 'Content-Type': 'application/json',
}, },
body: JSON.stringify({ args: form.ytdlp_config }), body: JSON.stringify({ args: form.config }),
}); });
const data = await response.json() const data = await response.json()
@ -257,20 +265,29 @@ const checkInfo = async () => {
return return
} }
form.ytdlp_config = JSON.stringify(data, null, 4) form.config = JSON.stringify(data, null, 4)
}
// -- check config
if (form.config) {
try {
form.config = JSON.parse(form.config)
} catch (e) {
toast.error(`Invalid JSON yt-dlp config. ${e.message}`)
return;
}
} }
// -- check cookies syntax // -- check cookies syntax
if (form.ytdlp_cookies) { if (form.cookies) {
try { try {
JSON.parse(form.ytdlp_cookies); JSON.parse(form.cookies);
} catch (e) { } catch (e) {
toast.error(`Invalid JSON yt-dlp cookies. ${e.message}`) toast.error(`Invalid JSON yt-dlp cookies. ${e.message}`)
return; return;
} }
} }
addInProgress.value = true;
emitter('submit', { reference: toRaw(props.reference), task: toRaw(form) }); emitter('submit', { reference: toRaw(props.reference), task: toRaw(form) });
} }
</script> </script>

70
ui/layouts/default.vue
View file

@ -10,37 +10,14 @@
</span> </span>
</NuxtLink> </NuxtLink>
</div> </div>
<div class="navbar-end is-flex" style="flex-flow:wrap">
<div class="navbar-item" v-if="socket.isConnected && config.app.has_cookies && !config.app.basic_mode">
<button class="button is-dark" @click="checkCookies" v-tooltip="'Check youtube cookies status.'"
:disabled="isChecking">
<span class="icon has-text-info"><i class="fas fa-cookie"></i></span>
</button>
</div>
<div class="navbar-item" v-if="socket.isConnected && !config.app.basic_mode"> <div class="navbar-end is-flex" style="flex-flow:wrap">
<button class="button is-dark" @click="pauseDownload" v-if="false === config.paused"
v-tooltip="'Pause non-active downloads.'">
<span class="icon has-text-warning"><i class="fas fa-pause"></i></span>
</button>
<button class="button is-dark" @click="socket.emit('resume', {})" v-else v-tooltip="'Resume downloading.'">
<span class="icon has-text-success"><i class="fas fa-play"></i></span>
</button>
</div>
<div class="navbar-item" v-if="!config.app.basic_mode"> <div class="navbar-item" v-if="!config.app.basic_mode">
<NuxtLink class="button is-dark has-tooltip-bottom" to="/console" v-tooltip.bottom="'Terminal'"> <NuxtLink class="button is-dark has-tooltip-bottom" to="/console" v-tooltip.bottom="'Terminal'">
<span class="icon"><i class="fa-solid fa-terminal" /></span> <span class="icon"><i class="fa-solid fa-terminal" /></span>
</NuxtLink> </NuxtLink>
</div> </div>
<div class="navbar-item" v-if="!config.app.basic_mode">
<button v-tooltip.bottom="'Toggle Add Form'" class="button is-dark has-tooltip-bottom"
@click="config.showForm = !config.showForm">
<span class="icon"><i class="fa-solid fa-plus" /></span>
</button>
</div>
<div class="navbar-item" v-if="!config.app.basic_mode" v-tooltip.bottom="'Tasks'"> <div class="navbar-item" v-if="!config.app.basic_mode" v-tooltip.bottom="'Tasks'">
<NuxtLink class="button is-dark has-tooltip-bottom" to="/tasks"> <NuxtLink class="button is-dark has-tooltip-bottom" to="/tasks">
<span class="icon"><i class="fa-solid fa-tasks" /></span> <span class="icon"><i class="fa-solid fa-tasks" /></span>
@ -64,7 +41,7 @@
</button> </button>
</div> </div>
<div class="navbar-item is-hidden-mobile"> <div class="navbar-item">
<button class="button is-dark" @click="reloadPage"> <button class="button is-dark" @click="reloadPage">
<span class="icon"><i class="fas fa-refresh"></i></span> <span class="icon"><i class="fas fa-refresh"></i></span>
</button> </button>
@ -72,9 +49,7 @@
</div> </div>
</nav> </nav>
<NewDownload v-if="config.showForm || config.app.basic_mode" @getInfo="url => get_info = url" /> <NuxtPage />
<NuxtPage @getInfo="url => get_info = url" />
<GetInfo v-if="get_info" :link="get_info" @closeModel="get_info = ''" />
<div class="columns mt-3 is-mobile"> <div class="columns mt-3 is-mobile">
<div class="column is-8-mobile"> <div class="column is-8-mobile">
@ -102,16 +77,12 @@ import 'assets/css/bulma.css'
import 'assets/css/style.css' import 'assets/css/style.css'
import 'assets/css/all.css' import 'assets/css/all.css'
import { useStorage } from '@vueuse/core' import { useStorage } from '@vueuse/core'
import moment from "moment"; import moment from 'moment'
import { request } from '~/utils/index'
const Year = new Date().getFullYear() const Year = new Date().getFullYear()
const selectedTheme = useStorage('theme', (() => window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light')()) const selectedTheme = useStorage('theme', (() => window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light')())
const socket = useSocketStore() const socket = useSocketStore()
const config = useConfigStore() const config = useConfigStore()
const isChecking = ref(false)
const toast = useToast()
const get_info = ref('')
const applyPreferredColorScheme = scheme => { const applyPreferredColorScheme = scheme => {
for (let s = 0; s < document.styleSheets.length; s++) { for (let s = 0; s < document.styleSheets.length; s++) {
@ -154,31 +125,6 @@ const applyPreferredColorScheme = scheme => {
} }
} }
const checkCookies = async () => {
if (true === isChecking.value) {
return
}
if (false === confirm(`Check for cookies status?`)) {
return
}
try {
isChecking.value = true
const response = await request('/api/youtube/auth')
const data = await response.json()
if (response.ok) {
toast.success('Succuss. ' + data.message)
} else {
toast.error('Failed. ' + data.message)
}
} catch (e) {
toast.error('Failed to check cookies state. ' + e.message)
} finally {
isChecking.value = false
}
}
onMounted(async () => { onMounted(async () => {
try { try {
applyPreferredColorScheme(selectedTheme.value) applyPreferredColorScheme(selectedTheme.value)
@ -186,7 +132,6 @@ onMounted(async () => {
} }
}) })
watch(selectedTheme, value => { watch(selectedTheme, value => {
try { try {
applyPreferredColorScheme(value) applyPreferredColorScheme(value)
@ -195,11 +140,4 @@ watch(selectedTheme, value => {
const reloadPage = () => window.location.reload() const reloadPage = () => window.location.reload()
const pauseDownload = () => {
if (false === confirm('Are you sure you want to pause all non-active downloads?')) {
return false
}
socket.emit('pause', {})
}
</script> </script>

87
ui/pages/index.vue
View file

@ -1,14 +1,69 @@
<template> <template>
<div> <div>
<div class="mt-1 columns is-multiline">
<div class="column is-12 is-clearfix is-unselectable">
<span class="title is-4">
<span class="icon-text">
<span class="icon"><i class="fa-solid fa-download" /></span>
<span>Downloads</span>
</span>
</span>
<div class="is-pulled-right" v-if="socket.isConnected && false === config.app.basic_mode">
<div class="field is-grouped">
<p class="control" v-if="config.app.has_cookies">
<button class="button is-purple" @click="checkCookies" v-tooltip.bottom="'Check youtube cookies status.'"
:disabled="isChecking">
<span class="icon"><i class="fas fa-cookie" :class="{ 'is-loading': isChecking }" /></span>
</button>
</p>
<p class="control">
<button class="button is-warning" @click="pauseDownload" v-if="false === config.paused"
v-tooltip.bottom="'Pause non-active downloads.'">
<span class="icon"><i class="fas fa-pause" /></span>
</button>
<button class="button is-danger" @click="socket.emit('resume', {})" v-else
v-tooltip.bottom="'Resume downloading.'">
<span class="icon"><i class="fas fa-play" /></span>
</button>
</p>
<p class="control" v-if="!config.app.basic_mode">
<button v-tooltip.bottom="'Toggle Add Form'" class="button is-primary has-tooltip-bottom"
@click="config.showForm = !config.showForm">
<span class="icon"><i class="fa-solid fa-plus" /></span>
</button>
</p>
</div>
</div>
<div class="is-hidden-mobile">
<span class="subtitle">
Queued and completed downloads are displayed here.
</span>
</div>
</div>
</div>
<NewDownload v-if="config.showForm || config.app.basic_mode" @getInfo="url => get_info = url" />
<GetInfo v-if="get_info" :link="get_info" @closeModel="get_info = ''" />
<Queue @getInfo="url => emitter('getInfo', url)" /> <Queue @getInfo="url => emitter('getInfo', url)" />
<History @getInfo="url => emitter('getInfo', url)" /> <History @getInfo="url => emitter('getInfo', url)" />
</div> </div>
</template> </template>
<script setup> <script setup>
import { request } from '~/utils/index'
const emitter = defineEmits(['getInfo']) const emitter = defineEmits(['getInfo'])
const config = useConfigStore() const config = useConfigStore()
const stateStore = useStateStore() const stateStore = useStateStore()
const socket = useSocketStore()
const toast = useToast()
const isChecking = ref(false)
const get_info = ref('')
onMounted(() => { onMounted(() => {
if (!config.app.ui_update_title) { if (!config.app.ui_update_title) {
@ -32,4 +87,36 @@ watch(() => stateStore.queue, () => {
useHead({ title: `YTPTube: ( ${Object.keys(stateStore.queue).length || 0}/${config.app.max_workers} | ${Object.keys(stateStore.history).length || 0} )` }) useHead({ title: `YTPTube: ( ${Object.keys(stateStore.queue).length || 0}/${config.app.max_workers} | ${Object.keys(stateStore.history).length || 0} )` })
}, { deep: true }) }, { deep: true })
const checkCookies = async () => {
if (true === isChecking.value) {
return
}
if (false === confirm(`Check for cookies status?`)) {
return
}
try {
isChecking.value = true
const response = await request('/api/youtube/auth')
const data = await response.json()
if (response.ok) {
toast.success('Succuss. ' + data.message)
} else {
toast.error('Failed. ' + data.message)
}
} catch (e) {
toast.error('Failed to check cookies state. ' + e.message)
} finally {
isChecking.value = false
}
}
const pauseDownload = () => {
if (false === confirm('Are you sure you want to pause all non-active downloads?')) {
return false
}
socket.emit('pause', {})
}
</script> </script>

22
ui/pages/notifications.vue
View file

@ -52,8 +52,8 @@ div.is-centered {
</div> </div>
<div class="column is-12" v-if="toggleForm"> <div class="column is-12" v-if="toggleForm">
<NotificationForm :reference="targetRef" :item="target" @cancel="resetForm(true);" @submit="updateItem" <NotificationForm :addInProgress="addInProgress" :reference="targetRef" :item="target"
:allowedEvents="allowedEvents" /> @cancel="resetForm(true);" @submit="updateItem" :allowedEvents="allowedEvents" />
</div> </div>
<div class="column is-12" v-if="!toggleForm"> <div class="column is-12" v-if="!toggleForm">
@ -133,6 +133,7 @@ const targetRef = ref('')
const toggleForm = ref(false) const toggleForm = ref(false)
const isLoading = ref(false) const isLoading = ref(false)
const initialLoad = ref(true) const initialLoad = ref(true)
const addInProgress = ref(false)
watch(() => config.app.basic_mode, async () => { watch(() => config.app.basic_mode, async () => {
if (!config.app.basic_mode) { if (!config.app.basic_mode) {
@ -247,17 +248,20 @@ const updateItem = async ({ reference, item }) => {
notifications.value.push(item) notifications.value.push(item)
} }
const status = await updateData(notifications.value) try {
const status = await updateData(notifications.value)
if (!status) { if (!status) {
return return
}
toast.success(`Notification target ${reference ? 'updated' : 'added'}.`)
resetForm(true)
} finally {
addInProgress.value = false
} }
toast.success(`Notification target ${reference ? 'updated' : 'added'}.`)
resetForm(true)
} }
const filterItem = item => { const filterItem = item => {
const { raw, ...rest } = item const { raw, ...rest } = item
return JSON.stringify(rest, null, 2) return JSON.stringify(rest, null, 2)

66
ui/pages/tasks.vue
View file

@ -31,7 +31,7 @@ div.is-centered {
</p> </p>
<p class="control"> <p class="control">
<button class="button is-info" @click="reloadContent" :class="{ 'is-loading': isLoading }" <button class="button is-info" @click="reloadContent" :class="{ 'is-loading': isLoading }"
:disabled="!socket.isConnected || isLoading"> :disabled="!socket.isConnected || isLoading" v-if="tasks && tasks.length > 0">
<span class="icon"><i class="fas fa-refresh"></i></span> <span class="icon"><i class="fas fa-refresh"></i></span>
</button> </button>
</p> </p>
@ -46,7 +46,8 @@ div.is-centered {
</div> </div>
<div class="column is-12" v-if="toggleForm"> <div class="column is-12" v-if="toggleForm">
<TaskForm :reference="taskRef" :task="task" @cancel="resetForm(true);" @submit="updateItem" /> <TaskForm :addInProgress="addInProgress" :reference="taskRef" :task="task" @cancel="resetForm(true);"
@submit="updateItem" />
</div> </div>
<div class="column is-12" v-if="!toggleForm"> <div class="column is-12" v-if="!toggleForm">
@ -73,7 +74,7 @@ div.is-centered {
<p> <p>
<span class="icon"><i class="fa-solid fa-clock" /></span> <span class="icon"><i class="fa-solid fa-clock" /></span>
<span> <span>
{{ moment(parseExpression(item.timer).next().toISOString()).fromNow() }} - {{ item.timer }} {{ tryParse(item.timer) }} - {{ item.timer }}
</span> </span>
</p> </p>
<p> <p>
@ -82,7 +83,7 @@ div.is-centered {
</p> </p>
<p> <p>
<span class="icon"><i class="fa-solid fa-file" /></span> <span class="icon"><i class="fa-solid fa-file" /></span>
<span>{{ item.output_template ?? config.app.output_template }}</span> <span>{{ item.template ?? config.app.output_template }}</span>
</p> </p>
<p> <p>
<span class="icon"><i class="fa-solid fa-tv" /></span> <span class="icon"><i class="fa-solid fa-tv" /></span>
@ -134,6 +135,7 @@ const taskRef = ref('')
const toggleForm = ref(false) const toggleForm = ref(false)
const isLoading = ref(false) const isLoading = ref(false)
const initialLoad = ref(true) const initialLoad = ref(true)
const addInProgress = ref(false)
watch(() => config.app.basic_mode, async () => { watch(() => config.app.basic_mode, async () => {
if (!config.app.basic_mode) { if (!config.app.basic_mode) {
@ -178,29 +180,39 @@ const reloadContent = async (fromMounted = false) => {
const resetForm = (closeForm = false) => { const resetForm = (closeForm = false) => {
task.value = {} task.value = {}
taskRef.value = null taskRef.value = null
addInProgress.value = false
if (closeForm) { if (closeForm) {
toggleForm.value = false toggleForm.value = false
} }
} }
const updateTasks = async tasks => { const updateTasks = async items => {
const response = await request('/api/tasks', { try {
method: 'PUT', addInProgress.value = true
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(tasks),
})
const data = await response.json() const response = await request('/api/tasks', {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(items),
})
if (200 !== response.status) { const data = await response.json()
if (200 !== response.status) {
toast.error(`Failed to update task. ${data.error}`);
return false
}
tasks.value = data
resetForm(true)
return true
} catch (e) {
toast.error(`Failed to update task. ${data.error}`); toast.error(`Failed to update task. ${data.error}`);
return false } finally {
addInProgress.value = false
} }
tasks.value = data
return true
} }
const deleteItem = async item => { const deleteItem = async item => {
@ -237,7 +249,6 @@ const updateItem = async ({ reference, task }) => {
} }
const status = await updateTasks(tasks.value) const status = await updateTasks(tasks.value)
if (!status) { if (!status) {
return return
} }
@ -246,7 +257,6 @@ const updateItem = async ({ reference, task }) => {
resetForm(true) resetForm(true)
} }
const filterItem = item => { const filterItem = item => {
const { raw, ...rest } = item const { raw, ...rest } = item
return JSON.stringify(rest, null, 2) return JSON.stringify(rest, null, 2)
@ -259,12 +269,22 @@ const editItem = item => {
} }
const calcPath = path => { const calcPath = path => {
if (!path) { let loc = config.app.download_path
return config.app.download_path
if (path) {
let loc = loc + '/' + sTrim(path, '/')
} }
return config.app.download_path + '/' + sTrim(path, '/') return loc
} }
onMounted(async () => socket.isConnected ? await reloadContent(true) : '') onMounted(async () => socket.isConnected ? await reloadContent(true) : '')
const tryParse = expression => {
try {
return moment(parseExpression(expression).next().toISOString()).fromNow()
} catch (e) {
return "Invalid"
}
}
</script> </script>

4
ui/stores/SocketStore.js
View file

@ -62,7 +62,7 @@ export const useSocketStore = defineStore('socket', () => {
stateStore.add('history', item._id, item); stateStore.add('history', item._id, item);
}); });
socket.value.on('canceled', stream => { socket.value.on('cancelled', stream => {
const item = JSON.parse(stream); const item = JSON.parse(stream);
const id = item._id const id = item._id
@ -70,7 +70,7 @@ export const useSocketStore = defineStore('socket', () => {
return return
} }
toast.info(`Download canceled: ${ag(stateStore.get('queue', id, {}), id)}`); toast.info(`Download cancelled: ${ag(stateStore.get('queue', id, {}), id)}`);
if (true === stateStore.has('queue', id)) { if (true === stateStore.has('queue', id)) {
stateStore.remove('queue', id); stateStore.remove('queue', id);

813
ui/yarn.lock
View file

File diff suppressed because it is too large Load diff