danvics/ytptube
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 1

Merge pull request #186 from arabcoders/dev

Browse source 
Major code changes.
This commit is contained in:
Abdulmohsen 2025-01-28 00:18:41 +03:00 committed by GitHub
commit f554871b9f
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
29 changed files with 2838 additions and 1237 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

11
.vscode/settings.json vendored
View file

@ -33,4 +33,15 @@
"writedescription",
"xerror"
],
"css.styleSheets": [
"ui/assets/css/*.css"
],
"spellright.language": [
"en-US-10-1."
],
"spellright.documentTypes": [
"markdown",
"latex",
"plaintext"
],
}

671
API.md Normal file
View file

@ -0,0 +1,671 @@
# HTTP API Documentation
This document describes the available endpoints and their usage. All endpoints return JSON responses (unless otherwise specified) and may require certain parameters (query, body, or path). Some endpoints serve static or streaming content (e.g., `.ts`, `.m3u8`, `.vtt` files).
> **Note**: If Basic Authentication is configured (via `auth_username` and `auth_password` in your configuration), you must include an `Authorization: Basic <base64-encoded-credentials>` header or use `?apikey=<base64-encoded-credentials>` query parameter (fallback) in every request.
- All responses use standard HTTP status codes to indicate success or error conditions.
- Endpoints support an `OPTIONS` request for CORS.
---
## Table of Contents
- [HTTP API Documentation](#http-api-documentation)
- [Table of Contents](#table-of-contents)
- [Authentication](#authentication)
- [Global Notes](#global-notes)
- [Endpoints](#endpoints)
- [GET /api/ping](#get-apiping)
- [POST /api/yt-dlp/convert](#post-apiyt-dlpconvert)
- [GET /api/yt-dlp/url/info](#get-apiyt-dlpurlinfo)
- [GET /api/history/add](#get-apihistoryadd)
- [POST /api/history](#post-apihistory)
- [DELETE /api/history](#delete-apihistory)
- [POST /api/history/{id}](#post-apihistoryid)
- [GET /api/history](#get-apihistory)
- [GET /api/tasks](#get-apitasks)
- [PUT /api/tasks](#put-apitasks)
- [GET /api/workers](#get-apiworkers)
- [POST /api/workers](#post-apiworkers)
- [PATCH /api/workers/{id}](#patch-apiworkersid)
- [DELETE /api/workers/{id}](#delete-apiworkersid)
- [GET /api/player/playlist/{file:.\*}.m3u8](#get-apiplayerplaylistfilem3u8)
- [GET /api/player/m3u8/{mode}/{file:.\*}.m3u8](#get-apiplayerm3u8modefilem3u8)
- [GET /api/player/segments/{segment}/{file:.\*}.ts](#get-apiplayersegmentssegmentfilets)
- [GET /api/player/subtitle/{file:.\*}.vtt](#get-apiplayersubtitlefilevtt)
- [GET /api/thumbnail](#get-apithumbnail)
- [GET /api/file/ffprobe/{file:.\*}](#get-apifileffprobefile)
- [GET /api/youtube/auth](#get-apiyoutubeauth)
- [GET /api/notifications](#get-apinotifications)
- [PUT /api/notifications](#put-apinotifications)
- [POST /api/notifications/test](#post-apinotificationstest)
- [Error Responses](#error-responses)
---
## Authentication
If `Config.auth_username` and `Config.auth_password` are set, all API requests must include a valid credential using one of the following:
1. HTTP Basic Auth header:
```
Authorization: Basic base64("<username>:<password>")
```
2. Query Parameter fallback:
```
?apikey=<base64("<username>:<password>")>
```
If you fail to provide valid credentials, a `401 Unauthorized` response is returned.
---
## Global Notes
- **Content-Type**
- Requests expecting a JSON body should include `Content-Type: application/json`.
- Responses typically include `Content-Type: application/json`, unless returning a file or streaming resource.
- **Status Codes**
- `200 OK` on success.
- `4xx` on client errors (e.g., missing parameters).
- `5xx` on server errors (e.g., unexpected failures).
- **Error Responses**
When an error occurs, responses follow a structure similar to:
```json
{
"error": "Description of the error",
"message": "More details if available"
}
```
with an appropriate HTTP status code.
---
## Endpoints
### GET /api/ping
**Purpose**: Health-check endpoint.
**Response**:
```json
{
"status": "pong"
}
```
---
### POST /api/yt-dlp/convert
**Purpose**: Convert a string of yt-dlp or youtube-dl CLI arguments into a JSON-friendly structure.
**Body**:
```json
{
"args": "<string>"
}
```
**Example**:
```json
{
"args": "--write-sub --embed-subs"
}
```
**Response**:
```json
{
"postprocessors": [
{
"already_have_subtitle": true,
"key": "FFmpegEmbedSubtitle"
}
],
"writesubtitles": true
}
```
or an error:
```json
{
"error": "Failed to convert args. '<reason>'."
}
```
---
### GET /api/yt-dlp/url/info
**Purpose**: Retrieves metadata (info) for a provided URL without adding it to the download queue.
**Query Parameters**:
- `?url=<video-url>`
**Response** (example):
```json
{
"title": "...",
"duration": 123.4,
"extractor": "youtube",
"_cached": {
"key": "<hash>",
"ttl": 300,
"ttl_left": 299.82,
"expires": 1690430096.429
},
...
}
```
or an error:
```json
{
"error": "text"
}
```
- If the URL is invalid or missing, returns `400 Bad Request`.
---
### GET /api/history/add
**Purpose**: **(Quick Add)** Add a single URL to the download queue via GET.
**Query Parameters**:
- `?url=<video-url>`
**Response**:
```json
{
"status": "(ok|error)",
}
```
or an error:
```json
{
"error": "text"
}
```
- If `url` is missing, returns `400 Bad Request`.
---
### POST /api/history
**Purpose**: Add one or multiple items (URLs) to the download queue via JSON body.
**Body**:
```json
// Single item
{
"url": "https://youtube.com/watch?v=...",
"preset": "default",
"folder": "folder relative to download_path",
"cookies": "...",
"template": "...",
"config": {
/* advanced config options for yt-dlp */
}
}
// Or multiple items (array of objects)
[
{
"url": "...",
"preset": "default",
...
},
{
"url": "...",
...
}
]
```
**Response**:
```json
[
{
"status": "queued",
},
...
]
```
- If any item is invalid (e.g., missing `url`), returns `400 Bad Request`.
---
### DELETE /api/history
**Purpose**: Delete items from either the "queue" or the "done" history.
**Body**:
```json
{
"ids": ["<id1>", "<id2>"],
"where": "queue" | "done",
"remove_file": true | false // optional, defaults to false, whether to delete the file from disk.
}
```
**Response**:
```json
{
"<id1>": "status",
"<id2>": "status",
...
}
```
A list or object indicating which items were removed.
---
### POST /api/history/{id}
**Purpose**: Update an item in the `completed` history.
**Path Parameter**:
- `id` = Unique item ID.
**Body** (example):
```json
{
"title": "My Custom Title",
"someOtherField": "new-value"
}
```
**Response**:
```json
{
"title": "My Custom Title",
....
}
```
or an error:
```json
{
"error": "text"
}
```
- `200 OK` with updated item if successful.
- `304 Not Modified` if nothing changed.
- `404 Not Found` if the item doesn’t exist.
- `400 Bad Request` if id is missing or the body is empty.
---
### GET /api/history
**Purpose**: Returns the download queue and the download history.
**Response**:
```json
{
"queue": [
{ ... },
...
],
"history": [
{ ... },
...
]
}
```
---
### GET /api/tasks
**Purpose**: Retrieves the scheduled tasks from the internal `Tasks` manager.
**Response**:
```json
[
{
"id": "<uuid>",
"name": "...",
"url": "...",
"folder": "...",
"preset": "...",
"timer": "<cron-expression>",
"template": "...",
"cookies": "...",
"config": { ... },
},
...
]
```
---
### PUT /api/tasks
**Purpose**: Overwrites the entire scheduled tasks list (Cron tasks).
**Body**: An array of task objects. Example:
```json
[
{
"id": "a2ae3f18-4428-4e32-9d4c-0cc45af8bb48",
"name": "My Task",
"url": "https://youtube.com/...",
"timer": "5 */2 * * *",
"cookies": "",
"config": {},
"template": "...",
"folder": "..."
},
{
"url": "https://youtube.com/...",
"timer": "*/15 * * * *"
}
]
```
If `id` or other fields are missing, they may be auto-generated or defaulted (e.g., a random ID, a default cron, etc.).
**Response**:
```json
[
{
"id": "<uuid>",
"name": "...",
"url": "...",
"timer": "...",
"cookies": "...",
"config": { ... },
"template": "...",
"folder": "..."
}
...
]
```
or on error
```json
{
"error": "text"
}
```
---
### GET /api/workers
**Purpose**: Returns the status of the worker pool and all workers.
**Response**:
```json
{
"open": true|false,
"count": 4,
"workers": [
{
"id": "worker-1",
"data": { "status": "downloading", ... }
},
{
"id": "worker-2",
"data": { "status": "Waiting for download." }
},
...
]
}
```
- `open`: Indicates if there are any available workers.
- `count`: Total number of available workers.
---
### POST /api/workers
**Purpose**: Restart the entire worker pool.
**Response**:
```json
{
"message": "Workers pool being restarted."
}
```
---
### PATCH /api/workers/{id}
**Purpose**: Restart a single worker by ID.
**Path Parameter**:
- `id` = The worker ID.
**Response**:
```json
{
"status": "restarted"
}
```
or
```json
{
"status": "in_error_state"
}
```
---
### DELETE /api/workers/{id}
**Purpose**: Stop a single worker by ID.
**Path Parameter**:
- `id` = The worker ID.
**Response**:
```json
{
"status": "stopped"
}
```
or
```json
{
"status": "in_error_state"
}
```
---
### GET /api/player/playlist/{file:.*}.m3u8
**Purpose**: Generate a playlist for a given local media file.
**Path Parameter**:
- `file` = Relative path of the media file within the `download_path`.
**Response**:
An `.m3u8` playlist.
---
### GET /api/player/m3u8/{mode}/{file:.*}.m3u8
**Purpose**: Dynamically generate an M3U8 playlist for video or subtitles.
**Path Parameters**:
- `mode`: either `video` or `subtitle`.
- `file`: relative path of the file.
**Query Parameters (when `mode=subtitle`)**:
- `duration`: The total duration of the subtitle track
**Response**:
- `Content-Type: application/x-mpegURL` containing the `.m3u8` text.
---
### GET /api/player/segments/{segment}/{file:.*}.ts
**Purpose**: Streams a single TS segment for adaptive HLS playback.
**Path Parameters**:
- `segment` = Numeric segment index.
- `file` = Relative file path.
**Query Parameters**:
- `sd` = The segment duration (float).
- `vc` = `1` or `0` (whether to convert video).
- `ac` = `1` or `0` (whether to convert audio).
**Response**:
Binary TS data (`Content-Type: video/mpegts`).
---
### GET /api/player/subtitle/{file:.*}.vtt
**Purpose**: Provides a `.vtt` (WebVTT) subtitle file for playback.
**Path Parameter**:
- `file` = Relative path of the subtitle file.
**Response**:
`Content-Type: text/vtt; charset=UTF-8`.
---
### GET /api/thumbnail
**Purpose**: Proxy/fetch a remote thumbnail image.
**Query Parameter**:
- `?url=<remote-thumbnail-url>`
**Response**:
Binary image data with the appropriate `Content-Type`.
---
### GET /api/file/ffprobe/{file:.*}
**Purpose**: Return the `ffprobe` data for a local file.
**Path Parameter**:
- `file` = Relative path to the file inside `download_path`.
**Response**:
```json
{
"streams": [...],
"format": { ... },
...
}
```
---
### GET /api/youtube/auth
**Purpose**: Checks if a valid YouTube session cookie is set (to confirm authentication).
**Response**:
```json
{
"message": "Authenticated."
}
```
Returns `200 OK` if cookies are valid, otherwise `401` with `{ "message": "Not authenticated." }`.
---
### GET /api/notifications
**Purpose**: Retrieve the configured notification targets and which event types are allowed.
**Response** (example):
```json
{
"notifications": [
{
"id": "uuid",
"name":"...",
"on": ["completed", "error",...], // empty array means all events.
"request":{
"type":"json|form",
"method":"POST|PUT",
"url":"https://...",
"headers":[
{"key":"...", "value":"..."},
...
]
}
}
}
],
"allowedTypes": ["added", "completed", "error", "cancelled", "cleared", "log_info", "log_success", ...]
}
```
---
### PUT /api/notifications
**Purpose**: Overwrites the entire list of notification targets.
**Body**: An array of notification target configurations. Example:
```json
[
{
"id": "uuid",
"name": "My Webhook",
"on": ["completed", "error"],
"request": {
"type": "json",
"method": "POST",
"url": "https://...",
"headers": [
{ "key": "Authorization", "value": "Bearer ..." }
]
}
},
{
"name": "Another Webhook",
"on": ["completed"],
"request": {
"type": "form",
"method": "PUT",
"url": "https://...",
"headers": []
}
}
...
]
```
- If `id` is not provided or is not a valid UUIDv4, it will be auto-generated.
- If the payload list is empty, all existing notifications are removed.
**Response**:
```json
{
"notifications": [
{
"id": "uuid",
"name": "...",
"on": ["completed", "error", ...],
"request": { ... }
},
...
],
"allowedTypes": ["added", "completed", "error", "cancelled", "cleared", "log_info", "log_success", ...]
}
```
---
### POST /api/notifications/test
**Purpose**: Triggers a test notification event to all configured targets.
**Response**:
```json
{
"type": "test",
"message": "This is a test notification."
}
```
---
## Error Responses
Most endpoints return standard error codes (`400`, `403`, `404`, `500`, etc.) and a JSON body on failure. For example:
```json
{
"error": "url param is required."
}
```
with `400 Bad Request`, or:
```json
{
"error": "Item not found."
}
```
with `404 Not Found`.
---

12
Pipfile.lock generated
View file

@ -142,11 +142,11 @@
},
"attrs": {
"hashes": [
"sha256:8f5c07333d543103541ba7be0e2ce16eeee8130cb0b3f9238ab904ce1e85baff",
"sha256:ac96cd038792094f438ad1f6ff80837353805ac950cd2aa0e0625ef19850c308"
"sha256:1c97078a80c814273a76b2a298a932eb681c87415c11dee0a6921de7f1b02c3e",
"sha256:c75a69e28a550a7e93789579c22aa26b0f5b83b75dc4e08fe092980051e1090a"
],
"markers": "python_version >= '3.8'",
"version": "==24.3.0"
"version": "==25.1.0"
},
"bidict": {
"hashes": [
@ -1247,12 +1247,12 @@
},
"yt-dlp": {
"hashes": [
"sha256:b8666b88e23c3fa5ee1e80920f4a9dfac7c405504a447214c0cf3d0c386edcfc",
"sha256:e8ec515d49bb62704915d13a22ee6fe03a5658d651e4e64574e3a17ee01f6e3b"
"sha256:1c9738266921ad43c568ad01ac3362fb7c7af549276fbec92bd72f140da16240",
"sha256:3e76bd896b9f96601021ca192ca0fbdd195e3c3dcc28302a3a34c9bc4979da7b"
],
"index": "pypi",
"markers": "python_version >= '3.9'",
"version": "==2025.1.15"
"version": "==2025.1.26"
}
},
"develop": {}

5
README.md
View file

@ -14,12 +14,15 @@ YTPTube started as a fork of [meTube](https://github.com/alexta69/metube), Since
* Support per link `yt-dlp JSON config or cli options`, `cookies` & `output format`.
* Queue multiple URLs separated by comma.
* A built in video player that can play any video file regardless of the format. **With support for sidecar external subtitles**.
* New `/api/add_batch` endpoint that allow multiple links to be sent.
* New `POST /api/history` endpoint that allow one or multiple links to be sent at the same time.
* New `GET /api/history/add?url=http://..` endpoint that allow to add single item via GET request.
* Completely redesigned the frontend UI.
* Switched out of binary file storage in favor of SQLite.
* Basic Authentication support.
* Support for curl_cffi, see [yt-dlp documentation](https://github.com/yt-dlp/yt-dlp?tab=readme-ov-file#impersonation)
* Support for both advanced and basic mode for WebUI.
For more API endpoints, please refer to the [API documentation](API.md).
### Tips
Your `yt-dlp` config should include the following options for optimal working conditions.

2
app/library/DataStore.py
View file

@ -106,7 +106,7 @@ class DataStore:
def getNextDownload(self) -> Download:
for key in self.dict:
if self.dict[key].started() is False and self.dict[key].is_canceled() is False:
if self.dict[key].started() is False and self.dict[key].is_cancelled() is False:
return self.dict[key]
return None

26
app/library/Download.py
View file

@ -28,15 +28,15 @@ class Download:
id: str = None
download_dir: str = None
temp_dir: str = None
output_template: str = None
output_template_chapter: str = None
template: str = None
template_chapter: str = None
ytdl_opts: dict = None
info: ItemDTO = None
default_ytdl_opts: dict = None
debug: bool = False
tempPath: str = None
emitter: Emitter = None
canceled: bool = False
cancelled: bool = False
is_live: bool = False
info_dict: dict = None
"yt-dlp metadata dict."
@ -72,15 +72,15 @@ class Download:
self.download_dir = info.download_dir
self.temp_dir = info.temp_dir
self.output_template_chapter = info.output_template_chapter
self.output_template = info.output_template
self.template = info.template
self.template_chapter = info.template_chapter
self.preset = info.preset
self.ytdl_opts = info.ytdlp_config if info.ytdlp_config else {}
self.ytdl_opts = info.config if info.config else {}
self.info = info
self.id = info._id
self.default_ytdl_opts = config.ytdl_options
self.debug = debug
self.canceled = False
self.cancelled = False
self.tmpfilename = None
self.status_queue = None
self.proc = None
@ -120,7 +120,7 @@ class Download:
{
"color": "no_color",
"paths": {"home": self.download_dir, "temp": self.tempPath},
"outtmpl": {"default": self.output_template, "chapter": self.output_template_chapter},
"outtmpl": {"default": self.template, "chapter": self.template_chapter},
"noprogress": True,
"break_on_existing": True,
"progress_hooks": [self._progress_hook],
@ -136,9 +136,9 @@ class Download:
params["verbose"] = True
params["noprogress"] = False
if self.info.ytdlp_cookies:
if self.info.cookies:
try:
data = jsonCookie(json.loads(self.info.ytdlp_cookies))
data = jsonCookie(json.loads(self.info.cookies))
if not data:
LOG.warning(
f"The cookie string that was provided for {self.info.title} is empty or not in expected spec."
@ -212,7 +212,7 @@ class Download:
if not self.started():
return False
self.canceled = True
self.cancelled = True
return True
@ -271,8 +271,8 @@ class Download:
except ValueError:
return False
def is_canceled(self) -> bool:
return self.canceled
def is_cancelled(self) -> bool:
return self.cancelled
def kill(self) -> bool:
if not self.running():

194
app/library/DownloadQueue.py
View file

@ -15,50 +15,116 @@ from .DataStore import DataStore
from .Download import Download
from .Emitter import Emitter
from .ItemDTO import ItemDTO
from .Utils import ExtractInfo, get_opts, calcDownloadPath, isDownloaded, mergeConfig
from .Singleton import Singleton
from .EventsSubscriber import Events
from .Utils import ExtractInfo, calcDownloadPath, get_opts, isDownloaded, mergeConfig
LOG = logging.getLogger("DownloadQueue")
TYPE_DONE: str = "done"
TYPE_QUEUE: str = "queue"
class DownloadQueue:
class DownloadQueue(metaclass=Singleton):
"""
DownloadQueue class is a singleton class that manages the download queue and the download history.
"""
TYPE_DONE: str = "done"
"""Queue type for completed downloads."""
TYPE_QUEUE: str = "queue"
"""Queue type for pending downloads."""
paused: asyncio.Event
event: asyncio.Event | None = None
pool: AsyncPool | None = None
"""Event to pause the download queue."""
def __init__(self, emitter: Emitter, connection: Connection):
self.config = Config.get_instance()
self.emitter = emitter
self.done = DataStore(type=TYPE_DONE, connection=connection)
self.queue = DataStore(type=TYPE_QUEUE, connection=connection)
event: asyncio.Event
"""Event to signal the download queue to start downloading."""
pool: AsyncPool | None = None
"""Pool of workers to download the files."""
_instance = None
"""Instance of the DownloadQueue."""
def __init__(self, connection: Connection, emitter: Emitter | None = None, config: Config | None = None):
DownloadQueue._instance = self
self.config = config or Config.get_instance()
self.emitter = emitter or Emitter.get_instance()
self.done = DataStore(type=DownloadQueue.TYPE_DONE, connection=connection)
self.queue = DataStore(type=DownloadQueue.TYPE_QUEUE, connection=connection)
self.done.load()
self.queue.load()
self.paused = asyncio.Event()
self.paused.set()
self.event = asyncio.Event()
@staticmethod
def get_instance():
"""
Get the instance of the DownloadQueue.
Returns:
DownloadQueue: The instance of the DownloadQueue
"""
if not DownloadQueue._instance:
DownloadQueue._instance = DownloadQueue()
return DownloadQueue._instance
async def test(self) -> bool:
"""
Test the datastore connection to the database.
Returns:
bool: True if the test is successful, False otherwise.
"""
await self.done.test()
return True
async def initialize(self):
self.event = asyncio.Event()
"""
Initialize the download queue.
"""
LOG.info(
f"Using '{self.config.max_workers}' worker/s for downloading. Can be configured via `YTP_MAX_WORKERS` environment variable."
)
asyncio.create_task(self.__download_pool(), name="download_pool")
def pause(self):
def pause(self) -> bool:
"""
Pause the download queue.
Returns:
bool: True if the download is paused, False otherwise
"""
if self.paused.is_set():
self.paused.clear()
LOG.warning(f"Download paused at. {datetime.datetime.now().isoformat()}")
return True
def resume(self):
return False
def resume(self) -> bool:
"""
Resume the download queue.
Returns:
bool: True if the download is resumed, False otherwise
"""
if not self.paused.is_set():
self.paused.set()
LOG.warning(f"Downloading resumed at. {datetime.datetime.now().isoformat()}")
return True
return False
def isPaused(self) -> bool:
"""
Check if the download queue is paused.
Returns:
bool: True if the download queue is paused, False otherwise
"""
return False if self.paused.is_set() else True
async def __add_entry(
@ -66,11 +132,26 @@ class DownloadQueue:
entry: dict,
preset: str,
folder: str,
ytdlp_config: dict = {},
ytdlp_cookies: str = "",
output_template: str = "",
config: dict = {},
cookies: str = "",
template: str = "",
already=None,
):
"""
Add an entry to the download queue.
Args:
entry (dict): The entry to add to the download queue.
preset (str): The preset to use for the download.
folder (str): The folder to save the download to.
config (dict): The yt-dlp configuration to use for the download.
cookies (str): The cookies to use for the download.
template (str): The output template to use for the download.
already (set): The set of already downloaded items.
Returns:
dict: The status of the operation.
"""
if not entry:
return {"status": "error", "msg": "Invalid/empty data was given."}
@ -99,9 +180,9 @@ class DownloadQueue:
entry=etr,
preset=preset,
folder=folder,
ytdlp_config=ytdlp_config,
ytdlp_cookies=ytdlp_cookies,
output_template=output_template,
config=config,
cookies=cookies,
template=template,
already=already,
)
)
@ -168,10 +249,10 @@ class DownloadQueue:
folder=folder,
download_dir=download_dir,
temp_dir=self.config.temp_path,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config,
output_template=output_template if output_template else self.config.output_template,
output_template_chapter=self.config.output_template_chapter,
cookies=cookies,
config=config,
template=template if template else self.config.output_template,
template_chapter=self.config.output_template_chapter,
datetime=formatdate(time.time()),
error=error,
is_live=is_live,
@ -182,24 +263,23 @@ class DownloadQueue:
for property, value in entry.items():
if property.startswith("playlist"):
dl.output_template = str(dl.output_template).replace(f"%({property})s", str(value))
dl.template = str(dl.template).replace(f"%({property})s", str(value))
dlInfo: Download = Download(info=dl, info_dict=entry, debug=bool(self.config.ytdl_debug))
if dlInfo.info.live_in or "is_upcoming" == entry.get("live_status", None):
dlInfo.info.status = "not_live"
itemDownload = self.done.put(dlInfo)
NotifyEvent = "completed"
NotifyEvent = Events.COMPLETED
elif self.config.allow_manifestless is False and is_manifestless is True:
dlInfo.info.status = "error"
dlInfo.info.error = "Video is in post-live manifestless mode."
itemDownload = self.done.put(dlInfo)
NotifyEvent = "completed"
NotifyEvent = Events.COMPLETED
else:
NotifyEvent = "added"
NotifyEvent = Events.ADDED
itemDownload = self.queue.put(dlInfo)
if self.event:
self.event.set()
self.event.set()
asyncio.create_task(
self.emitter.emit(NotifyEvent, itemDownload.info), name=f"notifier-{NotifyEvent}-{itemDownload.info.id}"
@ -211,9 +291,9 @@ class DownloadQueue:
url=str(entry.get("url")),
preset=preset,
folder=folder,
ytdlp_config=ytdlp_config,
ytdlp_cookies=ytdlp_cookies,
output_template=output_template,
config=config,
cookies=cookies,
template=template,
already=already,
)
@ -224,25 +304,25 @@ class DownloadQueue:
url: str,
preset: str,
folder: str,
ytdlp_config: dict = {},
ytdlp_cookies: str = "",
output_template: str = "",
config: dict = {},
cookies: str = "",
template: str = "",
already=None,
):
ytdlp_config = ytdlp_config if ytdlp_config else {}
config = config if config else {}
folder = str(folder) if folder else ""
filePath = calcDownloadPath(basePath=self.config.download_path, folder=folder)
LOG.info(
f"Adding 'URL: {url}' to 'Folder: {filePath}' with 'Preset: {preset}' 'Naming: {output_template}', 'Cookies: {ytdlp_cookies}' 'YTConfig: {ytdlp_config}'."
f"Adding 'URL: {url}' to 'Folder: {filePath}' with 'Preset: {preset}' 'Naming: {template}', 'Cookies: {cookies}' 'YTConfig: {config}'."
)
if isinstance(ytdlp_config, str):
if isinstance(config, str):
try:
ytdlp_config = json.loads(ytdlp_config)
config = json.loads(config)
except Exception as e:
LOG.error(f"Unable to load '{ytdlp_config=}'. {str(e)}")
LOG.error(f"Unable to load '{config=}'. {str(e)}")
return {"status": "error", "msg": f"Failed to parse json yt-dlp config. {str(e)}"}
already = set() if already is None else already
@ -267,7 +347,7 @@ class DownloadQueue:
fut=asyncio.get_running_loop().run_in_executor(
None,
ExtractInfo,
get_opts(preset, mergeConfig(self.config.ytdl_options, ytdlp_config)),
get_opts(preset, mergeConfig(self.config.ytdl_options, config)),
url,
bool(self.config.ytdl_debug),
),
@ -297,9 +377,9 @@ class DownloadQueue:
entry=entry,
preset=preset,
folder=folder,
ytdlp_config=ytdlp_config,
ytdlp_cookies=ytdlp_cookies,
output_template=output_template,
config=config,
cookies=cookies,
template=template,
already=already,
)
@ -326,9 +406,9 @@ class DownloadQueue:
await item.close()
LOG.debug(f"Deleting from queue {itemMessage}")
self.queue.delete(id)
asyncio.create_task(self.emitter.canceled(dl=item.info.serialize()), name=f"notifier-c-{id}")
item.info.status = "canceled"
item.info.error = "Canceled by user."
asyncio.create_task(self.emitter.cancelled(dl=item.info.serialize()), name=f"notifier-c-{id}")
item.info.status = "cancelled"
item.info.error = "Cancelled by user."
self.done.put(item)
asyncio.create_task(self.emitter.completed(dl=item.info.serialize()), name=f"notifier-d-{id}")
LOG.info(f"Deleted from queue {itemMessage}")
@ -384,6 +464,12 @@ class DownloadQueue:
return status
def get(self) -> dict[str, list[dict[str, ItemDTO]]]:
"""
Get the download queue and the download history.
Returns:
dict: The download queue and the download history.
"""
items = {"queue": {}, "done": {}}
for k, v in self.queue.saved_items():
@ -444,7 +530,7 @@ class DownloadQueue:
LOG.debug(f"Pushing {entry=} to executor.")
if entry.started() is False and entry.is_canceled() is False:
if entry.started() is False and entry.is_cancelled() is False:
await self.pool.push(id=entry.info._id, entry=entry)
LOG.debug(f"Pushed {entry=} to executor.")
await asyncio.sleep(1)
@ -458,7 +544,7 @@ class DownloadQueue:
try:
await entry.start(self.emitter)
if entry.info.status != "finished":
if "finished" != entry.info.status:
if entry.tmpfilename and os.path.isfile(entry.tmpfilename):
try:
os.remove(entry.tmpfilename)
@ -474,10 +560,10 @@ class DownloadQueue:
LOG.debug(f"Download '{id}' is done. Removing from queue.")
self.queue.delete(key=id)
if entry.is_canceled() is True:
asyncio.create_task(self.emitter.canceled(dl=entry.info.serialize()), name=f"notifier-c-{id}")
entry.info.status = "canceled"
entry.info.error = "Canceled by user."
if entry.is_cancelled() is True:
asyncio.create_task(self.emitter.cancelled(dl=entry.info.serialize()), name=f"notifier-c-{id}")
entry.info.status = "cancelled"
entry.info.error = "Cancelled by user."
self.done.put(value=entry)
asyncio.create_task(self.emitter.completed(dl=entry.info.serialize()), name=f"notifier-d-{id}")

101
app/library/Emitter.py
View file

@ -1,76 +1,129 @@
import asyncio
import logging
from typing import Awaitable
from .Singleton import Singleton
from .EventsSubscriber import Events
LOG = logging.getLogger("Emitter")
class Emitter:
class Emitter(metaclass=Singleton):
"""
This class is used to emit events to the registered emitters.
"""
emitters: list[callable] = [] # type: ignore
emitters: list[(str, Awaitable)] = []
"""The emitters for the events."""
def add_emitter(self, emitter: callable): # type: ignore
_instance = None
def __init__(self):
Emitter._instance = self
@staticmethod
def get_instance():
"""
Get the instance of the Emitter.
Returns:
Emitter: The instance of the Emitter
"""
if not Emitter._instance:
Emitter._instance = Emitter()
return Emitter._instance
def add_emitter(self, emitter: list[Awaitable] | Awaitable, local: bool = False) -> "Emitter":
"""
Add an emitter to the list of emitters.
Args:
emitter (function): The emitter function. The function must return a coroutine or None.
emitter (Awaitable|list[Awaitable]): The emitter function. The function must return a coroutine or None.
local (bool): Mark the emitter as target for local events.
Returns:
Emitter: The instance of the Emitter
"""
self.emitters.append(emitter)
if not isinstance(emitter, list):
emitter = [emitter]
async def added(self, dl: dict, **kwargs):
await self.emit("added", dl, **kwargs)
for e in emitter:
self.emitters.append((local, e))
async def updated(self, dl: dict, **kwargs):
await self.emit("updated", dl, **kwargs)
return self
async def completed(self, dl: dict, **kwargs):
await self.emit("completed", dl, **kwargs)
async def added(self, dl: dict, local: bool = False, **kwargs):
await self.emit(Events.ADDED, data=dl, local=local, **kwargs)
async def canceled(self, dl: dict, **kwargs):
await self.emit("canceled", dl, **kwargs)
async def updated(self, dl: dict, local: bool = False, **kwargs):
await self.emit(Events.UPDATED, data=dl, local=local, **kwargs)
async def cleared(self, dl: dict | None = None, **kwargs):
await self.emit("cleared", dl, **kwargs)
async def completed(self, dl: dict, local: bool = False, **kwargs):
await self.emit(Events.COMPLETED, data=dl, local=local, **kwargs)
async def error(self, message: str, data: dict = {}, **kwargs):
async def cancelled(self, dl: dict, local: bool = False, **kwargs):
await self.emit(Events.CANCELLED, data=dl, local=local, **kwargs)
async def cleared(self, dl: dict | None = None, local: bool = False, **kwargs):
await self.emit(Events.CLEARED, data=dl, local=local, **kwargs)
async def error(self, message: str, data: dict = {}, local: bool = False, **kwargs):
msg = {"type": "error", "message": message, "data": {}}
if data:
msg.update({"data": data})
await self.emit("error", msg, **kwargs)
await self.emit(Events.ERROR, data=msg, local=local, **kwargs)
async def info(self, message: str, data: dict = {}, **kwargs):
async def info(self, message: str, data: dict = {}, local: bool = False, **kwargs):
msg = {"type": "info", "message": message, "data": {}}
if data:
msg.update({"data": data})
await self.emit("log_info", msg, **kwargs)
await self.emit(Events.LOG_INFO, data=msg, local=local, **kwargs)
async def success(self, message: str, data: dict = {}, **kwargs):
async def success(self, message: str, data: dict = {}, local: bool = False, **kwargs):
msg = {"type": "success", "message": message, "data": {}}
if data:
msg.update({"data": data})
await self.emit("log_success", msg, **kwargs)
await self.emit(Events.LOG_SUCCESS, data=msg, local=local, **kwargs)
async def emit(self, event: str, data, **kwargs):
async def emit(self, event: str, data, local: bool = False, **kwargs):
"""
Emit an event.
Args:
event (str): The event to emit.
data (dict): The data to send with the event.
local (bool): If the event should be sent to the local emitters only.
kwargs: Additional arguments to pass to the emitters.
Returns:
None
"""
tasks = []
for emitter in self.emitters:
try:
_ret = emitter(event, data, **kwargs)
isLocal, callback = emitter
if local and not isLocal:
continue
_ret = callback(event, data, **kwargs)
if _ret:
if isinstance(_ret, list):
tasks.extend(_ret)
else:
tasks.append(_ret)
except Exception as e:
LOG.error(f"Emitter '{emitter}' failed with error '{e}'.")
LOG.error(f"Emitter '{callback}' failed with error '{e}'.")
if len(tasks) < 1:
return
try:
await asyncio.wait_for(asyncio.gather(*tasks), timeout=60)
except asyncio.CancelledError:
LOG.error(f"Cancelled sending event '{event}'.")
except asyncio.TimeoutError:
LOG.error(f"Timed out sending event '{event}'.")
except Exception as e:
LOG.exception(e)
LOG.error(f"Failed to send event '{event}'. '{str(e)}'.")

187
app/library/EventsSubscriber.py Normal file
View file

@ -0,0 +1,187 @@
import asyncio
import logging
from typing import Awaitable
from dataclasses import dataclass
from .Singleton import Singleton
LOG = logging.getLogger("EventsSubscriber")
class Events:
"""
The events that can be emitted.
"""
ADDED = "added"
UPDATED = "updated"
COMPLETED = "completed"
CANCELLED = "cancelled"
CLEARED = "cleared"
ERROR = "error"
LOG_INFO = "log_info"
LOG_SUCCESS = "log_success"
INITIAL_DATA = "initial_data"
YTDLP_CONVERT = "ytdlp_convert"
ITEM_DELETE = "item_delete"
ITEM_CANCEL = "item_cancel"
STATUS = "status"
CLI_CLOSE = "cli_close"
CLI_OUTPUT = "cli_output"
UPDATE = "update"
TEST = "test"
UPDATED = "updated"
ADD_URL = "add_url"
CLI_POST = "cli_post"
PAUSED = "paused"
TEST = "test"
TASKS_ADD = "task_add"
TASK_DISPATCHED = "task_dispatched"
TASK_FINISHED = "task_finished"
TASK_ERROR = "task_error"
@dataclass(kw_only=True)
class Event:
id: str
data: dict
class EventsSubscriber(metaclass=Singleton):
"""
This class is used to subscribe to and emit events to the registered listeners.
"""
_instance = None
"""the instance of the EventsSubscriber"""
_listeners: dict[str, list[str, Awaitable]] = {}
"""The listeners for the events."""
def __init__(self):
EventsSubscriber._instance = self
@staticmethod
def get_instance() -> "EventsSubscriber":
"""
Get the instance of the EventsSubscriber.
Returns:
EventsSubscriber: The instance of the EventsSubscriber
"""
if not EventsSubscriber._instance:
EventsSubscriber._instance = EventsSubscriber()
return EventsSubscriber._instance
def subscribe(self, event: str | list | tuple, id: str, callback: Awaitable) -> "EventsSubscriber":
"""
Subscribe to an event.
Args:
event (str): The event to subscribe to.
id (str|None): The id of the subscriber, if None a random uuid will be generated.
callback (Awaitable): The function to call. Must be a coroutine.
Returns:
EventsSubscriber: The instance of the EventsSubscriber
"""
if isinstance(event, str):
event = [event]
for e in event:
if e not in self._listeners:
self._listeners[e] = {}
self._listeners[e][id] = callback
return self
def unsubscribe(self, event: str | list | tuple, id: str) -> "EventsSubscriber":
"""
Unsubscribe from an event.
Args:
event (str): The event to unsubscribe from.
id (str): The id of the subscriber.
Returns:
EventsSubscriber: The instance of the EventsSubscriber
"""
if isinstance(event, str):
event = [event]
for e in event:
if e in self._listeners and id in self._listeners[e]:
del self._listeners[e][id]
return self
def emit_sync(self, event: str, *args, **kwargs):
"""
Emit an event synchronously.
Args:
event (str): The event to emit.
*args: The arguments to pass to the event.
**kwargs: The keyword arguments to pass to the event.
Returns:
list: The results are the return values of the coroutines. If the coroutine raises an exception,
the exception is caught and logged. If event does not exist, an empty list is returned.
"""
if event not in self._listeners:
return []
results = []
for id, callback in self._listeners[event].items():
try:
if "data" not in kwargs or not isinstance(kwargs["data"], Event):
data = Event(id=id, data={"args": args if args else [], **kwargs})
else:
data = kwargs["data"]
results.append(asyncio.get_event_loop().run_until_complete(callback(event, data)))
except Exception as e:
LOG.error(f"Failed to emit event '{event}' to '{id}'. Error message '{str(e)}'.")
LOG.exception(e)
return results
def emit(self, event: str, *args, **kwargs):
"""
Emit an event.
Args:
event (str): The event to emit.
*args: The arguments to pass to the event.
**kwargs: The keyword arguments to pass to the event.
Returns:
Awaitable: The task that was created to run the event.
"""
if event not in self._listeners:
return
tasks = []
for id, callback in self._listeners[event].items():
try:
if args and isinstance(args[0], Event):
data = args[0]
elif "data" in kwargs and isinstance(kwargs["data"], Event):
data = kwargs["data"]
else:
data = Event(id=id, data={"args": args if args else [], **kwargs})
tasks.append(asyncio.create_task(callback(event, data)))
except Exception as e:
LOG.error(f"Failed to emit event '{event}' to '{id}'. Error message '{str(e)}'.")
LOG.exception(e)
return asyncio.gather(*tasks)

657
app/library/HttpAPI.py
View file

@ -1,3 +1,4 @@
import asyncio
import base64
import functools
import hmac
@ -6,9 +7,10 @@ import logging
import os
import random
import time
import uuid
from datetime import datetime
from pathlib import Path
import uuid
from typing import Awaitable
import httpx
import magic
@ -21,13 +23,15 @@ from .config import Config
from .DownloadQueue import DownloadQueue
from .Emitter import Emitter
from .encoder import Encoder
from .EventsSubscriber import Events
from .ffprobe import ffprobe
from .M3u8 import M3u8
from .Notifications import Notification, NotificationEvents
from .Playlist import Playlist
from .Segments import Segments
from .Subtitle import Subtitle
from .Utils import StreamingError, arg_converter, calcDownloadPath, getVideoInfo, validate_url, validateUUID
from .Notifications import Notification
from .Tasks import Task, Tasks
from .Utils import StreamingError, arg_converter, calcDownloadPath, getVideoInfo, validate_url, validate_uuid
LOG = logging.getLogger("http_api")
MIME = magic.Magic(mime=True)
@ -43,23 +47,34 @@ class HttpAPI(common):
".ico": "image/x-icon",
}
def __init__(self, queue: DownloadQueue, emitter: Emitter, encoder: Encoder, load_tasks: callable):
super().__init__(queue=queue, encoder=encoder)
def __init__(
self,
queue: DownloadQueue | None = None,
emitter: Emitter | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
):
self.queue = queue or DownloadQueue.get_instance()
self.emitter = emitter or Emitter.get_instance()
self.encoder = encoder or Encoder()
self.config = config or Config.get_instance()
self.rootPath = str(Path(__file__).parent.parent.parent)
self.config = Config.get_instance()
self.routes = web.RouteTableDef()
self.encoder = encoder
self.emitter = emitter
self.queue = queue
self.cache = Cache()
self.load_tasks = load_tasks
def route(method: str, path: str):
super().__init__(queue=self.queue, encoder=self.encoder, config=self.config)
def route(method: str, path: str) -> Awaitable:
"""
Decorator to mark a method as an HTTP route handler.
Args:
method (str): The HTTP method.
path (str): The path to the route.
Returns:
Awaitable: The decorated function.
"""
def decorator(func):
@ -73,11 +88,49 @@ class HttpAPI(common):
return decorator
def attach(self, app: web.Application) -> "HttpAPI":
"""
Attach the routes to the application.
Args:
app (web.Application): The application to attach the routes to.
Returns:
HttpAPI: The instance of the HttpAPI.
"""
if self.config.auth_username and self.config.auth_password:
app.middlewares.append(HttpAPI.basic_auth(self.config.auth_username, self.config.auth_password))
self.add_routes(app)
async def on_prepare(request: Request, response: Response):
if "Server" in response.headers:
del response.headers["Server"]
if "Origin" in request.headers:
response.headers["Access-Control-Allow-Origin"] = request.headers["Origin"]
response.headers["Access-Control-Allow-Headers"] = "Content-Type, Authorization"
response.headers["Access-Control-Allow-Methods"] = "GET, PATCH, PUT, POST, DELETE"
try:
app.on_response_prepare.append(on_prepare)
except Exception as e:
LOG.exception(e)
return self
async def staticFile(self, req: Request) -> Response:
"""
Preload static files from the ui/exported folder.
Args:
req (Request): The request object.
Returns:
Response: The response object.
"""
path = req.path
if req.path not in self.staticHolder:
return web.json_response({"error": "File not found.", "file": path}, status=web.HTTPNotFound.status_code)
@ -94,10 +147,17 @@ class HttpAPI(common):
status=web.HTTPOk.status_code,
)
def preloadStatic(self, app: web.Application):
def preloadStatic(self, app: web.Application) -> "HttpAPI":
"""
Preload static files from the ui/exported folder.
Args:
app (web.Application): The application to attach the routes to.
Returns:
HttpAPI: The instance of the HttpAPI.
"""
staticDir = os.path.join(self.rootPath, "ui", "exported")
if not os.path.exists(staticDir):
raise ValueError(f"Could not find the frontend UI static assets. '{staticDir}'.")
@ -130,22 +190,28 @@ class HttpAPI(common):
preloaded += 2
if preloaded < 1:
message = f"Could not find the frontend UI static assets. '{staticDir}'."
message = f"Failed to find any static files in '{staticDir}'."
if self.config.ignore_ui:
LOG.warning(message)
return
return self
raise ValueError(message)
LOG.info(f"Preloaded '{preloaded}' static files.")
def attach(self, app: web.Application):
if self.config.auth_username and self.config.auth_password:
app.middlewares.append(HttpAPI.basic_auth(self.config.auth_username, self.config.auth_password))
return self
self.add_routes(app)
def add_routes(self, app: web.Application) -> "HttpAPI":
"""
Add the routes to the application.
Args:
app (web.Application): The application to attach the routes to.
Returns:
HttpAPI: The instance of the HttpAPI.
"""
def add_routes(self, app: web.Application):
for attr_name in dir(self):
method = getattr(self, attr_name)
if hasattr(method, "_http_method") and hasattr(method, "_http_path"):
@ -155,30 +221,33 @@ class HttpAPI(common):
self.routes.route(method._http_method, f"/{http_path}")(method)
async def on_prepare(request: Request, response: Response):
if "Server" in response.headers:
del response.headers["Server"]
if "Origin" in request.headers:
response.headers["Access-Control-Allow-Origin"] = request.headers["Origin"]
response.headers["Access-Control-Allow-Headers"] = "Content-Type, Authorization"
response.headers["Access-Control-Allow-Methods"] = "GET, PATCH, PUT, POST, DELETE"
self.routes.static("/api/download/", self.config.download_path)
self.preloadStatic(app)
try:
app.add_routes(self.routes)
app.on_response_prepare.append(on_prepare)
except ValueError as e:
if "ui/exported" in str(e):
raise RuntimeError(f"Could not find the frontend UI static assets. '{e}'.") from e
raise e
def basic_auth(username: str, password: str):
def basic_auth(username: str, password: str) -> Awaitable:
"""
Middleware to handle basic authentication.
Args:
username (str): The username.
password (str): The password.
Returns:
Awaitable: The middleware handler.
"""
@web.middleware
async def middleware_handler(request: Request, handler: RequestHandler) -> Response:
auth_header = request.headers.get("Authorization")
if auth_header is None and request.query.get("apikey") is not None:
auth_header = f"Basic {request.query.get('apikey')}"
if auth_header is None:
return web.json_response(
@ -214,20 +283,47 @@ class HttpAPI(common):
@route("OPTIONS", "/{path:.*}")
async def add_coors(self, _: Request) -> Response:
"""
Add CORS headers to the response.
Args:
_: The request object.
Returns:
Response: The response object.
"""
return web.json_response(data={"status": "ok"}, status=web.HTTPOk.status_code)
@route("GET", "api/ping")
async def ping(self, _) -> Response:
async def ping(self, _: Request) -> Response:
"""
Ping the server.
Args:
_: The request object.
Returns:
Response: The response object.
"""
await self.queue.test()
return web.json_response(data={"status": "pong"}, status=web.HTTPOk.status_code)
@route("POST", "api/yt-dlp/convert")
async def yt_dlp_convert(self, request: Request) -> Response:
"""
Convert the yt-dlp args to a dict.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
post = await request.json()
args: str | None = post.get("args")
if not args:
return web.json_response(data={"error": "args is required."}, status=web.HTTPBadRequest.status_code)
return web.json_response(data={"error": "args param is required."}, status=web.HTTPBadRequest.status_code)
try:
return web.json_response(data=arg_converter(args), status=web.HTTPOk.status_code)
@ -239,121 +335,25 @@ class HttpAPI(common):
data={"error": f"Failed to convert args. '{err}'."}, status=web.HTTPBadRequest.status_code
)
@route("POST", "api/add")
async def add_url(self, request: Request) -> Response:
post = await request.json()
url: str = post.get("url")
if not url:
return web.json_response(data={"error": "url is required."}, status=web.HTTPBadRequest.status_code)
preset: str = str(post.get("preset", self.config.default_preset))
folder: str = str(post.get("folder")) if post.get("folder") else ""
ytdlp_cookies: str = str(post.get("ytdlp_cookies")) if post.get("ytdlp_cookies") else ""
output_template: str = str(post.get("output_template")) if post.get("output_template") else ""
ytdlp_config = post.get("ytdlp_config")
if isinstance(ytdlp_config, str) and ytdlp_config:
try:
ytdlp_config = json.loads(ytdlp_config)
except Exception as e:
LOG.error(f"Failed to parse json yt-dlp config for '{url}'. {str(e)}")
return web.json_response(
data={"error": f"Failed to parse json yt-dlp config. {str(e)}"},
status=web.HTTPBadRequest.status_code,
)
status = await self.add(
url=url,
preset=preset,
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config if isinstance(ytdlp_config, dict) else {},
output_template=output_template,
)
return web.json_response(data=status, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("GET", "api/tasks")
async def tasks(self, _: Request) -> Response:
tasks_file: str = os.path.join(self.config.config_path, "tasks.json")
if os.path.exists(tasks_file):
try:
with open(tasks_file, "r") as f:
tasks = json.load(f)
except Exception as e:
LOG.exception(e)
return web.json_response(
{"error": "Failed to load tasks."}, status=web.HTTPInternalServerError.status_code
)
else:
tasks = []
return web.json_response(data=tasks, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("PUT", "api/tasks")
async def add_tasks(self, request: Request) -> Response:
tasks_file: str = os.path.join(self.config.config_path, "tasks.json")
post = await request.json()
if not isinstance(post, list):
return web.json_response(
{"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code,
)
tasks: list = []
for item in post:
if not isinstance(item, dict):
return web.json_response(
{"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code,
)
if not item.get("url"):
return web.json_response(
{"error": "url is required.", "data": post}, status=web.HTTPBadRequest.status_code
)
if not item.get("id", None):
item["id"] = str(uuid.uuid4())
if not item.get("timer", None):
item["timer"] = f"{random.randint(1,59)} */1 * * *"
tasks.append(item)
try:
with open(tasks_file, "w") as f:
json.dump(tasks, f, indent=4)
except Exception as e:
LOG.exception(e)
return web.json_response({"error": "Failed to save tasks."}, status=web.HTTPInternalServerError.status_code)
try:
self.load_tasks()
except Exception as e:
LOG.exception(e)
return web.json_response(
{"error": "Failed to reload tasks.", "message": str(e)},
status=web.HTTPInternalServerError.status_code,
)
return web.json_response(data=tasks, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("GET", "api/url/info")
@route("GET", "api/yt-dlp/url/info")
async def get_info(self, request: Request) -> Response:
"""
Get the video info.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
url = request.query.get("url")
if not url:
return web.json_response(data={"error": "URL is required."}, status=web.HTTPForbidden.status_code)
return web.json_response(data={"error": "URL is required."}, status=web.HTTPBadRequest.status_code)
try:
validate_url(url)
except ValueError as e:
return web.json_response(data={"error": str(e)}, status=web.HTTPForbidden.status_code)
return web.json_response(data={"error": str(e)}, status=web.HTTPBadRequest.status_code)
try:
key = self.cache.hash(url)
@ -393,52 +393,165 @@ class HttpAPI(common):
status=web.HTTPInternalServerError.status_code,
)
@route("POST", "api/add_batch")
async def add_batch(self, request: Request) -> Response:
status = {}
@route("GET", "api/history/add")
async def quick_add(self, request: Request) -> Response:
"""
Add a URL to the download queue.
post = await request.json()
if not isinstance(post, list):
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
url: str | None = request.query.get("url")
if not url:
return web.json_response(data={"error": "url param is required."}, status=web.HTTPBadRequest.status_code)
try:
status = await self.add(**self.format_item({"url": url}))
except ValueError as e:
return web.json_response(data={"error": str(e)}, status=web.HTTPBadRequest.status_code)
return web.json_response(data=status, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("POST", "api/history")
async def history_item_add(self, request: Request) -> Response:
"""
Add a URL to the download queue.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
data = await request.json()
if isinstance(data, dict):
data = [data]
for item in data:
try:
self.format_item(item)
except ValueError as e:
return web.json_response(data={"error": str(e), "data": item}, status=web.HTTPBadRequest.status_code)
return web.json_response(
data=await asyncio.wait_for(
fut=asyncio.gather(*[self.add(**self.format_item(item)) for item in data]),
timeout=None,
),
status=web.HTTPOk.status_code,
dumps=self.encoder.encode,
)
@route("GET", "api/tasks")
async def tasks(self, _: Request) -> Response:
"""
Get the tasks.
Args:
_: The request object.
Returns:
Response: The response object.
"""
return web.json_response(
data=Tasks.get_instance().getTasks(), status=web.HTTPOk.status_code, dumps=self.encoder.encode
)
@route("PUT", "api/tasks")
async def tasks_add(self, request: Request) -> Response:
"""
Add tasks to the queue.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
data = await request.json()
if not isinstance(data, list):
return web.json_response(
data={"error": "Invalid request body expecting list with dicts."},
{"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code,
)
for item in post:
tasks: list = []
ins = Tasks.get_instance()
for item in data:
if not isinstance(item, dict):
return web.json_response(
data={"error": "Invalid request body expecting list with dicts."},
{"error": "Invalid request body expecting list with dicts."},
status=web.HTTPBadRequest.status_code,
)
if not item.get("url"):
return web.json_response(
data={"error": "url is required.", "data": post}, status=web.HTTPBadRequest.status_code
{"error": "url is required.", "data": item}, status=web.HTTPBadRequest.status_code
)
for item in post:
status[item.get("url")] = await self.add(
url=item.get("url"),
preset=item.get("preset", "default"),
folder=item.get("folder"),
ytdlp_cookies=item.get("ytdlp_cookies"),
ytdlp_config=item.get("ytdlp_config"),
output_template=item.get("output_template"),
if not item.get("id", None) or not validate_uuid(item.get("id"), version=4):
item["id"] = str(uuid.uuid4())
if not item.get("timer", None) or str(item.get("timer")).strip() == "":
item["timer"] = f"{random.randint(1,59)} */1 * * *"
if not item.get("cookies", None):
item["cookies"] = ""
if not item.get("config", None) or str(item.get("config")).strip() == "":
item["config"] = {}
if not item.get("template", None):
item["template"] = str()
try:
ins.validate(item)
except ValueError as e:
return web.json_response(
{"error": f"Failed to validate task '{item.get('name')}'. '{str(e)}'"},
status=web.HTTPBadRequest.status_code,
)
tasks.append(Task(**item))
try:
tasks = ins.save(tasks=tasks).load().getTasks()
except Exception as e:
LOG.exception(e)
return web.json_response(
{"error": "Failed to save tasks.", "message": str(e)},
status=web.HTTPInternalServerError.status_code,
)
return web.json_response(data=status, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
return web.json_response(data=tasks, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("DELETE", "api/delete")
async def delete(self, request: Request) -> Response:
post = await request.json()
ids = post.get("ids")
where = post.get("where")
@route("DELETE", "api/history")
async def history_delete(self, request: Request) -> Response:
"""
Delete an item from the queue.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
data = await request.json()
ids = data.get("ids")
where = data.get("where")
if not ids or where not in ["queue", "done"]:
return web.json_response(
data={"error": "ids and where are required."}, status=web.HTTPBadRequest.status_code
)
remove_file: bool = bool(post.get("remove_file", True))
remove_file: bool = bool(data.get("remove_file", True))
return web.json_response(
data=await (self.queue.cancel(ids) if where == "queue" else self.queue.clear(ids, remove_file=remove_file)),
@ -447,7 +560,16 @@ class HttpAPI(common):
)
@route("POST", "api/history/{id}")
async def update_item(self, request: Request) -> Response:
async def history_item_update(self, request: Request) -> Response:
"""
Update an item in the history.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
id: str = request.match_info.get("id")
if not id:
return web.json_response(data={"error": "id is required."}, status=web.HTTPBadRequest.status_code)
@ -475,7 +597,7 @@ class HttpAPI(common):
if updated:
self.queue.done.put(item)
await self.emitter.emit("update", item.info)
await self.emitter.emit(Events.UPDATE, item.info)
return web.json_response(
data=item.info,
@ -485,6 +607,15 @@ class HttpAPI(common):
@route("GET", "api/history")
async def history(self, _: Request) -> Response:
"""
Get the history.
Args:
_: The request object.
Returns:
Response: The response object.
"""
data: dict = {"queue": [], "history": []}
for _, v in self.queue.queue.saved_items():
@ -495,7 +626,16 @@ class HttpAPI(common):
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("GET", "api/workers")
async def workers(self, _) -> Response:
async def pool_list(self, _) -> Response:
"""
Get the workers status.
Args:
_: The request object.
Returns:
Response: The response object.
"""
if self.queue.pool is None:
return web.json_response({"error": "Worker pool not initialized."}, status=web.HTTPNotFound.status_code)
@ -523,7 +663,16 @@ class HttpAPI(common):
)
@route("POST", "api/workers")
async def restart_pool(self, _) -> Response:
async def pool_restart(self, _) -> Response:
"""
Restart the workers pool.
Args:
_: The request object.
Returns:
Response: The response object.
"""
if self.queue.pool is None:
return web.json_response({"error": "Worker pool not initialized."}, status=web.HTTPNotFound.status_code)
@ -532,7 +681,16 @@ class HttpAPI(common):
return web.json_response({"message": "Workers pool being restarted."}, status=web.HTTPOk.status_code)
@route("PATCH", "api/workers/{id}")
async def restart_worker(self, request: Request) -> Response:
async def worker_restart(self, request: Request) -> Response:
"""
Restart a worker.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
id: str = request.match_info.get("id")
if not id:
return web.json_response({"error": "worker id is required."}, status=web.HTTPBadRequest.status_code)
@ -544,8 +702,17 @@ class HttpAPI(common):
return web.json_response({"status": "restarted" if status else "in_error_state"}, status=web.HTTPOk.status_code)
@route("delete", "api/workers/{id}")
async def stop_worker(self, request: Request) -> Response:
@route("DELETE", "api/workers/{id}")
async def worker_stop(self, request: Request) -> Response:
"""
Stop a worker.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
id: str = request.match_info.get("id")
if not id:
raise web.HTTPBadRequest(text="worker id is required.")
@ -559,6 +726,15 @@ class HttpAPI(common):
@route("GET", "api/player/playlist/{file:.*}.m3u8")
async def playlist(self, request: Request) -> Response:
"""
Get the playlist.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file")
if not file:
@ -583,6 +759,15 @@ class HttpAPI(common):
@route("GET", "api/player/m3u8/{mode}/{file:.*}.m3u8")
async def m3u8(self, request: Request) -> Response:
"""
Get the m3u8 file.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file")
mode: str = request.match_info.get("mode")
@ -624,6 +809,15 @@ class HttpAPI(common):
@route("GET", r"api/player/segments/{segment:\d+}/{file:.*}.ts")
async def segments(self, request: Request) -> Response:
"""
Get the segments.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file")
segment: int = request.match_info.get("segment")
sd: int = request.query.get("sd")
@ -673,6 +867,15 @@ class HttpAPI(common):
@route("GET", "api/player/subtitle/{file:.*}.vtt")
async def subtitles(self, request: Request) -> Response:
"""
Get the subtitles.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file")
file_path: str = os.path.normpath(os.path.join(self.config.download_path, file))
if not file_path.startswith(self.config.download_path):
@ -707,7 +910,16 @@ class HttpAPI(common):
)
@route("GET", "/")
async def index(self, _) -> Response:
async def index(self, _: Request) -> Response:
"""
Get the index file.
Args:
_: The request object.
Returns:
Response: The response object.
"""
if "/index.html" not in self.staticHolder:
LOG.error("Static frontend files not found.")
return web.json_response(
@ -724,6 +936,15 @@ class HttpAPI(common):
@route("GET", "api/thumbnail")
async def get_thumbnail(self, request: Request) -> Response:
"""
Get the thumbnail.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
url = request.query.get("url")
if not url:
return web.json_response(data={"error": "URL is required."}, status=web.HTTPForbidden.status_code)
@ -759,13 +980,21 @@ class HttpAPI(common):
)
except Exception as e:
LOG.error(f"Error fetching thumbnail from '{url}'. '{e}'.")
LOG.exception(e)
return web.json_response(
data={"error": "failed to retrieve the thumbnail."}, status=web.HTTPInternalServerError.status_code
)
@route("GET", "api/ffprobe/{file:.*}")
@route("GET", "api/file/ffprobe/{file:.*}")
async def get_ffprobe(self, request: Request) -> Response:
"""
Get the ffprobe data.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
file: str = request.match_info.get("file")
try:
realFile: str = calcDownloadPath(basePath=self.config.download_path, folder=file, createPath=False)
@ -782,6 +1011,15 @@ class HttpAPI(common):
@route("GET", "api/youtube/auth")
async def is_authenticated(self, request: Request) -> Response:
"""
Check if the user yt-dlp cookie is valid & authenticated.
Args:
request (Request): The request object.
Returns:
Response: The response object
"""
cookie_file = self.config.ytdl_options.get("cookiefile", None)
if not cookie_file:
return web.json_response(data={"message": "No cookie file provided."}, status=web.HTTPForbidden.status_code)
@ -834,22 +1072,35 @@ class HttpAPI(common):
@route("GET", "api/notifications")
async def notifications(self, _: Request) -> Response:
data = {
"notifications": Notification.get_instance().getTargets(),
"allowedTypes": Notification.allowed_events,
}
"""
Get the notification targets.
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
Args:
_: The request object.
@route("POST", "api/notifications/test")
async def test_notifications(self, _: Request) -> Response:
data = {"type": "test", "message": "This is a test notification."}
await self.emitter.emit("test", data)
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
Returns:
Response: The response object.
"""
return web.json_response(
data={
"notifications": Notification.get_instance().getTargets(),
"allowedTypes": list(NotificationEvents.getEvents().values()),
},
status=web.HTTPOk.status_code,
dumps=self.encoder.encode,
)
@route("PUT", "api/notifications")
async def add_notifications(self, request: Request) -> Response:
async def notification_add(self, request: Request) -> Response:
"""
Add notification targets.
Args:
request (Request): The request object.
Returns:
Response: The response object.
"""
post = await request.json()
if not isinstance(post, list):
return web.json_response(
@ -859,6 +1110,7 @@ class HttpAPI(common):
targets: list = []
ins = Notification.get_instance()
for item in post:
if not isinstance(item, dict):
return web.json_response(
@ -866,7 +1118,7 @@ class HttpAPI(common):
status=web.HTTPBadRequest.status_code,
)
if not item.get("id", None) or validateUUID(item.get("id"), version=4):
if not item.get("id", None) or validate_uuid(item.get("id"), version=4):
item["id"] = str(uuid.uuid4())
try:
@ -877,13 +1129,11 @@ class HttpAPI(common):
status=web.HTTPBadRequest.status_code,
)
targets.append(item)
ins = Notification.get_instance()
targets.append(ins.makeTarget(item))
try:
if len(targets) < 1:
ins.clearTargets()
ins.clear()
ins.save(targets=targets)
ins.load()
@ -891,9 +1141,22 @@ class HttpAPI(common):
LOG.exception(e)
return web.json_response({"error": "Failed to save tasks."}, status=web.HTTPInternalServerError.status_code)
data = {
"notifications": targets,
"allowedTypes": Notification.allowed_events,
}
data = {"notifications": targets, "allowedTypes": list(NotificationEvents.getEvents().values())}
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)
@route("POST", "api/notifications/test")
async def notification_test(self, _: Request) -> Response:
"""
Test the notification.
Args:
_: The request object.
Returns:
Response: The response object.
"""
data = {"type": "test", "message": "This is a test notification."}
await self.emitter.emit(Events.TEST, data)
return web.json_response(data=data, status=web.HTTPOk.status_code, dumps=self.encoder.encode)

108
app/library/HttpSocket.py
View file

@ -1,7 +1,6 @@
import asyncio
import errno
import functools
import json
import logging
import os
import pty
@ -11,6 +10,7 @@ from datetime import datetime
import socketio
from aiohttp import web
from typing import Any
from .common import common
from .config import Config
@ -18,6 +18,7 @@ from .DownloadQueue import DownloadQueue
from .Emitter import Emitter
from .encoder import Encoder
from .Utils import isDownloaded, arg_converter
from .EventsSubscriber import EventsSubscriber, Events, Event
LOG = logging.getLogger("socket_api")
@ -29,17 +30,29 @@ class HttpSocket(common):
config: Config
sio: socketio.AsyncServer
queue: DownloadQueue
emitter: Emitter
def __init__(self, queue: DownloadQueue, emitter: Emitter, encoder: Encoder):
super().__init__(queue=queue, encoder=encoder)
def __init__(
self,
queue: DownloadQueue | None = None,
emitter: Emitter | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
sio: socketio.AsyncServer | None = None,
):
self.config = config or Config.get_instance()
self.queue = queue or DownloadQueue.get_instance()
self.emitter = emitter or Emitter.get_instance()
self.sio = sio or socketio.AsyncServer(cors_allowed_origins="*")
encoder = encoder or Encoder()
self.sio = socketio.AsyncServer(cors_allowed_origins="*")
emitter.add_emitter(lambda event, data, **kwargs: self.sio.emit(event, encoder.encode(data), **kwargs))
def emit(event: str, data: Any, **kwargs):
return self.sio.emit(event=event, data=encoder.encode(data), **kwargs)
self.config = Config.get_instance()
self.emitter.add_emitter([emit], local=False)
self.queue = queue
self.emitter = emitter
super().__init__(queue=queue, encoder=encoder, config=config)
def ws_event(func): # type: ignore
"""
@ -61,10 +74,18 @@ class HttpSocket(common):
if hasattr(method, "_ws_event") and self.sio:
self.sio.on(method._ws_event)(method) # type: ignore
# self.sio.on("*", es.emit)
async def handle_event(_: str, data: Event):
LOG.debug(f"Event received. '{data}'")
await self.add(**data.data)
EventsSubscriber.get_instance().subscribe(Events.ADD_URL, "socket_add_url", handle_event)
@ws_event # type: ignore
async def cli_post(self, sid: str, data):
if not data:
await self.emitter.emit("cli_close", {"exitcode": 0}, to=sid)
await self.emitter.emit(Events.CLI_CLOSE, {"exitcode": 0}, to=sid)
return
try:
@ -100,7 +121,7 @@ class HttpSocket(common):
except Exception as e:
LOG.error(f"Error closing PTY. '{str(e)}'.")
async def read_pty():
async def read_pty(sid: str):
loop = asyncio.get_running_loop()
buffer = b""
while True:
@ -117,14 +138,18 @@ class HttpSocket(common):
if buffer:
# Emit any remaining partial line
await self.emitter.emit(
"cli_output", {"type": "stdout", "line": buffer.decode("utf-8", errors="replace")}
Events.CLI_OUTPUT,
{"type": "stdout", "line": buffer.decode("utf-8", errors="replace")},
to=sid,
)
break
buffer += chunk
*lines, buffer = buffer.split(b"\n")
for line in lines:
await self.emitter.emit(
"cli_output", {"type": "stdout", "line": line.decode("utf-8", errors="replace")}
Events.CLI_OUTPUT,
{"type": "stdout", "line": line.decode("utf-8", errors="replace")},
to=sid,
)
try:
os.close(master_fd)
@ -132,7 +157,7 @@ class HttpSocket(common):
LOG.error(f"Error closing PTY. '{str(e)}'.")
# Start reading output from PTY
read_task = asyncio.create_task(read_pty())
read_task = asyncio.create_task(read_pty(sid=sid))
# Wait until process finishes
returncode = await proc.wait()
@ -140,51 +165,29 @@ class HttpSocket(common):
# Ensure reading is done
await read_task
await self.emitter.emit("cli_close", {"exitcode": returncode}, to=sid)
await self.emitter.emit(Events.CLI_CLOSE, {"exitcode": returncode}, to=sid)
except Exception as e:
LOG.error(f"CLI execute exception was thrown for client '{sid}'.")
LOG.exception(e)
await self.emitter.emit(
"cli_out1put",
{
"type": "stderr",
"line": str(e),
},
to=sid,
)
await self.emitter.emit("cli_close", {"exitcode": -1}, to=sid)
await self.emitter.emit(Events.CLI_OUTPUT, {"type": "stderr", "line": str(e)}, to=sid)
await self.emitter.emit(Events.CLI_CLOSE, {"exitcode": -1}, to=sid)
@ws_event # type: ignore
async def add_url(self, sid: str, data: dict):
url: str | None = data.get("url")
if not url:
await self.emitter.error("No URL provided.", to=sid)
await self.emitter.error("No URL provided.", data={"unlock": True}, to=sid)
return
preset: str = str(data.get("preset", self.config.default_preset))
folder: str = str(data.get("folder")) if data.get("folder") else ""
ytdlp_cookies: str = str(data.get("ytdlp_cookies")) if data.get("ytdlp_cookies") else ""
output_template: str = str(data.get("output_template")) if data.get("output_template") else ""
try:
item = self.format_item(data)
except ValueError as e:
return web.json_response(data={"error": str(e)}, status=web.HTTPBadRequest.status_code)
ytdlp_config = data.get("ytdlp_config")
if isinstance(ytdlp_config, str) and ytdlp_config:
try:
ytdlp_config = json.loads(ytdlp_config)
except Exception as e:
await self.emitter.error(f"Failed to parse json yt-dlp config. {str(e)}", to=sid)
return
status = await self.add(**item)
status = await self.add(
url=url,
preset=preset,
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config if isinstance(ytdlp_config, dict) else {},
output_template=output_template,
)
await self.emitter.emit("status", status, to=sid)
await self.emitter.emit(event=Events.STATUS, data=status, to=sid)
@ws_event # type: ignore
async def item_cancel(self, sid: str, id: str):
@ -196,7 +199,7 @@ class HttpSocket(common):
status = await self.queue.cancel([id])
status.update({"identifier": id})
await self.emitter.emit("item_cancel", status)
await self.emitter.emit(event=Events.ITEM_CANCEL, data=status)
@ws_event # type: ignore
async def item_delete(self, sid: str, data: dict):
@ -213,7 +216,7 @@ class HttpSocket(common):
status = await self.queue.clear([id], remove_file=bool(data.get("remove_file", False)))
status.update({"identifier": id})
await self.emitter.emit("item_delete", status)
await self.emitter.emit(event=Events.ITEM_DELETE, data=status)
@ws_event # type: ignore
async def archive_item(self, sid: str, data: dict):
@ -256,26 +259,25 @@ class HttpSocket(common):
data = {
**self.queue.get(),
"config": self.config.frontend(),
"tasks": self.config.tasks,
"presets": self.config.presets,
"paused": self.queue.isPaused(),
}
# get download folder listing
# get download folder listing.
downloadPath: str = self.config.download_path
data["folders"] = [name for name in os.listdir(downloadPath) if os.path.isdir(os.path.join(downloadPath, name))]
await self.emitter.emit("initial_data", data, to=sid)
await self.emitter.emit(event=Events.INITIAL_DATA, data=data, to=sid)
@ws_event # type: ignore
async def pause(self, sid: str, _=None):
self.queue.pause()
await self.emitter.emit("paused", {"paused": True, "at": time.time()})
await self.emitter.emit(event=Events.PAUSED, data={"paused": True, "at": time.time()})
@ws_event # type: ignore
async def resume(self, sid: str, _=None):
self.queue.resume()
await self.emitter.emit("paused", {"paused": False, "at": time.time()})
await self.emitter.emit(event=Events.PAUSED, data={"paused": False, "at": time.time()})
@ws_event
async def ytdlp_convert(self, sid: str, data: dict):
@ -290,7 +292,7 @@ class HttpSocket(common):
return
try:
await self.emitter.emit("ytdlp_convert", arg_converter(args), to=sid)
await self.emitter.emit(event=Events.YTDLP_CONVERT, data=arg_converter(args), to=sid)
return
except Exception as e:
err = str(e).strip()

36
app/library/ItemDTO.py
View file

@ -5,6 +5,7 @@ from dataclasses import dataclass, field
from email.utils import formatdate
from typing import Any
@dataclass(kw_only=True)
class ItemDTO:
"""
@ -14,22 +15,21 @@ class ItemDTO:
_id: str = field(default_factory=lambda: str(uuid.uuid4()), init=False)
error: str|None = None
error: str | None = None
id: str
title: str
url: str
quality: str | None = None
format: str | None = None
thumbnail: str | None = None
preset: str = "default"
folder: str
download_dir: str | None = None
temp_dir: str | None = None
status: str | None = None
ytdlp_cookies: str | None = None
ytdlp_config: dict = field(default_factory=dict)
output_template: str | None = None
output_template_chapter: str | None = None
cookies: str | None = None
config: dict = field(default_factory=dict)
template: str | None = None
template_chapter: str | None = None
timestamp: float = time.time_ns()
is_live: bool | None = None
datetime: str = field(default_factory=lambda: str(formatdate(time.time())))
@ -49,19 +49,41 @@ class ItemDTO:
speed: str | None = None
eta: str | None = None
# DEPRECATED: These fields are deprecated and will be removed in the future.
thumbnail: str | None = None
ytdlp_cookies: str | None = None
ytdlp_config: dict = field(default_factory=dict)
output_template: str | None = None
output_template_chapter: str | None = None
def serialize(self) -> dict:
deprecated: tuple = (
"thumbnail",
"quality",
"format",
"ytdlp_cookies",
"ytdlp_config",
"output_template",
"output_template_chapter",
)
if self.thumbnail and "thumbnail" not in self.extras:
self.extras["thumbnail"] = self.thumbnail
mapper: dict = {
"cookies": "ytdlp_cookies",
"config": "ytdlp_config",
"template": "output_template",
"template_chapter": "output_template_chapter",
}
for k, v in mapper.items():
if not self.get(k) and self.get(v):
self.__dict__[k] = self.get(v)
dump = self.__dict__.copy()
for f in deprecated:
dump.pop(f)
dump.pop(f, None)
return dump

268
app/library/Notifications.py
View file

@ -3,38 +3,65 @@ from datetime import datetime, timezone
import json
import logging
import os
from typing import List, TypedDict
import uuid
from typing import List, Any
from dataclasses import dataclass, field
import httpx
from .config import Config
from .ItemDTO import ItemDTO
from .Singleton import Singleton
from .Utils import ag, validateUUID
from .Utils import validate_uuid, ag
from .version import APP_VERSION
from .encoder import Encoder
from .EventsSubscriber import Events
LOG = logging.getLogger("notifications")
class targetRequestHeader(TypedDict):
@dataclass(kw_only=True)
class targetRequestHeader:
"""Request header details for a notification target."""
key: str
value: str
def serialize(self) -> dict:
return {"key": self.key, "value": self.value}
class targetRequest(TypedDict):
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return self.serialize().get(key, default)
@dataclass(kw_only=True)
class targetRequest:
"""Request details for a notification target."""
type: str
method: str
url: str
headers: list[targetRequestHeader] = []
headers: list[targetRequestHeader] = field(default_factory=list)
def serialize(self) -> dict:
return {
"type": self.type,
"method": self.method,
"url": self.url,
"headers": [h.serialize() for h in self.headers],
}
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return ag(self.serialize(), key, default)
class Target(TypedDict):
@dataclass(kw_only=True)
class Target:
"""Notification target details."""
id: str
@ -42,37 +69,76 @@ class Target(TypedDict):
on: List[str]
request: targetRequest
def serialize(self) -> dict:
return {
"id": self.id,
"name": self.name,
"on": self.on,
"request": self.request.serialize(),
}
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return self.serialize().get(key, default)
class NotificationEvents:
ADDED = Events.ADDED
COMPLETED = Events.COMPLETED
ERROR = Events.ERROR
CANCELLED = Events.CANCELLED
CLEARED = Events.CLEARED
LOG_INFO = Events.LOG_INFO
LOG_SUCCESS = Events.LOG_SUCCESS
TEST = Events.TEST
@staticmethod
def getEvents() -> dict[str, str]:
data: dict[str, str] = {}
for k, v in vars(NotificationEvents).items():
if not k.startswith("__") and not callable(v):
data[k] = v
return data
@staticmethod
def isValid(event: str) -> bool:
return event in NotificationEvents.getEvents().values()
class Notification(metaclass=Singleton):
targets: list[Target] = []
_targets: list[Target] = []
"""Notification targets to send events to."""
allowed_events: tuple = (
"added",
"completed",
"error",
"not_live",
"canceled",
"cleared",
"log_info",
"log_success",
)
"""Events that can be sent to the targets."""
_instance = None
"""The instance of the Notification class."""
def __init__(self):
def __init__(
self,
file: str | None = None,
client: httpx.AsyncClient | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
):
Notification._instance = self
config = Config.get_instance()
config: Config = config or Config.get_instance()
self._debug = config.debug
self.file: str = os.path.join(config.config_path, "notifications.json")
self._client: httpx.AsyncClient = httpx.AsyncClient()
self._encoder: Encoder = Encoder()
self._file: str = file or os.path.join(config.config_path, "notifications.json")
self._client: httpx.AsyncClient = client or httpx.AsyncClient()
self._encoder: Encoder = encoder or Encoder()
if os.path.exists(self.file) and os.path.getsize(self.file) > 10:
self.load()
if os.path.exists(self._file):
try:
if "600" != oct(os.stat(self._file).st_mode)[-3:]:
os.chmod(self._file, 0o600)
except Exception:
pass
if os.path.getsize(self._file) > 10:
self.load()
@staticmethod
def get_instance() -> "Notification":
@ -83,14 +149,14 @@ class Notification(metaclass=Singleton):
def getTargets(self) -> list[Target]:
"""Get the list of notification targets."""
return self.targets
return self._targets
def clearTargets(self) -> "Notification":
def clear(self) -> "Notification":
"""Clear the list of notification targets."""
self.targets.clear()
self._targets.clear()
return self
def save(self, targets: list[Target] | None = None) -> "Notification":
def save(self, targets: list[Target]) -> "Notification":
"""
Save notification targets to the file.
@ -100,36 +166,36 @@ class Notification(metaclass=Singleton):
Returns:
Notification: The Notification instance.
"""
LOG.info(f"Saving notification targets to '{self.file}'.")
LOG.info(f"Saving notification targets to '{self._file}'.")
try:
with open(self.file, "w") as f:
f.write(json.dumps(targets or self.targets, indent=4))
with open(self._file, "w") as f:
json.dump([t.serialize() for t in targets], fp=f, indent=4)
except Exception as e:
LOG.error(f"Error saving notification targets to '{self.file}'. '{e}'")
LOG.exception(e)
LOG.error(f"Error saving notification targets to '{self._file}'. '{e}'")
pass
return self
def load(self):
def load(self) -> "Notification":
"""Load or reload notification targets from the file."""
if len(self.targets) > 0:
if len(self._targets) > 0:
LOG.info("Clearing existing notification targets.")
self.targets.clear()
self.clear()
modified = False
if not os.path.exists(self._file) or os.path.getsize(self._file) < 10:
return self
targets = []
if not os.path.exists(self.file) or os.path.getsize(self.file) < 10:
return
LOG.info(f"Loading notification targets from '{self.file}'.")
LOG.info(f"Loading notification targets from '{self._file}'.")
try:
with open(self.file, "r") as f:
with open(self._file, "r") as f:
targets = json.load(f)
except Exception as e:
LOG.error(f"Error loading notification targets from '{self.file}'. '{e}'")
LOG.error(f"Error loading notification targets from '{self._file}'. '{e}'")
pass
for target in targets:
@ -140,48 +206,18 @@ class Notification(metaclass=Singleton):
LOG.error(f"Invalid notification target '{target}'. '{e}'")
continue
if "on" not in target:
target["on"] = []
modified = True
target = self.makeTarget(target)
if "type" not in target["request"]:
target["request"]["type"] = "json"
modified = True
if "method" not in target["request"]:
target["request"]["method"] = "POST"
modified = True
if "id" not in target or validateUUID(target["id"], version=4) is False:
target["id"] = str(uuid.uuid4())
modified = True
target = Target(
id=target.get("id"),
name=target.get("name"),
on=target.get("on", []),
request=targetRequest(
type=target.get("request", {}).get("type", "json"),
method=target.get("request", {}).get("method", "POST"),
url=target.get("request", {}).get("url"),
headers=[
targetRequestHeader(key=h.get("key"), value=h.get("value"))
for h in target.get("request", {}).get("headers", [])
],
),
)
self.targets.append(target)
self._targets.append(target)
LOG.info(
f"Will send '{target['on'] if len(target['on']) > 0 else 'all'}' as {target['request']['type']} notification events to '{target['name']}'."
f"Will send '{target.on if len(target.on) > 0 else 'all'}' as {target.request.type} notification events to '{target.name}'."
)
except Exception as e:
LOG.error(f"Error loading notification target '{target}'. '{e}'")
pass
if modified:
self.save()
return self
@staticmethod
def validate(target: Target | dict) -> bool:
@ -195,7 +231,10 @@ class Notification(metaclass=Singleton):
bool: True if the target is valid, False otherwise.
"""
if "id" not in target or validateUUID(target["id"], version=4) is False:
if not isinstance(target, dict):
target = target.serialize()
if "id" not in target or validate_uuid(target["id"], version=4) is False:
raise ValueError("Invalid notification target. No ID found.")
if "name" not in target:
@ -218,7 +257,7 @@ class Notification(metaclass=Singleton):
raise ValueError("Invalid notification target. Invalid 'on' event list found.")
for e in target["on"]:
if e not in Notification.allowed_events:
if e not in NotificationEvents.getEvents().values():
raise ValueError(f"Invalid notification target. Invalid event '{e}' found.")
if "headers" in target["request"]:
@ -234,7 +273,7 @@ class Notification(metaclass=Singleton):
return True
async def send(self, event: str, item: ItemDTO | dict) -> list[dict]:
if len(self.targets) < 1:
if len(self._targets) < 1:
return []
if not isinstance(item, ItemDTO) and not isinstance(item, dict):
@ -243,8 +282,8 @@ class Notification(metaclass=Singleton):
tasks = []
for target in self.targets:
if len(target["on"]) > 0 and event not in target["on"] and "test" != event:
for target in self._targets:
if len(target.on) > 0 and event not in target.on and "test" != event:
continue
tasks.append(self._send(event, target, item))
@ -252,44 +291,42 @@ class Notification(metaclass=Singleton):
return await asyncio.gather(*tasks)
async def _send(self, event: str, target: Target, item: ItemDTO | dict) -> dict:
req: targetRequest = target["request"]
try:
itemId = ag(item, ["id", "_id"], "??")
itemId = item.get("id", item.get("_id", "??"))
except Exception:
itemId = "??"
try:
LOG.info(f"Sending Notification event '{event}' id '{itemId}' to '{target.get('name')}'.")
LOG.info(f"Sending Notification event '{event}' id '{itemId}' to '{target.name}'.")
reqBody = {
"method": req["method"].upper(),
"url": req["url"],
"method": target.request.method.upper(),
"url": target.request.url,
"headers": {
"User-Agent": f"YTPTube/{APP_VERSION}",
"Content-Type": "application/json"
if "json" == req["type"].lower()
if "json" == target.request.type.lower()
else "application/x-www-form-urlencoded",
},
}
if len(req["headers"]) > 0:
for h in req["headers"]:
reqBody["headers"][h["key"]] = h["value"]
if len(target.request.headers) > 0:
for h in target.request.headers:
reqBody["headers"][h.key] = h.value
reqBody["json" if "json" == req["type"].lower() else "data"] = {
reqBody["json" if "json" == target.request.type.lower() else "data"] = {
"event": event,
"created_at": datetime.now(tz=timezone.utc).isoformat(),
"payload": item.__dict__ if isinstance(item, ItemDTO) else item,
}
if "form" == req["type"].lower():
if "form" == target.request.type.lower():
reqBody["data"]["payload"] = self._encoder.encode(reqBody["data"]["payload"])
response = await self._client.request(**reqBody)
respData = {"url": req["url"], "status": response.status_code, "text": response.text}
respData = {"url": target.request.url, "status": response.status_code, "text": response.text}
msg = f"Notification target '{target['name']}' Responded to event '{event}' id '{itemId}' with status '{response.status_code}'."
msg = f"Notification target '{target.name}' Responded to event '{event}' id '{itemId}' with status '{response.status_code}'."
if self._debug and respData.get("text"):
msg += f" body '{respData.get('text','??')}'."
@ -297,14 +334,39 @@ class Notification(metaclass=Singleton):
return respData
except Exception as e:
LOG.error(f"Error sending Notification event '{event}' id '{itemId}' to '{target['name']}'. '{e}'.")
return {"url": req["url"], "status": 500, "text": str(e)}
LOG.error(f"Error sending Notification event '{event}' id '{itemId}' to '{target.name}'. '{e}'.")
return {"url": target.request.url, "status": 500, "text": str(e)}
def makeTarget(self, target: dict) -> Target:
"""
Make a notification target from a dictionary.
Args:
target (dict): The target details.
Returns:
Target: The notification target.
"""
return Target(
id=target.get("id"),
name=target.get("name"),
on=target.get("on", []),
request=targetRequest(
type=target.get("request", {}).get("type", "json"),
method=target.get("request", {}).get("method", "POST"),
url=target.get("request", {}).get("url"),
headers=[
targetRequestHeader(key=h.get("key"), value=h.get("value"))
for h in target.get("request", {}).get("headers", [])
],
),
)
def emit(self, event, data, **kwargs):
if len(self.targets) < 1:
if len(self._targets) < 1:
return False
if event not in self.allowed_events and "test" != event:
if not NotificationEvents.isValid(event):
return False
return self.send(event, data)

336
app/library/Tasks.py Normal file
View file

@ -0,0 +1,336 @@
import asyncio
from datetime import datetime
import json
import logging
import os
import time
from dataclasses import dataclass, field
from typing import Any, List
import httpx
from aiocron import Cron, crontab
from aiohttp import web
from .config import Config
from .Emitter import Emitter
from .encoder import Encoder
from .EventsSubscriber import Event, Events, EventsSubscriber
from .Singleton import Singleton
LOG = logging.getLogger("notifications")
@dataclass(kw_only=True)
class Task:
id: str
name: str
url: str
folder: str = ""
preset: str = ""
timer: str = ""
template: str = ""
cookies: str = ""
config: dict[str, str] = field(default_factory=dict)
def serialize(self) -> dict:
return self.__dict__
def json(self) -> str:
return Encoder().encode(self.serialize())
def get(self, key: str, default: Any = None) -> Any:
return self.serialize().get(key, default)
@dataclass(kw_only=True)
class Job:
id: str
name: str
task: Task
job: Cron
class Tasks(metaclass=Singleton):
"""
This class is used to manage the tasks.
"""
_jobs: List[Job] = []
"""The jobs for the tasks."""
_instance = None
"""The instance of the Tasks."""
def __init__(
self,
file: str | None = None,
emitter: Emitter | None = None,
loop: asyncio.AbstractEventLoop | None = None,
config: Config | None = None,
encoder: Encoder | None = None,
client: httpx.AsyncClient | None = None,
):
Tasks._instance = self
config = config or Config.get_instance()
self._debug = config.debug
self._default_preset = config.default_preset
self._file: str = file or os.path.join(config.config_path, "tasks.json")
self._client: httpx.AsyncClient = client or httpx.AsyncClient()
self._encoder: Encoder = encoder or Encoder()
self._loop: asyncio.AbstractEventLoop = loop or asyncio.get_event_loop()
self._emitter: Emitter = emitter or Emitter.get_instance()
if os.path.exists(self._file) and "600" != oct(os.stat(self._file).st_mode)[-3:]:
try:
os.chmod(self._file, 0o600)
except Exception:
pass
def handle_event(_, e: Event):
self.save(**e.data)
EventsSubscriber.get_instance().subscribe(Events.TASKS_ADD, f"{__class__}.save", handle_event)
@staticmethod
def get_instance() -> "Tasks":
"""
Get the instance of the Tasks.
Returns:
Tasks: The instance of the Tasks
"""
if not Tasks._instance:
Tasks._instance = Tasks()
return Tasks._instance
def attach(self, _: web.Application):
"""
Attach the tasks to the aiohttp application.
Args:
_ (web.Application): The aiohttp application.
Returns:
None
"""
self.load()
def shutdown(self):
"""
Shutdown the tasks service.
"""
self.clear()
def getTasks(self) -> List[Task]:
"""Return the tasks."""
return [job.task for job in self._jobs]
def load(self) -> "Tasks":
"""
Load the tasks.
Returns:
Tasks: The current instance.
"""
self.clear()
if not os.path.exists(self._file) or os.path.getsize(self._file) < 10:
return self
LOG.info(f"Loading tasks from '{self._file}'.")
try:
with open(self._file, "r") as f:
tasks = json.load(f)
except Exception as e:
LOG.error(f"Failed to parse tasks from '{self._file}'. '{e}'.")
return self
if not tasks or len(tasks) < 1:
LOG.info(f"No tasks were defined in '{self._file}'.")
return self
for i, task in enumerate(tasks):
try:
task = Task(**task)
except Exception as e:
LOG.error(f"Failed to parse task at list position '{i}'. '{str(e)}'.")
continue
try:
self._jobs.append(
Job(
id=task.id,
name=task.name,
task=task,
job=crontab(spec=task.timer, func=self._runner, args=(task,), start=True, loop=self._loop),
)
)
LOG.info(f"Task '{i}: {task.name}' queued to be executed every '{task.timer}'.")
except Exception as e:
LOG.exception(e)
LOG.error(f"Failed to queue task '{i}: {task['name']}'. '{str(e)}'.")
return self
def clear(self) -> "Tasks":
"""
Clear all tasks.
Returns:
Tasks: The current instance.
"""
if len(self._jobs) < 1:
return self
for task in self._jobs:
try:
LOG.info(f"Stopping job '{task.id}: {task.name}'.")
task.job.stop()
except Exception as e:
LOG.exception(e)
LOG.error(f"Failed to stop job '{task.id}: {task.name}'. '{str(e)}'.")
self._jobs.clear()
return self
def validate(self, task: Task | dict) -> bool:
"""
Validate the task.
Args:
tasks (Task|dict): The task to validate.
Returns:
bool: True if the task is valid, False otherwise.
"""
if not isinstance(task, dict):
if not isinstance(task, Task):
raise ValueError("Invalid task type.")
task = task.serialize()
if not task.get("name"):
raise ValueError("No name found.")
if not task.get("timer"):
raise ValueError("No timer found.")
if not task.get("url"):
raise ValueError("No URL found.")
if not isinstance(task.get("cookies"), str):
raise ValueError("Invalid cookies type.")
if not isinstance(task.get("config"), dict):
raise ValueError("Invalid config type.")
if not isinstance(task.get("template"), str):
raise ValueError("Invalid template type.")
return True
def save(self, tasks: List[Task | dict]) -> "Tasks":
"""
Save the tasks.
Args:
tasks (List[Task]): The tasks to save.
Returns:
Tasks: The current instance.
"""
for i, task in enumerate(tasks):
try:
if not isinstance(task, Task):
task = Task(**task)
tasks[i] = task
except Exception as e:
LOG.error(f"Failed to save task '{i}' unable to parse task. '{str(e)}'.")
continue
try:
self.validate(task)
except ValueError as e:
LOG.error(f"Failed to add task '{i}: {task.name}'. '{e}'.")
continue
try:
with open(self._file, "w") as f:
json.dump(obj=[task.serialize() for task in tasks], fp=f, indent=4)
LOG.info(f"Tasks saved to '{self._file}'.")
except Exception as e:
LOG.error(f"Failed to save tasks to '{self._file}'. '{str(e)}'.")
return self
async def _runner(self, task: Task):
"""
Run the task.
Args:
task (Task): The task to run.
Returns:
None
"""
try:
timeNow = datetime.now().isoformat()
started = time.time()
if not task.url:
LOG.error(f"Failed to dispatch task '{task.id}: {task.name}'. No URL found.")
return
preset: str = str(task.preset or self._default_preset)
folder: str = task.folder if task.folder else ""
cookies: str = str(task.cookies) if task.cookies else ""
template: str = task.template if task.template else ""
config = task.config if task.config else {}
if isinstance(config, str) and config:
try:
config = json.loads(config)
except Exception as e:
LOG.error(f"Failed to parse json yt-dlp config for '{task.name}'. {str(e)}")
return
LOG.info(f"Task '{task.id}: {task.name}' dispatched at '{timeNow}'.")
tasks = []
tasks.append(self._emitter.info(f"Task '{task.name}' dispatched at '{timeNow}'."))
tasks.append(
self._emitter.emit(
event=Events.ADD_URL,
data=Event(
id=task.id,
data={
"url": task.url,
"preset": preset,
"folder": folder,
"cookies": cookies,
"config": config,
"template": template,
},
),
local=True,
)
)
await asyncio.wait_for(asyncio.gather(*tasks), timeout=None)
timeNow = datetime.now().isoformat()
ended = time.time()
LOG.info(f"Task '{task.id}: {task.name}' completed at '{timeNow}' took '{ended - started:.2f}' seconds.")
await self._emitter.success(f"Task '{task.name}' completed in '{ended - started:.2f}' seconds.")
except Exception as e:
timeNow = datetime.now().isoformat()
LOG.error(f"Task '{task.id}: {task.name}' has failed to execute at '{timeNow}'. '{str(e)}'.")
await self._emitter.error(f"Task '{task.name}' failed to execute at '{timeNow}'. '{str(e)}'.")

2
app/library/Utils.py
View file

@ -537,7 +537,7 @@ def arg_converter(args: str) -> dict:
return json.loads(json.dumps(diff))
def validateUUID(uuid_str: str, version: int = 4) -> bool:
def validate_uuid(uuid_str: str, version: int = 4) -> bool:
"""
Validate if the UUID is valid.

90
app/library/common.py
View file

@ -1,7 +1,9 @@
import json
import logging
from .DownloadQueue import DownloadQueue
from .encoder import Encoder
from .config import Config
LOG = logging.getLogger("common")
@ -11,27 +13,87 @@ class common:
This class is used to share common methods between the socket and the API gateways.
"""
queue: DownloadQueue
encoder: Encoder
def __init__(self, queue: DownloadQueue, encoder: Encoder):
def __init__(
self,
queue: DownloadQueue | None = None,
encoder: Encoder | None = None,
config: Config | None = None,
):
super().__init__()
self.queue = queue
self.encoder = encoder
self.queue = queue or DownloadQueue.get_instance()
self.encoder = encoder or Encoder()
config = config or Config.get_instance()
self.default_preset = config.default_preset
async def add(
self, url: str, preset: str, folder: str, ytdlp_cookies: str, ytdlp_config: dict, output_template: str
self, url: str, preset: str, folder: str, cookies: str, config: dict, template: str
) -> dict[str, str]:
if ytdlp_cookies and isinstance(ytdlp_cookies, dict):
ytdlp_cookies = self.encoder.encode(ytdlp_cookies)
"""
Add an item to the download queue.
status = await self.queue.add(
Args:
url (str): The url to be added to the queue.
preset (str): The preset to be used for the download.
folder (str): The folder to save the download to.
cookies (str): The cookies to be used for the download.
config (dict): The yt-dlp config to be used for the download.
template (str): The template to be used for the download.
Returns:
dict[str, str]: The status of the download.
{ "status": "text" }
"""
if cookies and isinstance(cookies, dict):
cookies = self.encoder.encode(cookies)
return await self.queue.add(
url=url,
preset=preset if preset else "default",
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config if isinstance(ytdlp_config, dict) else {},
output_template=output_template,
cookies=cookies,
config=config if isinstance(config, dict) else {},
template=template,
)
return status
def format_item(self, item: dict) -> dict:
"""
Format the item to be added to the download queue.
Args:
item (dict): The item to be formatted.
Raises:
ValueError: If the url is not provided.
ValueError: If the yt-dlp config is not a valid json.
Returns:
dict: The formatted item
"""
url: str = item.get("url")
if not url:
raise ValueError("url param is required.")
preset: str = str(item.get("preset", self.config.default_preset))
folder: str = str(item.get("folder")) if item.get("folder") else ""
cookies: str = str(item.get("cookies")) if item.get("cookies") else ""
template: str = str(item.get("template")) if item.get("template") else ""
config = item.get("config")
if isinstance(config, str) and config:
try:
config = json.loads(config)
except Exception as e:
raise ValueError(f"Failed to parse json yt-dlp config for '{url}'. {str(e)}")
item = {
"url": url,
"preset": preset,
"folder": folder,
"cookies": cookies,
"config": config if isinstance(config, dict) else {},
"template": template,
}
return item

168
app/main.py
View file

@ -1,52 +1,39 @@
#!/usr/bin/env python3
import asyncio
import json
import logging
import os
import random
import sqlite3
from datetime import datetime
from pathlib import Path
import time
from typing import TypedDict
import uuid
import caribou
import magic
from aiocron import crontab, Cron
from aiohttp import web
from library.Utils import load_file
from library.config import Config
from library.DownloadQueue import DownloadQueue
from library.Emitter import Emitter
from library.encoder import Encoder
from library.HttpAPI import HttpAPI, LOG as http_logger
from library.EventsSubscriber import EventsSubscriber
from library.HttpAPI import LOG as http_logger
from library.HttpAPI import HttpAPI
from library.HttpSocket import HttpSocket
from library.Notifications import Notification
from library.PackageInstaller import PackageInstaller
from library.Tasks import Tasks
LOG = logging.getLogger("app")
MIME = magic.Magic(mime=True)
class job_item(TypedDict):
name: str
job: Cron
class Main:
config: Config
app: web.Application
http: HttpAPI
socket: HttpSocket
cron: list[job_item] = []
def __init__(self):
self.config = Config.get_instance()
self.rootPath = str(Path(__file__).parent.absolute())
self.app = web.Application()
self.encoder = Encoder()
self.checkFolders()
@ -61,18 +48,27 @@ class Main:
connection = sqlite3.connect(database=self.config.db_file, isolation_level=None)
connection.row_factory = sqlite3.Row
connection.execute("PRAGMA journal_mode=wal")
try:
if "600" != oct(os.stat(self.config.db_file).st_mode)[-3:]:
os.chmod(self.config.db_file, 0o600)
except Exception:
pass
self.emitter = Emitter()
queue = DownloadQueue(connection=connection)
self.http = HttpAPI(queue=queue)
self.socket = HttpSocket(queue=queue)
Emitter.get_instance().add_emitter([Notification().emit], local=False).add_emitter(
[EventsSubscriber().emit], local=True
)
queue = DownloadQueue(emitter=self.emitter, connection=connection)
self.app.on_startup.append(lambda _: queue.initialize())
self.http = HttpAPI(queue=queue, emitter=self.emitter, encoder=self.encoder, load_tasks=self.load_tasks)
self.socket = HttpSocket(queue=queue, emitter=self.emitter, encoder=self.encoder)
self.emitter.add_emitter(Notification().emit)
def checkFolders(self) -> None:
def checkFolders(self):
"""
Check if the required folders exist and create them if they do not.
"""
try:
LOG.debug(f"Checking download folder at '{self.config.download_path}'.")
if not os.path.exists(self.config.download_path):
@ -110,127 +106,13 @@ class Main:
LOG.error(f"Could not create database file at '{self.config.db_file}'.")
raise e
async def cron_runner(self, task: dict):
try:
taskName = task.get("name", task.get("url"))
timeNow = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
started = time.time()
url = task.get("url")
if not url:
LOG.error(f"Invalid task '{task}'. No URL found.")
return
preset: str = str(task.get("preset", self.config.default_preset))
folder: str = str(task.get("folder")) if task.get("folder") else ""
ytdlp_cookies: str = str(task.get("ytdlp_cookies")) if task.get("ytdlp_cookies") else ""
output_template: str = str(task.get("output_template")) if task.get("output_template") else ""
ytdlp_config = task.get("ytdlp_config")
if isinstance(ytdlp_config, str) and ytdlp_config:
try:
ytdlp_config = json.loads(ytdlp_config)
except Exception as e:
LOG.error(f"Failed to parse json yt-dlp config for '{taskName}'. {str(e)}")
return
await self.emitter.info(f"Started 'Task: {taskName}' at '{timeNow}'.")
LOG.info(f"Started 'Task: {taskName}' at '{timeNow}'.")
await self.socket.add(
url=url,
preset=preset,
folder=folder,
ytdlp_cookies=ytdlp_cookies,
ytdlp_config=ytdlp_config,
output_template=output_template,
)
timeNow = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
ended = time.time()
LOG.info(f"Completed 'Task: {taskName}' at '{timeNow}' ")
await self.emitter.success(f"Completed 'Task: {taskName}' '{ended - started:.2f}' seconds.")
except Exception as e:
timeNow = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
LOG.error(f"Failed 'Task: {taskName}' at '{timeNow}'. Error message '{str(e)}'.")
await self.emitter.error(f"Failed 'Task: {taskName}' at '{timeNow}'. Error message '{str(e)}'.")
def load_tasks(self):
tasksFile = os.path.join(self.config.config_path, "tasks.json")
if not os.path.exists(tasksFile):
return
LOG.info(f"Loading tasks from '{tasksFile}'.")
try:
(tasks, status, error) = load_file(tasksFile, list)
if not status:
LOG.error(f"Could not load tasks file from '{tasksFile}'. '{error}'.")
return
except Exception:
pass
for job in self.cron:
try:
LOG.info(f"Stopping job '{job['name']}'.")
job["job"].stop()
except Exception as e:
LOG.error(f"Failed to stop job. Error message '{str(e)}'.")
LOG.exception(e)
self.cron.clear()
if not tasks or len(tasks) < 1:
LOG.info(f"No tasks found in '{tasksFile}'.")
return
modified = False
loop = asyncio.get_event_loop()
for task in tasks:
if not task.get("url"):
LOG.warning(f"Invalid task '{task}'. No URL found.")
continue
try:
if not task.get("timer", None):
task["timer"] = f"{random.randint(1,59)} */1 * * *"
modified = True
if not task.get("id", None):
task["id"] = str(uuid.uuid4())
modified = True
self.cron.append(
job_item(
name=task.get("name", "??"),
job=crontab(
spec=task.get("timer"),
func=self.cron_runner,
args=(task,),
start=True,
loop=loop,
),
)
)
LOG.info(f"Queued 'Task: {task.get('name','??')}' to be executed every '{task.get('timer')}'.")
except Exception as e:
LOG.error(f"Failed to add 'Task: {task.get('name', '??')}'. Error message '{str(e)}'.")
LOG.exception(e)
if modified:
try:
with open(tasksFile, "w") as f:
f.write(json.dumps(tasks, indent=4))
except Exception as e:
LOG.error(f"Failed to save tasks file '{tasksFile}'. Error message '{str(e)}'.")
LOG.exception(e)
self.config.tasks = tasks
def start(self):
"""
Start the application.
"""
self.socket.attach(self.app)
self.http.attach(self.app)
self.load_tasks()
Tasks.get_instance().attach(self.app)
def started(_):
LOG.info("=" * 40)

7
ui/assets/css/style.css
View file

@ -160,6 +160,13 @@ hr {
background-color: #00d1b2;
}
.button.is-purple {
background-color: #5f00d1;
border-color: transparent;
color: #fff;
}
@media (prefers-color-scheme: dark) {
.footer {
background-color: #121212;

13
ui/components/GetInfo.vue
View file

@ -29,6 +29,7 @@ import { request } from '~/utils/index'
const emitter = defineEmits(['closeModel'])
const isLoading = ref(false)
const data = ref({})
const toast = useToast()
const props = defineProps({
link: {
@ -47,15 +48,17 @@ const eventFunc = e => {
onMounted(async () => {
window.addEventListener('keydown', eventFunc)
const url = '/api/url/info?url=' + encodePath(props.link)
const url = '/api/yt-dlp/url/info?url=' + encodePath(props.link)
try {
isLoading.value = true
const response = await request(url, { credentials: 'include' });
data.value = await response.json();
data.value = await response.json()
} catch (e) {
console.log(e)
}
finally {
console.error(e)
toast.error(`Error: ${e.message}`)
} finally {
isLoading.value = false
}
})

8
ui/components/History.vue
View file

@ -421,7 +421,7 @@ const setIcon = item => {
return 'fa-solid fa-circle-xmark'
}
if (item.status === 'canceled') {
if (item.status === 'cancelled') {
return 'fa-solid fa-eject'
}
@ -468,9 +468,9 @@ const reQueueItem = item => {
url: item.url,
preset: item.preset,
folder: item.folder,
ytdlp_config: item.ytdlp_config,
ytdlp_cookies: item.ytdlp_cookies,
output_template: item.output_template,
config: item.config,
cookies: item.cookies,
template: item.template,
})
}
</script>

46
ui/components/NewDownload.vue
View file

@ -20,7 +20,8 @@
</div>
<div class="control is-expanded">
<div class="select is-fullwidth">
<select id="preset" class="is-fullwidth" :disabled="!socket.isConnected" v-model="selectedPreset">
<select id="preset" class="is-fullwidth" :disabled="!socket.isConnected || addInProgress"
v-model="selectedPreset">
<option v-for="item in config.presets" :key="item.name" :value="item.name">
{{ item.name }}
</option>
@ -36,7 +37,7 @@
</div>
<div class="control is-expanded">
<input type="text" class="input is-fullwidth" id="path" v-model="downloadPath" placeholder="Default"
:disabled="!socket.isConnected" list="folders">
:disabled="!socket.isConnected || addInProgress" list="folders">
</div>
</div>
</div>
@ -65,6 +66,7 @@
</label>
<div class="control">
<input type="text" class="input" v-model="output_template" id="output_format"
:disabled="!socket.isConnected || addInProgress"
placeholder="Uses default output template naming if empty.">
</div>
<span class="help">
@ -80,12 +82,13 @@
JSON yt-dlp config or CLI options.
</label>
<div class="control">
<textarea class="textarea" id="ytdlpConfig" v-model="ytdlpConfig" :disabled="!socket.isConnected"
<textarea class="textarea" id="ytdlpConfig" v-model="ytdlpConfig"
:disabled="!socket.isConnected || addInProgress"
placeholder="--no-embed-metadata --no-embed-thumbnail"></textarea>
</div>
<span class="help">
Some config fields are ignored like cookiefile, path, and output_format etc.
Available option can be found at <NuxtLink target="_blank"
Some config fields are ignored like <code>format</code> <code>cookiefile</code>, <code>paths</code>,
and <code>outtmpl</code> etc. Available option can be found at <NuxtLink target="_blank"
to="https://github.com/yt-dlp/yt-dlp/blob/a0b19d319a6ce8b7059318fa17a34b144fde1785/yt_dlp/YoutubeDL.py#L194">
this page</NuxtLink>. Warning: Use with caution some of those options can break yt-dlp or the
frontend.
@ -99,7 +102,7 @@
</label>
<div class="control">
<textarea class="textarea" id="ytdlpCookies" v-model="ytdlpCookies"
:disabled="!socket.isConnected"></textarea>
:disabled="!socket.isConnected || addInProgress"></textarea>
</div>
<span class="help">
Use <NuxtLink target="_blank" to="https://github.com/jrie/flagCookies">
@ -157,7 +160,7 @@ const addInProgress = ref(false)
const addDownload = async () => {
// -- send request to convert cli options to JSON
if (ytdlpConfig.value && ytdlpConfig.value.length > 2 && !ytdlpConfig.value.trim().startsWith('{')) {
if (ytdlpConfig.value && !ytdlpConfig.value.trim().startsWith('{')) {
const response = await request('/api/yt-dlp/convert', {
method: 'POST',
headers: {
@ -175,7 +178,7 @@ const addDownload = async () => {
ytdlpConfig.value = JSON.stringify(data, null, 2)
}
if (ytdlpConfig.value && ytdlpConfig.value.length > 2) {
if (ytdlpConfig.value) {
try {
JSON.parse(ytdlpConfig.value)
} catch (e) {
@ -193,9 +196,9 @@ const addDownload = async () => {
url: url,
preset: config.app.basic_mode ? config.app.default_preset : selectedPreset.value,
folder: config.app.basic_mode ? null : downloadPath.value,
ytdlp_config: config.app.basic_mode ? '' : ytdlpConfig.value,
ytdlp_cookies: config.app.basic_mode ? '' : ytdlpCookies.value,
output_template: config.app.basic_mode ? null : output_template.value,
config: config.app.basic_mode ? '' : ytdlpConfig.value,
cookies: config.app.basic_mode ? '' : ytdlpCookies.value,
template: config.app.basic_mode ? null : output_template.value,
})
})
}
@ -216,8 +219,8 @@ const resetConfig = () => {
toast.success('Local configuration has been reset.')
}
const statusHandler = async data => {
const { status, msg } = JSON.parse(data)
const statusHandler = async stream => {
const { status, msg } = JSON.parse(stream)
addInProgress.value = false
@ -229,11 +232,26 @@ const statusHandler = async data => {
url.value = ''
}
const unlockDownload = async stream => {
const json = JSON.parse(stream)
if (!json?.data) {
return
}
if ("unlock" in json.data && json.data.unlock === true) {
addInProgress.value = false
}
}
onMounted(() => {
socket.on('status', statusHandler)
socket.on('error', unlockDownload)
if ('' === selectedPreset.value) {
selectedPreset.value = config.app.default_preset.value
}
})
onUnmounted(() => socket.off('status', statusHandler))
onUnmounted(() => {
socket.off('status', statusHandler)
socket.off('error', unlockDownload)
})
</script>

14
ui/components/NotificationForm.vue
View file

@ -205,11 +205,8 @@
</template>
<script setup>
import { request } from '~/utils/index'
const emitter = defineEmits(['cancel', 'submit']);
const toast = useToast();
const addInProgress = ref(false);
const props = defineProps({
reference: {
type: String,
@ -223,7 +220,12 @@ const props = defineProps({
item: {
type: Object,
required: true,
}
},
addInProgress: {
type: Boolean,
required: false,
default: false,
},
})
const form = reactive(props.item);
@ -252,16 +254,12 @@ const checkInfo = async () => {
return;
}
// -- validate headers
for (const header of form.request.headers) {
if (!header.key || !header.value) {
form.request.headers.splice(form.request.headers.indexOf(header), 1);
return;
}
}
addInProgress.value = true;
emitter('submit', { reference: toRaw(props.reference), item: toRaw(form) });
}

51
ui/components/TaskForm.vue
View file

@ -113,25 +113,24 @@
</label>
<div class="control has-icons-left">
<input type="text" class="input" id="output_template" placeholder="The output template to use"
v-model="form.output_template" :disabled="addInProgress">
v-model="form.template" :disabled="addInProgress">
<span class="icon is-small is-left"><i class="fa-solid fa-file" /></span>
</div>
<span class="help">
<span class="icon"><i class="fa-solid fa-info" /></span>
<span>The output template to use, if not set, it will defaults to
<code>{{ config.app.output_template }}</code></span>
<code>{{ config.app.template }}</code></span>
</span>
</div>
</div>
<div class="column is-6-tablet is-12-mobile">
<div class="field">
<label class="label is-inline" for="ytdlp_config"
v-tooltip="'Extends current global yt-dlp config. (JSON)'">
<label class="label is-inline" for="config" v-tooltip="'Extends current global yt-dlp config. (JSON)'">
JSON yt-dlp config or CLI options.
</label>
<div class="control">
<textarea class="textarea" id="ytdlp_config" v-model="form.ytdlp_config" :disabled="addInProgress"
<textarea class="textarea" id="config" v-model="form.config" :disabled="addInProgress"
placeholder="--no-embed-metadata --no-embed-thumbnail"></textarea>
</div>
<span class="help">
@ -144,12 +143,11 @@
<div class="column is-6-tablet is-12-mobile">
<div class="field">
<label class="label is-inline" for="ytdlp_cookies" v-tooltip="'JSON exported cookies for downloading.'">
yt-dlp Cookies
<label class="label is-inline" for="cookies" v-tooltip="'JSON exported cookies for downloading.'">
JSON Cookies
</label>
<div class="control">
<textarea class="textarea" id="ytdlp_cookies" v-model="form.ytdlp_cookies"
:disabled="addInProgress"></textarea>
<textarea class="textarea" id="cookies" v-model="form.cookies" :disabled="addInProgress"></textarea>
</div>
<span class="help">
<span class="icon"><i class="fa-solid fa-info" /></span>
@ -195,7 +193,7 @@ import { request } from '~/utils/index'
const emitter = defineEmits(['cancel', 'submit']);
const toast = useToast();
const config = useConfigStore();
const addInProgress = ref(false);
const props = defineProps({
reference: {
type: String,
@ -205,12 +203,22 @@ const props = defineProps({
task: {
type: Object,
required: true,
}
},
addInProgress: {
type: Boolean,
required: false,
default: false,
},
})
const form = reactive(props.task);
onMounted(() => {
if (props.task?.config && (typeof props.task.config === 'object')) {
form.config = JSON.stringify(props.task.config, null, 4);
}
if (!props.task?.preset || '' === props.task.preset) {
form.preset = toRaw(config.app.default_preset);
}
@ -242,13 +250,13 @@ const checkInfo = async () => {
}
// -- send request to convert cli options to JSON
if (form.ytdlp_config && form.ytdlp_config.length > 2 && !form.ytdlp_config.trim().startsWith('{')) {
if (form.config && form.config.length > 2 && !form.config.trim().startsWith('{')) {
const response = await request('/api/yt-dlp/convert', {
method: 'POST',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify({ args: form.ytdlp_config }),
body: JSON.stringify({ args: form.config }),
});
const data = await response.json()
@ -257,20 +265,29 @@ const checkInfo = async () => {
return
}
form.ytdlp_config = JSON.stringify(data, null, 4)
form.config = JSON.stringify(data, null, 4)
}
// -- check config
if (form.config) {
try {
form.config = JSON.parse(form.config)
} catch (e) {
toast.error(`Invalid JSON yt-dlp config. ${e.message}`)
return;
}
}
// -- check cookies syntax
if (form.ytdlp_cookies) {
if (form.cookies) {
try {
JSON.parse(form.ytdlp_cookies);
JSON.parse(form.cookies);
} catch (e) {
toast.error(`Invalid JSON yt-dlp cookies. ${e.message}`)
return;
}
}
addInProgress.value = true;
emitter('submit', { reference: toRaw(props.reference), task: toRaw(form) });
}
</script>

70
ui/layouts/default.vue
View file

@ -10,37 +10,14 @@
</span>
</NuxtLink>
</div>
<div class="navbar-end is-flex" style="flex-flow:wrap">
<div class="navbar-item" v-if="socket.isConnected && config.app.has_cookies && !config.app.basic_mode">
<button class="button is-dark" @click="checkCookies" v-tooltip="'Check youtube cookies status.'"
:disabled="isChecking">
<span class="icon has-text-info"><i class="fas fa-cookie"></i></span>
</button>
</div>
<div class="navbar-item" v-if="socket.isConnected && !config.app.basic_mode">
<button class="button is-dark" @click="pauseDownload" v-if="false === config.paused"
v-tooltip="'Pause non-active downloads.'">
<span class="icon has-text-warning"><i class="fas fa-pause"></i></span>
</button>
<button class="button is-dark" @click="socket.emit('resume', {})" v-else v-tooltip="'Resume downloading.'">
<span class="icon has-text-success"><i class="fas fa-play"></i></span>
</button>
</div>
<div class="navbar-end is-flex" style="flex-flow:wrap">
<div class="navbar-item" v-if="!config.app.basic_mode">
<NuxtLink class="button is-dark has-tooltip-bottom" to="/console" v-tooltip.bottom="'Terminal'">
<span class="icon"><i class="fa-solid fa-terminal" /></span>
</NuxtLink>
</div>
<div class="navbar-item" v-if="!config.app.basic_mode">
<button v-tooltip.bottom="'Toggle Add Form'" class="button is-dark has-tooltip-bottom"
@click="config.showForm = !config.showForm">
<span class="icon"><i class="fa-solid fa-plus" /></span>
</button>
</div>
<div class="navbar-item" v-if="!config.app.basic_mode" v-tooltip.bottom="'Tasks'">
<NuxtLink class="button is-dark has-tooltip-bottom" to="/tasks">
<span class="icon"><i class="fa-solid fa-tasks" /></span>
@ -64,7 +41,7 @@
</button>
</div>
<div class="navbar-item is-hidden-mobile">
<div class="navbar-item">
<button class="button is-dark" @click="reloadPage">
<span class="icon"><i class="fas fa-refresh"></i></span>
</button>
@ -72,9 +49,7 @@
</div>
</nav>
<NewDownload v-if="config.showForm || config.app.basic_mode" @getInfo="url => get_info = url" />
<NuxtPage @getInfo="url => get_info = url" />
<GetInfo v-if="get_info" :link="get_info" @closeModel="get_info = ''" />
<NuxtPage />
<div class="columns mt-3 is-mobile">
<div class="column is-8-mobile">
@ -102,16 +77,12 @@ import 'assets/css/bulma.css'
import 'assets/css/style.css'
import 'assets/css/all.css'
import { useStorage } from '@vueuse/core'
import moment from "moment";
import { request } from '~/utils/index'
import moment from 'moment'
const Year = new Date().getFullYear()
const selectedTheme = useStorage('theme', (() => window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light')())
const socket = useSocketStore()
const config = useConfigStore()
const isChecking = ref(false)
const toast = useToast()
const get_info = ref('')
const applyPreferredColorScheme = scheme => {
for (let s = 0; s < document.styleSheets.length; s++) {
@ -154,31 +125,6 @@ const applyPreferredColorScheme = scheme => {
}
}
const checkCookies = async () => {
if (true === isChecking.value) {
return
}
if (false === confirm(`Check for cookies status?`)) {
return
}
try {
isChecking.value = true
const response = await request('/api/youtube/auth')
const data = await response.json()
if (response.ok) {
toast.success('Succuss. ' + data.message)
} else {
toast.error('Failed. ' + data.message)
}
} catch (e) {
toast.error('Failed to check cookies state. ' + e.message)
} finally {
isChecking.value = false
}
}
onMounted(async () => {
try {
applyPreferredColorScheme(selectedTheme.value)
@ -186,7 +132,6 @@ onMounted(async () => {
}
})
watch(selectedTheme, value => {
try {
applyPreferredColorScheme(value)
@ -195,11 +140,4 @@ watch(selectedTheme, value => {
const reloadPage = () => window.location.reload()
const pauseDownload = () => {
if (false === confirm('Are you sure you want to pause all non-active downloads?')) {
return false
}
socket.emit('pause', {})
}
</script>

87
ui/pages/index.vue
View file

@ -1,14 +1,69 @@
<template>
<div>
<div class="mt-1 columns is-multiline">
<div class="column is-12 is-clearfix is-unselectable">
<span class="title is-4">
<span class="icon-text">
<span class="icon"><i class="fa-solid fa-download" /></span>
<span>Downloads</span>
</span>
</span>
<div class="is-pulled-right" v-if="socket.isConnected && false === config.app.basic_mode">
<div class="field is-grouped">
<p class="control" v-if="config.app.has_cookies">
<button class="button is-purple" @click="checkCookies" v-tooltip.bottom="'Check youtube cookies status.'"
:disabled="isChecking">
<span class="icon"><i class="fas fa-cookie" :class="{ 'is-loading': isChecking }" /></span>
</button>
</p>
<p class="control">
<button class="button is-warning" @click="pauseDownload" v-if="false === config.paused"
v-tooltip.bottom="'Pause non-active downloads.'">
<span class="icon"><i class="fas fa-pause" /></span>
</button>
<button class="button is-danger" @click="socket.emit('resume', {})" v-else
v-tooltip.bottom="'Resume downloading.'">
<span class="icon"><i class="fas fa-play" /></span>
</button>
</p>
<p class="control" v-if="!config.app.basic_mode">
<button v-tooltip.bottom="'Toggle Add Form'" class="button is-primary has-tooltip-bottom"
@click="config.showForm = !config.showForm">
<span class="icon"><i class="fa-solid fa-plus" /></span>
</button>
</p>
</div>
</div>
<div class="is-hidden-mobile">
<span class="subtitle">
Queued and completed downloads are displayed here.
</span>
</div>
</div>
</div>
<NewDownload v-if="config.showForm || config.app.basic_mode" @getInfo="url => get_info = url" />
<GetInfo v-if="get_info" :link="get_info" @closeModel="get_info = ''" />
<Queue @getInfo="url => emitter('getInfo', url)" />
<History @getInfo="url => emitter('getInfo', url)" />
</div>
</template>
<script setup>
import { request } from '~/utils/index'
const emitter = defineEmits(['getInfo'])
const config = useConfigStore()
const stateStore = useStateStore()
const socket = useSocketStore()
const toast = useToast()
const isChecking = ref(false)
const get_info = ref('')
onMounted(() => {
if (!config.app.ui_update_title) {
@ -32,4 +87,36 @@ watch(() => stateStore.queue, () => {
useHead({ title: `YTPTube: ( ${Object.keys(stateStore.queue).length || 0}/${config.app.max_workers} | ${Object.keys(stateStore.history).length || 0} )` })
}, { deep: true })
const checkCookies = async () => {
if (true === isChecking.value) {
return
}
if (false === confirm(`Check for cookies status?`)) {
return
}
try {
isChecking.value = true
const response = await request('/api/youtube/auth')
const data = await response.json()
if (response.ok) {
toast.success('Succuss. ' + data.message)
} else {
toast.error('Failed. ' + data.message)
}
} catch (e) {
toast.error('Failed to check cookies state. ' + e.message)
} finally {
isChecking.value = false
}
}
const pauseDownload = () => {
if (false === confirm('Are you sure you want to pause all non-active downloads?')) {
return false
}
socket.emit('pause', {})
}
</script>

22
ui/pages/notifications.vue
View file

@ -52,8 +52,8 @@ div.is-centered {
</div>
<div class="column is-12" v-if="toggleForm">
<NotificationForm :reference="targetRef" :item="target" @cancel="resetForm(true);" @submit="updateItem"
:allowedEvents="allowedEvents" />
<NotificationForm :addInProgress="addInProgress" :reference="targetRef" :item="target"
@cancel="resetForm(true);" @submit="updateItem" :allowedEvents="allowedEvents" />
</div>
<div class="column is-12" v-if="!toggleForm">
@ -133,6 +133,7 @@ const targetRef = ref('')
const toggleForm = ref(false)
const isLoading = ref(false)
const initialLoad = ref(true)
const addInProgress = ref(false)
watch(() => config.app.basic_mode, async () => {
if (!config.app.basic_mode) {
@ -247,17 +248,20 @@ const updateItem = async ({ reference, item }) => {
notifications.value.push(item)
}
const status = await updateData(notifications.value)
try {
const status = await updateData(notifications.value)
if (!status) {
return
if (!status) {
return
}
toast.success(`Notification target ${reference ? 'updated' : 'added'}.`)
resetForm(true)
} finally {
addInProgress.value = false
}
toast.success(`Notification target ${reference ? 'updated' : 'added'}.`)
resetForm(true)
}
const filterItem = item => {
const { raw, ...rest } = item
return JSON.stringify(rest, null, 2)

66
ui/pages/tasks.vue
View file

@ -31,7 +31,7 @@ div.is-centered {
</p>
<p class="control">
<button class="button is-info" @click="reloadContent" :class="{ 'is-loading': isLoading }"
:disabled="!socket.isConnected || isLoading">
:disabled="!socket.isConnected || isLoading" v-if="tasks && tasks.length > 0">
<span class="icon"><i class="fas fa-refresh"></i></span>
</button>
</p>
@ -46,7 +46,8 @@ div.is-centered {
</div>
<div class="column is-12" v-if="toggleForm">
<TaskForm :reference="taskRef" :task="task" @cancel="resetForm(true);" @submit="updateItem" />
<TaskForm :addInProgress="addInProgress" :reference="taskRef" :task="task" @cancel="resetForm(true);"
@submit="updateItem" />
</div>
<div class="column is-12" v-if="!toggleForm">
@ -73,7 +74,7 @@ div.is-centered {
<p>
<span class="icon"><i class="fa-solid fa-clock" /></span>
<span>
{{ moment(parseExpression(item.timer).next().toISOString()).fromNow() }} - {{ item.timer }}
{{ tryParse(item.timer) }} - {{ item.timer }}
</span>
</p>
<p>
@ -82,7 +83,7 @@ div.is-centered {
</p>
<p>
<span class="icon"><i class="fa-solid fa-file" /></span>
<span>{{ item.output_template ?? config.app.output_template }}</span>
<span>{{ item.template ?? config.app.output_template }}</span>
</p>
<p>
<span class="icon"><i class="fa-solid fa-tv" /></span>
@ -134,6 +135,7 @@ const taskRef = ref('')
const toggleForm = ref(false)
const isLoading = ref(false)
const initialLoad = ref(true)
const addInProgress = ref(false)
watch(() => config.app.basic_mode, async () => {
if (!config.app.basic_mode) {
@ -178,29 +180,39 @@ const reloadContent = async (fromMounted = false) => {
const resetForm = (closeForm = false) => {
task.value = {}
taskRef.value = null
addInProgress.value = false
if (closeForm) {
toggleForm.value = false
}
}
const updateTasks = async tasks => {
const response = await request('/api/tasks', {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(tasks),
})
const updateTasks = async items => {
try {
addInProgress.value = true
const data = await response.json()
const response = await request('/api/tasks', {
method: 'PUT',
headers: {
'Content-Type': 'application/json',
},
body: JSON.stringify(items),
})
if (200 !== response.status) {
const data = await response.json()
if (200 !== response.status) {
toast.error(`Failed to update task. ${data.error}`);
return false
}
tasks.value = data
resetForm(true)
return true
} catch (e) {
toast.error(`Failed to update task. ${data.error}`);
return false
} finally {
addInProgress.value = false
}
tasks.value = data
return true
}
const deleteItem = async item => {
@ -237,7 +249,6 @@ const updateItem = async ({ reference, task }) => {
}
const status = await updateTasks(tasks.value)
if (!status) {
return
}
@ -246,7 +257,6 @@ const updateItem = async ({ reference, task }) => {
resetForm(true)
}
const filterItem = item => {
const { raw, ...rest } = item
return JSON.stringify(rest, null, 2)
@ -259,12 +269,22 @@ const editItem = item => {
}
const calcPath = path => {
if (!path) {
return config.app.download_path
let loc = config.app.download_path
if (path) {
let loc = loc + '/' + sTrim(path, '/')
}
return config.app.download_path + '/' + sTrim(path, '/')
return loc
}
onMounted(async () => socket.isConnected ? await reloadContent(true) : '')
const tryParse = expression => {
try {
return moment(parseExpression(expression).next().toISOString()).fromNow()
} catch (e) {
return "Invalid"
}
}
</script>

4
ui/stores/SocketStore.js
View file

@ -62,7 +62,7 @@ export const useSocketStore = defineStore('socket', () => {
stateStore.add('history', item._id, item);
});
socket.value.on('canceled', stream => {
socket.value.on('cancelled', stream => {
const item = JSON.parse(stream);
const id = item._id
@ -70,7 +70,7 @@ export const useSocketStore = defineStore('socket', () => {
return
}
toast.info(`Download canceled: ${ag(stateStore.get('queue', id, {}), id)}`);
toast.info(`Download cancelled: ${ag(stateStore.get('queue', id, {}), id)}`);
if (true === stateStore.has('queue', id)) {
stateStore.remove('queue', id);

813
ui/yarn.lock
View file

File diff suppressed because it is too large Load diff