**The API.** Every route now lives under `/api/v1`, with `/api/...` rewritten
onto it — one route, two spellings, so they cannot drift and the OpenAPI
document describes each endpoint once. Errors carry an `error` object with a
stable code, one human sentence and, for a validation failure, the fields that
were wrong; `detail` is untouched so nothing that reads it breaks. The whole
surface — 320 routes, their parameters and their status codes — is checked in
as `backend/tests/api-contract.json`, and a test fails on any difference,
naming the routes that moved. `docs/api.md` is the contract in prose.
**Refresh tokens**, so an app can stay signed in without keeping a password.
Rows rather than signatures: listable, withdrawable, stored as hashes, rotated
on every use. A spent token coming back ends the whole session, because a theft
and a replay look identical from the server and the safe reading is the unsafe
one. A browser is not given one — it has nowhere to put it and a person to ask.
**An end-to-end stack**: `docker-compose.test.yml` with its own Postgres and
Redis, `e2e/seed.py` for the smallest world the tests name, and Playwright with
five projects — desktop, iPhone, Pixel, iPad and a browserless API project.
Devices because every bug reported this week was a phone bug found by a person
looking at a screenshot; a desktop-only suite would have passed through all of
them. Forty tests, five clean runs.
It found four things in its first hour:
- **A fresh deploy could not start.** `create_all()` ran before
`CREATE EXTENSION vector`, so any database that had never had pgvector
installed died on the first table with a vector column. Invisible here
because this one has had the extension for a year.
- **A figure in a published article was a 404 for everyone but an admin.**
Media in the library is nobody's to read by default, and nothing made an
exception for a drawing an article actually shows — so every illustration
added this week was an empty box for every real user.
- **Every rate limit was one bucket for the whole site.** The backend saw
nginx's address for every request, so ten bad passwords from anybody locked
out everybody, and no log line could say who. nginx now takes the real
address from the proxy and overwrites the header on the way in; uvicorn runs
with --proxy-headers.
- **The reading page's breakpoints disagreed** — 1150px in the component,
820px in the stylesheet. Between them the menu button claimed the contents
drawer and then toggled a class on a rail that was still in the layout: the
contents did not open and the site menu did not either. The button was dead
on every tablet.
And two smaller ones: the login limiter counted successful sign-ins, so eleven
people behind one hospital NAT locked each other out — it is cleared by a
correct password now; and `/uploads/{path}` served GET and HEAD from one route
with one operation id, which makes every OpenAPI client generator refuse the
document.
The first admin's password is generated and printed once at first start when
`DEFAULT_ADMIN_PASSWORD` is blank, rather than the account not existing:
`docker compose logs backend | grep -A3 "FIRST ADMIN"`.
CI (`.forgejo/workflows/tests.yml`) runs the backend suite, the contract, the
frontend suite and the build on every push to dev, main or master, and the
end-to-end stack on those branches and on pull requests into them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
235 lines
9.8 KiB
Python
235 lines
9.8 KiB
Python
"""Local upload paths and current-reference ACLs; no external URL fetching."""
|
|
from pathlib import Path
|
|
import posixpath
|
|
import re
|
|
import socket
|
|
from urllib.parse import unquote, urlsplit
|
|
|
|
from fastapi import HTTPException
|
|
from sqlalchemy import or_
|
|
from sqlalchemy.orm import load_only
|
|
|
|
from app.config import settings
|
|
from app.models.flashcard import Flashcard, FlashcardDeck
|
|
from app.models.media import MediaAsset
|
|
from app.models.pdf_document import PDFDocument
|
|
from app.models.question import Question
|
|
from app.services.quiz_builder import bank_question_predicate, general_question_predicate
|
|
from app.utils.quiz_access import require_question_access
|
|
|
|
# Left behind by the LMS that used to live here. The code that wrote them is
|
|
# gone; the files are not, so what they may reach is still spelled out.
|
|
# They retain their legacy policy; arbitrary unknown/orphan files do not become public.
|
|
LEGACY_LMS_PREFIXES = ('course_files/', 'course_thumbnails/', 'scorm/', 'certificates/')
|
|
|
|
|
|
def local_upload_path(value):
|
|
if not isinstance(value, str):
|
|
raise HTTPException(400, "Invalid image path")
|
|
path = value.removeprefix("/uploads/")
|
|
if (not path or any(c in path for c in "\\%?#:") or any(ord(c) < 32 for c in path)
|
|
or path.endswith(' ') or any(part in ("", ".", "..") for part in path.split("/"))):
|
|
raise HTTPException(400, "Unsafe upload path")
|
|
return path
|
|
|
|
|
|
def upload_file(path):
|
|
"""Reject even in-root symlink aliases before any ACL lookup."""
|
|
try:
|
|
root = Path(settings.UPLOAD_DIR).resolve()
|
|
target = root
|
|
for part in path.split("/"):
|
|
target = target / part
|
|
if target.is_symlink():
|
|
raise HTTPException(404, "File not found")
|
|
if not target.resolve().is_relative_to(root):
|
|
raise HTTPException(404, "File not found")
|
|
return target
|
|
except (OSError, ValueError):
|
|
raise HTTPException(404, "File not found")
|
|
|
|
|
|
def _authority(url):
|
|
host = unquote(url.hostname or '').rstrip('.').encode('idna').decode('ascii').lower()
|
|
try:
|
|
host = socket.inet_ntoa(socket.inet_aton(host)) # Numeric IPv4 spellings, no DNS/network.
|
|
except OSError:
|
|
pass
|
|
port = url.port
|
|
return host, None if port in (80, 443) else port
|
|
|
|
|
|
def stored_upload_path(value):
|
|
"""Interpret old image strings like the browser, without rewriting stored content."""
|
|
if not isinstance(value, str) or not value:
|
|
return None
|
|
# Match the existing image URL helper's prefix behavior before URL normalization.
|
|
raw = value if re.match(r'^(https?:)?//', value, re.I) or value.startswith('/uploads/') else '/uploads/' + value
|
|
raw = raw.replace('\\', '/').replace('\t', '').replace('\r', '').replace('\n', '')
|
|
raw = raw.strip(''.join(map(chr, range(33))))
|
|
if raw.startswith('//'):
|
|
raw = '//' + raw.lstrip('/')
|
|
try:
|
|
url = urlsplit(raw)
|
|
if url.netloc:
|
|
if url.scheme and url.scheme.lower() not in ('http', 'https'):
|
|
return None
|
|
if _authority(url) != _authority(urlsplit(settings.APP_URL)):
|
|
return None
|
|
elif url.scheme:
|
|
return None
|
|
path = posixpath.normpath(unquote(url.path))
|
|
if not path.startswith('/uploads/'):
|
|
return None
|
|
return local_upload_path(path)
|
|
except (ValueError, UnicodeError, HTTPException):
|
|
return None
|
|
|
|
|
|
def references(db, path):
|
|
# ponytail: scan reference metadata, not question bodies. Add indexed canonical
|
|
# media keys if library size makes this hot; do not cache stale access grants.
|
|
rows = db.query(Question.id, Question.image_path, Question.explanation_image_path).filter(
|
|
or_(Question.image_path.isnot(None), Question.explanation_image_path.isnot(None)),
|
|
).all()
|
|
ids = [qid for qid, stem, explanation in rows
|
|
if stored_upload_path(stem) == path or stored_upload_path(explanation) == path]
|
|
if not ids:
|
|
return []
|
|
return db.query(Question).options(load_only(
|
|
Question.id, Question.source_quiz_id, Question.image_path, Question.explanation_image_path,
|
|
)).filter(Question.id.in_(ids)).all()
|
|
|
|
|
|
def card_deck_ids(db, path):
|
|
return {deck_id for deck_id, image in db.query(Flashcard.deck_id, Flashcard.image_path).filter(
|
|
Flashcard.image_path.isnot(None),
|
|
).all() if stored_upload_path(image) == path}
|
|
|
|
|
|
def document_for_file(db, path):
|
|
for document_id, filename in db.query(PDFDocument.id, PDFDocument.filename).all():
|
|
if stored_upload_path(filename) == path:
|
|
return db.get(PDFDocument, document_id)
|
|
return None
|
|
|
|
|
|
def owns_source(db, path, user, cards):
|
|
if user.is_admin:
|
|
return True
|
|
if path.startswith(f"questions/{user.id}/"):
|
|
return True
|
|
# Recovery of legacy question drafts is not ownership of private card-only files.
|
|
if user.is_moderator and not cards and path.startswith('questions/'):
|
|
return True
|
|
match = re.fullmatch(r"images/doc_(\d+)/.+", path)
|
|
if match and not cards: # Card references stay on card ACL even inside extraction dirs.
|
|
document = db.get(PDFDocument, int(match[1]))
|
|
return document is not None and (document.user_id == user.id or user.is_moderator)
|
|
return False
|
|
|
|
|
|
def shown_in_a_readable_article(db, path, user) -> bool:
|
|
"""True when a published article puts this picture in front of the reader.
|
|
|
|
A figure in the media library is not, on its own, anybody's to read: the
|
|
library is an educator's workspace and grants decide who may manage it. But
|
|
a drawing embedded in a published article is *the article* — refusing it
|
|
leaves a caption above an empty box, which is exactly what every
|
|
illustration in the library did for everyone who was not an administrator.
|
|
|
|
Matched against the article's own prose rather than a link table, because
|
|
an educator writes a figure in by typing a markdown image into a section,
|
|
and there is no row anywhere that says so.
|
|
"""
|
|
from app.models.article import Article
|
|
|
|
if db.query(MediaAsset.id).filter(MediaAsset.path == path).first() is None:
|
|
return False
|
|
query = db.query(Article.id, Article.content, Article.sections).filter(
|
|
Article.deleted_at.is_(None))
|
|
if not user.is_moderator:
|
|
query = query.filter(Article.status == "published")
|
|
for _id, content, sections in query.all():
|
|
if path in (content or ""):
|
|
return True
|
|
for section in sections or []:
|
|
if path in (section.get("content") or ""):
|
|
return True
|
|
return False
|
|
|
|
|
|
def card_access(db, cards, user):
|
|
return bool(cards) and db.query(FlashcardDeck.id).filter(
|
|
FlashcardDeck.id.in_(cards), FlashcardDeck.deleted_at.is_(None),
|
|
or_(FlashcardDeck.user_id == user.id, FlashcardDeck.is_shared == 1, user.is_admin),
|
|
).first() is not None
|
|
|
|
|
|
def can_read_upload(db, path, user, questions, cards, attempt_id=None):
|
|
# Sharing a question must never share its original source PDF.
|
|
document = document_for_file(db, path)
|
|
if document:
|
|
return user.is_moderator or document.user_id == user.id
|
|
if owns_source(db, path, user, cards) or card_access(db, cards, user):
|
|
return True
|
|
if shown_in_a_readable_article(db, path, user):
|
|
return True
|
|
for question in questions:
|
|
try:
|
|
require_question_access(db, question, user, attempt_id,
|
|
review=stored_upload_path(question.image_path) != path)
|
|
return True
|
|
except HTTPException:
|
|
pass
|
|
return False
|
|
|
|
|
|
def validate_image_attachments(db, user, values):
|
|
"""Validate every supplied image before callers mutate any ORM state."""
|
|
result = dict(values)
|
|
for field in ("image_path", "explanation_image_path"):
|
|
value = result.get(field)
|
|
if value is None or value == "":
|
|
continue
|
|
if not isinstance(value, str):
|
|
raise HTTPException(400, "Invalid image path")
|
|
if re.match(r"^https?://", value, re.I):
|
|
try:
|
|
url = urlsplit(value)
|
|
if (not url.hostname or url.username or url.password or '\\' in value
|
|
or '%' in url.netloc or any(ord(c) < 32 for c in value)):
|
|
raise ValueError()
|
|
authority = _authority(url) # Also validates ports/hostname encoding.
|
|
except (ValueError, UnicodeError):
|
|
raise HTTPException(400, "Invalid image URL")
|
|
if authority != _authority(urlsplit(settings.APP_URL)) or not url.path.startswith('/uploads/'):
|
|
continue # External links are never fetched or granted local-file access.
|
|
if url.query or url.fragment:
|
|
raise HTTPException(400, "Use the original upload URL without query parameters or fragments")
|
|
value = url.path
|
|
path = local_upload_path(value)
|
|
upload_file(path)
|
|
document = document_for_file(db, path)
|
|
if document:
|
|
allowed = user.is_moderator or document.user_id == user.id
|
|
else:
|
|
cards = card_deck_ids(db, path)
|
|
refs = references(db, path)
|
|
if refs:
|
|
ids = [question.id for question in refs]
|
|
if user.is_moderator:
|
|
# Legacy LMS media can never be republished into the general bank.
|
|
allowed = db.query(Question.id).filter(
|
|
Question.id.in_(ids), general_question_predicate(),
|
|
).first() is not None
|
|
else:
|
|
allowed = db.query(Question.id).filter(
|
|
Question.id.in_(ids), bank_question_predicate(user),
|
|
).first() is not None
|
|
else:
|
|
allowed = owns_source(db, path, user, cards) or card_access(db, cards, user)
|
|
if not allowed:
|
|
raise HTTPException(403, "Image is not available for attachment")
|
|
result[field] = path
|
|
return result
|