Commit graph

10 commits

Author SHA1 Message Date
Daniel
954b13e7b3 fix: a speech model is added with its voices, and Orpheus is sent where it works
An Orpheus id (groq-orpheus-english) did not start with "local-", so
generate_tts_audio sent it down the OpenAI path and every call failed. It
also could be added with no voice, and the "local-%" filters in /tts/voices
and the default lookup hid any non-local voice from learners even when it
was added and marked default.

services/tts_voices.py is the one table of which voices belong to which
model (Kokoro, Orpheus English/Arabic, Fish), the same table the scribe app
keeps. Anything the table knows, or anything local-*, goes through the
LiteLLM gateway with the options its family needs (Orpheus: wav). Adding a
bare model id creates one row per voice with friendly names, so an
administrator adds "groq-orpheus-english" and six voices appear to test one
by one; a voice from another family is refused, naming the ones that work.
Learners are offered every active voice, each saying which model serves it,
and /tts/speak answers with the media type the model actually returned.
Kitten and Supertonic tables go — those models left the gateway.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-13 05:04:37 +02:00
Daniel
3418ed023b fix: WebP figures, the openai SDK removed, and a voice a site can add to
Three things landed together; the message names all of them, because a commit
that mentions one is a commit nobody finds the other two in.

**Figures.** Thirty-four JPEG 2000 files — 21 on questions, the rest unattached
in the media library — are WebP now, with `questions.image_path`,
`questions.explanation_image_path` and `media_assets.path` repointed together.
Serving already converted them on the way out, so nothing was broken; this
removes the step and makes what is stored the same thing that is served. The
originals stay: they are the only copy of what came out of the PDF, they cost a
few megabytes between them, and a conversion nobody can undo is not one to run
against a live bank. Paths are found by what the columns say rather than by
listing a bucket, because three tables record them and updating two would be
worse than none.

**The openai SDK is gone.** Ten call sites — one more than the map said, the
Celery article drafter — every one of them a POST with a JSON body, and not one
reading usage, cost, tool calls or logprobs. Every other call to the same proxy
was already plain httpx: embeddings, the ChromaDB embedding function, speech
both ways, model discovery, the vision probe. So this deletes an abstraction
rather than swapping one for another, and leaves one HTTP client instead of
two. `chat()` and `achat()` return the message content; a `ProxyError` carries
the status and the first 500 characters of the body, which is where the proxy
explains itself.

Behaviour is preserved deliberately, including a 600-second fallback timeout
for the four call sites that were running on the SDK's ten-minute default.
Lowering that is a real change and belongs in its own commit.

Proved against the live proxy on both services rather than only against mocks:
a completion, an async completion, a real 400 the vision probe still classifies
as a refusal, 407 models read from the catalogue, and a word read off an image.

**Voice.** A chosen voice is honoured whatever serves it. The prefix check only
accepted a locally served one, so a site adding a hosted voice would offer it
in Settings, save the learner's choice, and then quietly read every question in
the default voice. The list has always come from the database — adding a voice
is a row in Settings → AI models, never a code change.

And the sign-in page stops offering a locked door: `signup-policy` reports
whether registration is open at all, and the Sign up link goes when it is not.
The switch existed and the only way to discover it was to fill the form in.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
2026-09-12 17:13:05 +02:00
Daniel
3f57bda6aa feat: a reading voice in Settings, and a tutor that reads every figure
The voice picker was a dropdown in the quiz player, beside the question — the
one control on that screen with nothing to do with answering it, and one a
learner sets once and never touches. It is a setting now, on the user rather
than in a Redis blob, with a play button beside each voice because a voice is
worth hearing before it is chosen. Choosing nothing stays a real choice: it
means whatever an administrator marked default, so a site that changes its
default reaches everybody without a row being edited.

The tutor reads figures from `question_media` rather than the two legacy path
columns. Those agree exactly today, so nothing was being lost — the first
question given a second figure in the editor would have been the one that
broke it, silently and only for the tutor. The legacy columns remain as a
fallback for anything not projected into that table yet.

And the retrieval thresholds are written down in docs/retrieval-thresholds.md:
the three answers, the sixteen queries they were measured against, why they are
deliberately not the retrieval floor, and how to re-measure when the corpus
grows. Worth keeping the headline in mind — "discuss love" scores 0.491,
alongside "tell me a joke". A number in the 0.4s is noise, not a weak signal.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
2026-09-12 16:48:09 +02:00
Daniel
fdebda993c Improve quiz TTS and Android release build
Some checks failed
Mobile Android Release / android-release (push) Failing after 1s
2026-05-12 01:01:01 +02:00
Daniel
d59c8bed6f Remove coach modes and improve quiz audio 2026-05-10 01:24:06 +02:00
Daniel
4c48a5cf94 Security hardening, async TeachChat, rate limit UX, unthrottle
Security:
- admin.py: move api_key from URL query params to POST body (litellm/models, tts/voices) — prevents key logging
- admin.py: sanitize exception messages in voice discovery — log internally, return generic errors
- teach.py: log LLM errors server-side, show friendly message to user
- nextcloud.py: normalize path with posixpath.normpath to prevent ../ traversal
- auth.py: check_rate_limit now accepts user param — admins/moderators/unthrottled always exempt

Performance:
- teach.py: make /chat endpoint async, use litellm.acompletion() — no longer blocks a uvicorn thread per request

Features:
- users: add is_unthrottled column (DB migration in setup_pgvector)
- admin.py: PUT /users/{id}/unthrottle endpoint
- AdminPage.jsx: Unlimited/Throttle toggle button per user, shows "unlimited" badge
- Rate limit messages improved with user-friendly context

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 01:09:42 +02:00
Daniel
7f1f14537b Security: fix SQL injection, add rate limiting, markdown in TeachChat
- teach.py, questions.py: replace f-string SQL with parameterized CAST(:vec AS vector) queries
- auth.py: add reusable check_rate_limit() Redis helper
- teach.py: rate limit /chat to 30 req/10min per user
- tts.py: rate limit /speak to 60 req/hr per user
- teach.py: stronger system prompt — no clarifying questions, use markdown, answer directly
- TeachChat.jsx: render assistant messages with ReactMarkdown (already in package.json)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-04 00:45:28 +02:00
Daniel
12d99d3609 Add switchable embedding model, Polly toggle, job cancellation, and UI fixes
Embedding:
- Embedding model now configurable via Admin UI (More tab) or LITELLM_EMBEDDING_MODEL env
- Calls LiteLLM proxy directly via httpx (bypasses LiteLLM library param validation)
- Passes dimensions=1024 to proxy; Redis setting overrides env var
- Default model: ge-gemini-embedding-001 (Gemini AI Studio, 1024-dim)
- Test button in admin UI to verify model works
- Fixed vector_service to use httpx + Redis model (was broken with non-prefixed model names)

Polly:
- Global enable/disable toggle in Admin → More settings (stored in Redis)
- /tts/voices filters out polly/* when disabled
- /tts/speak rejects polly requests when disabled

Job cancellation:
- POST /quizzes/job/{job_id}/cancel endpoint
- Cancel button on JobsPage for running jobs
- Celery task checks Redis status at each chunk boundary and exits cleanly
- Fixes DB lock on restart caused by cancelled jobs leaving open transactions

Admin UI:
- Settings tab renamed to "More" (heading: More Settings)
- Model row overflow fixed (minWidth: 0 + ellipsis on model_id)
- Embedding model search shows all proxy models (no auto-filter by "embed")
- Navbar correctly excludes cancelled/failed jobs from "extracting" count

README:
- Added Rebuild & Restart section with commands
- Updated embedding model reference

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-03 20:44:11 +02:00
Daniel
47ba213ae3 Major platform update: pgvector search, multi-provider TTS, settings page, CLI
Features:
- Hybrid semantic + keyword quiz search (pgvector HNSW + PostgreSQL ILIKE)
- AWS Bedrock Titan Embed V2 embeddings via LiteLLM proxy (0.71 cosine sim)
- Multi-provider TTS: OpenAI, AWS Polly (neural), ElevenLabs, Google Cloud TTS
- Unified Settings page (profile, theme, Nextcloud integration, admin shortcuts)
- Good morning/afternoon greeting on dashboard
- manage.py CLI: reset-password, list-users, reembed
- Email verification enforced: register no longer returns JWT for unverified users
- Quiz search with debounced input, semantic/keyword/title modes, highlighted snippets
- TTS button: loading/playing states, voice selector locked during playback
- TTS auto-stops when navigating between questions
- Footer added; mobile quiz nav overflow fixed; markdown theme body selector fixed
- OpenAI Alloy as default TTS voice; favicon added
- SMTP configured via smtp2go; password reset rate limiting (3/hour)
- PostgreSQL upgraded to pgvector/pgvector:pg16

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 18:03:10 +02:00
ifedan-ed
b876f13fac Initial commit: PDF Quiz Generator app
- FastAPI backend with JWT auth, roles (admin/moderator/user)
- PDF upload (up to 500MB) with streaming, PyMuPDF text extraction
- ChromaDB vectorization per page with metadata
- LiteLLM AI question extraction from PDF (not generation)
- Image extraction from PDF pages, graceful fallback
- Quiz modes: timed (countdown timer) + learning (answers shown inline)
- Page-by-page question navigation with dot navigator
- TTS endpoint using LiteLLM (Google Vertex / OpenAI voices)
- Admin dashboard: AI model management per task, user role management
- Moderator role: upload PDFs, create sections, generate quizzes
- Spaced repetition reminders via SMTP email (SM-2 intervals)
- APScheduler daily reminder jobs
- Celery + Redis for background PDF processing
- React frontend with all pages
- Docker Compose deployment (nginx + backend + celery + redis)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-30 20:04:53 +00:00