Commit graph

20 commits

Author SHA1 Message Date
Daniel
4f5687982d feat: Learning resources can be grounded in the clinical corpus
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 47s
Forgejo Docker Build / Root app tests (push) Successful in 46s
Forgejo Android APK / Build signed APK (push) Successful in 2m2s
Forgejo Docker Build / Build Docker image (push) Successful in 19s
Forgejo Docker Build / Deploy to the host (push) Failing after 1s
Learning generated everything from the model alone. A deck on bronchiolitis was
whatever the model remembered about bronchiolitis, with no connection to the
documents this institution actually indexed — while the assistant had been
searching that corpus all along.

Same collection, deliberately. mcp_bge_m3_1024 is already embedded with
openrouter-bge-m3 at 1024 dimensions; a second index over the same documents
with the same embedder would be a copy that drifts. What differs is the budget:
a chat answer wants a few tight excerpts because the reader is waiting, a
teaching resource synthesises a whole topic. So learning.search_limit and
learning.context_chars default to 30 and 2500 against the assistant's 8 and
1400, and are separate keys so tuning one cannot move the other.

Not unbounded, though. "No limit" only moves the ceiling from a setting to the
model's context window, where overflow truncates the middle of the prompt
silently — the worst place to lose source material. 60 results and 8000
characters per excerpt are the caps.

Opt in per generation: a resource on something the library does not cover is
better written without it than padded with the nearest unrelated excerpts.
Retrieval never fails a generation — the resource is then written from the model
alone, which is what happened before this existed — and every response reports
what it was grounded on, so a caller can say "24 excerpts" or "the library had
nothing on this" rather than quietly serving ungrounded material.

Verified against the live corpus: bronchiolitis, neonatal jaundice and febrile
seizure each returned 12 excerpts and ~23k characters from Nelson, Rudolph and
the Pediatric Clinical Practice Guidelines. A deck generated through the full
chain came back with textbook specificity that is not general recall —
bronchiolar diameter, birth-weight thresholds, the full pathogen list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-11 14:03:05 +02:00
Daniel
689e9bc6c7 fix: the slide prompt carries the rules the renderer actually enforces
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 48s
Forgejo Docker Build / Root app tests (push) Successful in 48s
Forgejo Android APK / Build signed APK (push) Successful in 2m2s
Forgejo Docker Build / Build Docker image (push) Successful in 14s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
Generated a deck with ds-deepseek-v4-flash and rendered it to look at. The model
produced exactly the six headings it was asked for; the deck came out with eight
slides. The extra ones were pandoc's, not the model's.

Two rules, both found by rendering rather than reading:

  - pandoc splits a slide after a table. Anything following one becomes a new
    slide with no title — that was the stray "Key differentials to consider:"
    slide floating with no heading.
  - A table with no blank line before it is not parsed as a table at all. It
    renders as literal pipe characters in the preceding paragraph.

And one that was visible on the slide itself: a nested ordered list inside a
bullet ran off the bottom.

None of these are the model failing. A cheap model writes perfectly good slide
markdown — bold, italics, nested lists and a table with a subscript all came
through correctly. It just needs to be told the shape the renderer wants.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-11 13:29:12 +02:00
Daniel
15a8b399ba feat: slides are built by pandoc from markdown, with a reference template
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 57s
Forgejo Docker Build / Root app tests (push) Successful in 53s
Forgejo Android APK / Build signed APK (push) Successful in 1m59s
Forgejo Docker Build / Build Docker image (push) Has been cancelled
Forgejo Docker Build / Deploy to the host (push) Has been cancelled
pptxgenjs is gone, and with it 269 lines of hand-rolled markdown parsing.

It stretched every image. Reading the slide XML it emitted shows why: it writes
the target box verbatim with <a:stretch/> and a no-op srcRect, so a 200x800
image handed an 11.8x3.9 box came out 1:4 squashed to 3:1. It could not do
better — it never measures an image, and its own getSizeFromImage is commented
out and marked "currently unused", reaching for a package called sizeof that
does not exist. pandoc measures them: a 300x175 source renders at aspect 1.714
and a 160x360 at 0.445, verified by rendering the deck to PDF and looking at it.

Tables, ordered and unordered lists, bold, italic and subscripts all come out
natively, and the fonts, palette and slide layouts come from
assets/learning/slides-reference.pptx. Design now lives in that file: restyling
the decks means editing it in PowerPoint, not editing this route.

Only images the requester owns can reach a deck. pandoc resolves an image link
against the filesystem, so a markdown link naming any local path would read that
file into the presentation. Images are fetched by id through the ownership
check, written into a per-request temporary directory under names we choose, and
every image link that did not resolve is removed rather than passed through. The
directory is removed in a finally block, and the conversion has a 60s timeout so
it cannot hang a request.

pandoc is in the image rather than a sidecar, because an export must not fail
for reasons outside this container. It costs 197MB (307 -> 504).

Removing pptxgenjs also removed image-size, and with it both high-severity
advisories — GHSA-w3rx-r6r6-pgpr and GHSA-5p2g-fcmc-qvqq, ICNS/JXL/HEIF parser
denial of service, ranged <=2.0.2 with no fixed release to upgrade to. npm audit
goes from 2 high and 2 moderate to 2 moderate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-11 13:25:24 +02:00
Daniel
61da9e6bf2 fix: slide images keep their shape, and only safe parsers measure them
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 50s
Forgejo Docker Build / Root app tests (push) Successful in 47s
Forgejo Android APK / Build signed APK (push) Successful in 2m13s
Forgejo Docker Build / Build Docker image (push) Successful in 23s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
Every generated image in an exported deck was distorted. PowerPoint scales an
image to whatever extent it is given, and the export handed it the content box
verbatim. pptxgenjs has a `sizing: contain` option that looks like it solves
this; reading the emitted slide XML shows it does not — a 200x800 image in an
11.8x3.9 box came out as cx=10789920 cy=3566160 with <a:stretch/>, stretched
from 1:4 to 3:1. It cannot do better: it never measures the image, and its own
getSizeFromImage is commented out and marked "currently unused".

So the export measures the image itself and hands PowerPoint an extent that
already has the right shape, centred in the space available. Verified: a 200x800
image now places 0.97x3.90 and a 4x3 places 5.20x3.90, both matching their
source aspect exactly, neither overflowing. An image that cannot be measured
keeps the old behaviour rather than failing the export.

image-size becomes a real dependency rather than one borrowed transitively, and
an override collapses it to a single copy — pptxgenjs declares it but the string
appears in none of its four shipped bundles, so npm was placing a second copy in
the production image that nothing could load.

Its ICNS, JXL and HEIF parsers have open denial-of-service advisories against
every published version (GHSA-w3rx-r6r6-pgpr, GHSA-5p2g-fcmc-qvqq, both ranged
<=2.0.2), so there is no release to upgrade to. They are disabled instead:
this application measures PNG, JPEG, WebP and GIF and nothing else. An ICNS
buffer is now refused and falls back to the box rather than entering the parser.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
2026-09-11 12:46:11 +02:00
Daniel
6be2d1375a feat: integrate durable image jobs/private assets into current core 2026-09-07 16:53:42 +02:00
Daniel
cfaf8e957b feat: ship reviewed prompt history, conversation limits and account protections 2026-09-07 04:01:01 +02:00
Daniel
a08524d95f harden logging and observability 2026-05-08 19:08:19 +02:00
Daniel
6d13765fc4 improve learning hub generation flow 2026-05-08 09:32:58 +02:00
Daniel
2f6e5a7d8f feat: neonatal calculator, DOCX/PPTX/ODT/EPUB support, gateway-agnostic URL helper, TTS/STT fixes
- Add neonatal assessment calculator: GA classification (extremely preterm through
  post term), weight-for-GA percentile (AGA/SGA/LGA) using Fenton 2013 LMS data,
  birth weight category (ELBW/VLBW/LBW/normal/macrosomia)
- Add DOCX support via mammoth, PPTX/ODT/EPUB via jszip in Learning Hub content
  generator file upload
- Add gatewayUrl() helper for consistent API URL construction — handles
  LITELLM_API_BASE with or without /v1 suffix, works with any OpenAI-compatible
  gateway (LiteLLM, Bifrost, etc.)
- Fix TTS model/voice separation: discovery now tags items as MODEL or VOICE,
  auto-detects provider from voice name (Vertex, ElevenLabs, OpenAI)
- Fix STT discovery to include ElevenLabs Scribe and Chirp models
- Fix TTS discovery to include ElevenLabs and Vertex voices alongside models
- Fix admin model test to bypass allowlist check (skipAllowlistCheck) so
  discovered models can be tested before adding
- Fix Nextcloud token decryption in learningAI.js WebDAV browse and file import
- Fix admin embedding test to show DB model name instead of hardcoded default
- Fix admin STT test to use correct endpoint for Whisper models
- Add AI gateway migration guide to configuration docs
- Add Grafana dashboard JSON for Loki log visualization
2026-04-19 02:17:06 +02:00
Daniel
74c5cde8e1 Stop leaking e.message to clients across all routes
88 occurrences of res.status(500).json({ error: e.message }) (or
err.message) swept to generic 'Request failed'. Server-side
console.error / logger.error calls are untouched, so the full detail
still lands in logs and Grafana.

Covers: admin, adminConfig, adminMilestones, chartReview, documents,
encounters, hospitalCourse, hpi, learningAdmin, learningAI, learningHub,
logs, memories, milestones, oidc, refine, sessions, sickVisit, soap,
userPreferences, wellVisit.

Also extends .gitignore to exclude .env.backup-* files.
2026-04-14 03:04:24 +02:00
Daniel
869fa14a77 v6.1: Turnstile bot protection, LiteLLM provider, PPTX tables, audio backup fixes, docs
- Add Cloudflare Turnstile to login, register, and password reset forms
- Switch AI provider to LiteLLM, transcription to OpenAI Whisper
- Change domain to scribe.pedshub.com
- Fix PPTX export: add tables, bold/italic, numbered lists, code blocks, blockquotes
- Fix announcement banner close button (CSP was blocking inline onclick)
- Fix auth middleware: empty Bearer token now falls through to cookie auth
- Fix audio backups: only save on transcription failure, stop auto-deleting on success
- Soften AI correction injection to prevent model hallucination from correction history
- Fix LiteLLM TTS model name handling (no incorrect openai/ prefix)
- Expand AI instructions textarea in Learning Hub CMS
- Update README for v6 with all features and providers
- Add comprehensive docs/: architecture, API reference, database schema,
  authentication, AI providers, speech, learning hub, configuration, deployment
2026-04-04 22:56:24 +02:00
ifedan-ed
dc2e000e88 v4: Fix milestones display + add OpenID auth + 100MB PDF support
FIXES:
- Milestones now show correctly on encounter page (use static fallback if DB empty)
- Static data preserved as MILESTONES_DATA_STATIC for compatibility
- Database-driven milestones still work (admin can edit via CMS)

NEW FEATURES:
- OpenID Connect (OIDC) authentication support (PocketID, Keycloak, Azure AD, etc.)
- Comprehensive setup guide: OPENID_SETUP.md
- Auto-linking existing users by email on SSO login
- Multiple PDF upload support in Learning Hub (up to 10 files)
- 100 MB per file limit (was 20 MB)
- Full PDF content used for AI generation
- Embeddings use first ~8K chars for semantic search

IMPROVEMENTS:
- Updated UI to show multiple file selection with list
- Drag-and-drop supports multiple files
- Better file upload validation and error handling
- Added clarifying comments about embedding truncation
2026-04-01 17:59:51 +00:00
Daniel Onyejesi
1b1fe535da Increase PDF/doc context to 50k chars, raise maxTokens ceiling to 8k
Supports large PDFs (~30 pages) from upload and Nextcloud.
maxTokens is a ceiling, not a target — short topics still get short responses.
2026-03-24 19:22:51 -04:00
Daniel Onyejesi
5e3c95d163 Fix JSON parse for Sonnet 4.6: escape literal newlines in strings
Sonnet 4.6 outputs literal newline characters inside JSON string
values (e.g. in HTML body field) instead of \n escape sequences.
Added sanitizeJsonString that walks the string character by character
and escapes unescaped control chars inside quoted values.
2026-03-24 19:01:16 -04:00
Daniel Onyejesi
4dc14aba0d Fix AI invalid JSON: add system prompt + robust JSON extraction
Opus 4.6 and other large models add preamble/thinking text before
JSON output. Fix: add system message enforcing JSON-only, strip
leading text before first {, trim trailing text after last }, and
log raw output on parse failure for debugging.
2026-03-24 17:52:49 -04:00
Daniel Onyejesi
01f6a38ee2 Fix login flash, quiz in presentations, revert viewer body, feed labels
Login flash:
- Auth screen now hidden by CSS default (#auth-screen{display:none !important})
- auth.js shows it only when there is no valid token or token is expired
- No more flash of login page on refresh for logged-in users

Presentation quiz option (AI Generate panel):
- Presentation type now shows optional quiz question toggle (same as article)
- Backend: when questionCount>0, returns JSON {marpMarkdown, questions[]}
  instead of plain Marp markdown
- applyAiContent fills both the Marp textarea and question blocks for presentations

Viewer revert:
- Removed the viewer presentation card change (user wants markdown as-is)
- Removed dangling viewer button wiring

Feed/CMS list:
- Presentation type shows correct icon (fa-presentation-screen) and
  label "Slides" in the CMS content list
2026-03-24 02:50:31 -04:00
Daniel Onyejesi
9d63625987 AI panel context-aware options, delete wording, httpOnly cookie auth
AI Generate panel:
- Content type synced from editor on open
- Article/Pearl: optional word count field
- Presentation: optional slide count field
- Questions row: checkbox toggle for article/pearl (optional), always on for quiz,
  hidden for presentation; count input shown only when enabled
- Backend: passes wordCount/slideCount to prompt; questionCount=0 skips questions

Delete confirmation:
- Wording built dynamically from content type
- Presentation: no mention of questions
- Quiz: "All quiz questions will be permanently removed"
- Article/pearl: "Any quiz questions attached will also be removed"

Auth refactor — JWT → httpOnly cookie:
- Backend: setAuthCookie() sets ped_auth as httpOnly, Secure, SameSite=Lax, 7d
- Backend: clearAuthCookie() on POST /api/auth/logout (new endpoint)
- Backend: auth middleware reads ped_auth cookie first, falls back to Bearer header
- Login/register: no longer return token in JSON body; cookie is the session
- Frontend: getAuthHeaders() returns only Content-Type (no Authorization header)
- Frontend: session check uses /api/auth/me via cookie (no localStorage token)
- Frontend: clearSession() calls /api/auth/logout to clear cookie server-side
- Frontend: no token in localStorage; window.AUTH_TOKEN removed
- All FormData fetches use credentials:'same-origin' (cookie sent automatically)
- index.html: optimistic auth-screen hide (removed localStorage token check)

Tested: login, logout, /api/auth/me, admin routes, FormData uploads, cookie httpOnly
flag, 401 without cookie, cookie cleared on logout.
2026-03-24 01:46:07 -04:00
Daniel Onyejesi
10620a3152 Fix delete bar always visible, slide preview modal, Marp textarea hint
Delete bar bug:
- Root cause: .lh-delete-confirm-bar{display:flex} defined after .hidden{display:none}
  so flex overrode hidden (no !important on global .hidden rule)
- Fix: remove display:flex from base rule, use .lh-delete-confirm-bar:not(.hidden){display:flex}
  so hidden class always wins regardless of CSS order

Slide preview (replace window.open + document.write approach):
- Backend /preview-slides now returns JSON: {css, slides[]} (individual <section> HTML)
- In-page full-screen modal with dark overlay, white slide frame
- Prev/Next arrow buttons, dot indicators, slide counter
- Keyboard navigation (arrow keys, Escape to close)
- Touch/swipe support for mobile
- Overlay click closes modal
- No popups required, works in all browsers

Marp textarea:
- Placeholder updated to guide users: "Click AI Generate above to create slides"
- Makes it clear AI is the primary authoring path; textarea for review/edits
2026-03-24 01:13:55 -04:00
Daniel Onyejesi
dc40e94749 Delete confirm inline bar, lighter login, Presentation type with Marp+PPTX
Delete confirmation:
- Replaced browser confirm() popup with inline red warning bar in editor
- Shows content title, Cancel and Delete buttons inline in the editor header
- Category delete uses double-click pattern with toast feedback

Login screen:
- Changed from dark blue/purple gradient to light gray background (var(--g50))
- Card now matches 404 page style: white, subtle blue shadow, indigo border

Presentation content type:
- New 'Presentation' button in CMS toolbar (green style)
- Marp markdown textarea editor (dark code editor style) shown when type=presentation
- Body/Tiptap editor hidden; Marp section shown (toggleEditorMode)
- Preview button: renders Marp HTML via @marp-team/marp-core, opens in new tab
- Download PPTX: pure-JS pptxgenjs (no Chromium), parses slides, exports .pptx
- AI generate: returns Marp markdown directly when contentType=presentation
- Body column stores Marp markdown for presentations

Backend:
- POST /api/admin/learning/generate-pptx — pptxgenjs PPTX from Marp markdown
- POST /api/admin/learning/preview-slides — Marp HTML preview

GitHub repo set to private via API.
2026-03-24 00:59:36 -04:00
Daniel Onyejesi
e16e47f634 Add AI content generation to Learning Hub CMS
Backend (src/routes/learningAI.js):
- POST /api/admin/learning/ai-generate
  * Accepts: topic text, uploaded file (PDF/TXT/MD/HTML), or Nextcloud WebDAV path
  * Extracts text from PDFs via pdf-parse; plain text read directly
  * Fetches WebDAV files using stored Nextcloud credentials
  * Prompts AI to return structured JSON: title, subject, body (HTML), questions[]
  * Strips code fences, falls back to regex JSON extraction if needed
- POST /api/admin/learning/ai-refine — refine existing body with instructions
- GET  /api/admin/learning/webdav-browse — PROPFIND-based Nextcloud file browser
- POST /api/admin/learning/webdav-path — save user's default browse path

Frontend:
- AI Generate button in CMS editor header (gradient purple→blue)
- Panel with 3 source tabs: By Topic / Upload File / Nextcloud
- Drag-and-drop file dropzone with DataTransfer API support
- WebDAV file browser: navigate folders, select files, breadcrumb path
- Options: question count, content type (article/quiz/pearl), model selector, instructions
- Refine Current Body: prompt-driven AI rewrite of existing content
- Generated content auto-fills title, subject, type, Tiptap body, quiz questions

Settings:
- Nextcloud section: "Learning Hub Default Browse Path" field (shown when connected)
- Saves to webdav_learning_path column via /api/admin/learning/webdav-path

DB: ALTER TABLE users ADD COLUMN IF NOT EXISTS webdav_learning_path TEXT
2026-03-24 00:17:28 -04:00