From fac8757ce8edfaa77af7fc75a3ba9b938f37b170 Mon Sep 17 00:00:00 2001 From: Daniel Date: Fri, 11 Sep 2026 15:00:19 +0200 Subject: [PATCH] feat: My Resources has a menu, a library and three downloads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pathway existed but was reachable only by API. It now has a tab of its own next to the Learning Hub — related, not the same thing, and sitting together is how someone discovers the difference — visible to every signed-in user with no role gate in the markup. Generate a deck or an article, see everything you have made, download each as PowerPoint, Word or PDF, delete what you no longer want. The screen says "Private to you" and "Nobody else sees these", because the distinction from published Learning content is the thing a person needs to understand before typing a patient's condition into it. Downloads are fetched rather than linked: an cannot carry the Authorization header. The blob is saved under the filename the server chose and the object URL is revoked afterwards. Resource titles come from a model, so rows are built as elements and a title is only ever assigned to textContent. The e2e stack now joins danvics_convert too. It could previously reach only Postgres and Redis, so a PDF download failed there in a way production would not — which did at least prove the degradation path works: with Gotenberg unreachable the response is "PDF conversion is unavailable right now. PowerPoint and Word still work", and the other two formats download unaffected. Verified in a browser as an ordinary user: the tab appears and opens, the form swaps slide count for word count when the format changes, the library lists their own work, and pptx, docx and pdf all download with sensible filenames (36360, 13285 and 68310 bytes). Also documents retrieval sizing in docs/retrieval-tuning.md — the per-feature budgets, and RERANKER_TOP_K, which caps all of them and had until now appeared in no configuration file at all. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU --- docker-compose.e2e.yml | 10 ++ docs/CLINICAL_ASSISTANT.md | 3 + docs/configuration.md | 6 + docs/learning-hub.md | 6 + docs/retrieval-tuning.md | 141 ++++++++++++++++++ public/components/my-resources.html | 69 +++++++++ public/index.html | 6 + public/js/myResources.js | 218 ++++++++++++++++++++++++++++ test/my-resources.test.js | 31 ++++ 9 files changed, 490 insertions(+) create mode 100644 docs/retrieval-tuning.md create mode 100644 public/components/my-resources.html create mode 100644 public/js/myResources.js diff --git a/docker-compose.e2e.yml b/docker-compose.e2e.yml index e475873f..a03122c2 100644 --- a/docker-compose.e2e.yml +++ b/docker-compose.e2e.yml @@ -18,6 +18,12 @@ services: image: ped-ai-local:latest ports: - "127.0.0.1:3553:3000" + networks: + # Its own project network, plus the converter so PDF export is exercised + # here too. Without this the e2e stack could only reach Postgres and + # Redis, and a PDF download failed in a way production would not. + - default + - danvics_convert env_file: - .env environment: @@ -57,3 +63,7 @@ services: volumes: scribe-logs-e2e: + +networks: + danvics_convert: + external: true diff --git a/docs/CLINICAL_ASSISTANT.md b/docs/CLINICAL_ASSISTANT.md index e9923083..489c8f37 100644 --- a/docs/CLINICAL_ASSISTANT.md +++ b/docs/CLINICAL_ASSISTANT.md @@ -110,6 +110,9 @@ default in the right-hand column. | `clinical_assistant.image_model_roster` | Image models an admin added from Admin → Image Generation (**+ Add**). This is the pool the Image models tick-list offers; it is not itself an allowlist. Validated as up to 100 ids | | `clinical_assistant.search_limit` | Number of MCP results requested | | `clinical_assistant.context_chars` | Context characters requested from MCP | + +These are capped by `RERANKER_TOP_K` in the MCP deployment, which is the real +ceiling on every search. See [retrieval-tuning.md](retrieval-tuning.md). | `clinical_assistant.conversation_chars` | Input budget in UTF-16 code units. Empty means use `CLINICAL_ASSISTANT_CONVERSATION_CHARS`; a value must be 1000-1000000 | | `clinical_assistant.show_sources` | `true`/`false`. Display only: hides the Sources panel and the citation markers. The prompt, the retrieval and the stored answer are byte-for-byte identical either way, so it cannot bias an answer; turning it back on restores the citations | | `clinical_assistant.preview_enabled` | `true`/`false`. Lets signed-out visitors try the assistant read-only; anything needing an account asks them to sign in | diff --git a/docs/configuration.md b/docs/configuration.md index 5564d99e..d8e6583d 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -198,3 +198,9 @@ OpenAI-compatible gateway — LiteLLM, Bifrost, or other proxies. | Bifrost | `provider/model` | Virtual keys, semantic caching, MCP gateway | | LiteLLM | Custom aliases | Requires PostgreSQL + Redis | | Any OpenAI-compatible | Varies | Must serve `/v1/chat/completions`, `/v1/audio/speech`, `/v1/audio/transcriptions`, `/v1/embeddings` | + +## Retrieval sizing + +How many corpus excerpts the Clinical Assistant, the Learning Hub and My +Resources each receive, and the reranker cap that overrides all three: +[retrieval-tuning.md](retrieval-tuning.md). diff --git a/docs/learning-hub.md b/docs/learning-hub.md index a82afd29..253b22f7 100644 --- a/docs/learning-hub.md +++ b/docs/learning-hub.md @@ -85,3 +85,9 @@ available. | `learning_questions` | Quiz question prompts (FK to content) | | `learning_options` | Answer options (FK to question) | | `learning_progress` | Per-user attempt history | + +## Retrieval sizing + +How many corpus excerpts the Clinical Assistant, the Learning Hub and My +Resources each receive, and the reranker cap that overrides all three: +[retrieval-tuning.md](retrieval-tuning.md). diff --git a/docs/retrieval-tuning.md b/docs/retrieval-tuning.md new file mode 100644 index 00000000..fb1ca673 --- /dev/null +++ b/docs/retrieval-tuning.md @@ -0,0 +1,141 @@ +# Retrieval tuning — how many excerpts each feature gets + +Three features read from the same clinical corpus, and each takes a different +amount of it. This is where the numbers live and what actually changes them. + +Everything here is a Milvus collection called `mcp_bge_m3_1024`, embedded with +`openrouter-bge-m3` at 1024 dimensions, searched through the clinical MCP +(`clinical-assist-query`, deployed from `clinical-assist-deploy/`). There is one +corpus. Only the budgets differ. + +## The one number that caps everything + +`RERANKER_TOP_K`, in `clinical-assist-deploy/docker-compose.yml`. + +A search runs in two stages. Milvus returns a wide set of candidates by vector +similarity, then a reranker (`cohere-rerank-v4.0-pro`) scores each against the +query and keeps the best. `rerank_results()` takes `min(reranker_top_k, limit)`, +so **this value is the ceiling on every search, regardless of what the caller +asks for**. With it at 12, an app requesting 30 excerpts receives 12. + +This caused real confusion before it was written down: it was a library default +with no mention in any config file, so nothing explained where 12 came from. + +```yaml +# clinical-assist-deploy/docker-compose.yml — set on both mcp and mcp-indexer +- RERANKER_TOP_K=${RERANKER_TOP_K:-12} +- RERANKER_FETCH_MULTIPLIER=${RERANKER_FETCH_MULTIPLIER:-5} +``` + +To change it: + +```bash +cd /home/danvics/docker/clinical-assist-deploy +# either edit the default in docker-compose.yml, or set it in .env +echo 'RERANKER_TOP_K=20' >> .env +docker compose up -d mcp mcp-indexer +docker inspect mcp-server-mcp-1 --format '{{range .Config.Env}}{{println .}}{{end}}' | grep RERANKER_TOP_K +``` + +`RERANKER_FETCH_MULTIPLIER` decides how many candidates the reranker sees: +`candidate_limit = max(limit, limit × multiplier)`. Raising it gives the +reranker more to choose from at the cost of a larger Milvus query and a larger +rerank call. 5 is the default and has not needed changing. + +### Is 12 enough? + +It is what the clinical assistant has always answered from, and 12 reranked +excerpts at 2500 characters is roughly 23,000 characters of closely matched +material — enough that a generated teaching resource reads with textbook +specificity (bilirubin production rates, conjugation timelines, thresholds in +mg/dL, all traceable to the indexed books). + +Raising it to 30 was tried and reverted. The reranker exists precisely to +discard near-misses; asking for more of what it already rejected adds length, +not signal. Raise it if a topic is genuinely broad and the output feels thin — +not by default. + +## Per-feature budgets + +These live in the `app_settings` table, are read live (2-minute cache), and are +clamped on read so a bad value cannot break a search. + +| Feature | Keys | Default | Clamp | +|---|---|---|---| +| Clinical Assistant | `clinical_assistant.search_limit`, `clinical_assistant.context_chars` | 8, 1400 | 3–20, 300–4000 | +| Learning Hub | `learning.search_limit`, `learning.context_chars` | 30, 2500 | 3–60, 300–8000 | +| My Resources | *the same `learning.*` keys* | 30, 2500 | 3–60, 300–8000 | + +`search_limit` is how many excerpts to request; `context_chars` is how much text +to pull around each one. + +**My Resources shares the Learning budget deliberately.** Both generate a whole +teaching resource from a topic, so they want the same shape of context. If they +ever need to diverge, `src/utils/learningRetrieval.js` is the single place that +reads these keys. + +Why the assistant is so much smaller: a chat answer is a paragraph and the +reader is waiting. A teaching resource synthesises an entire topic. Tuning one +must never move the other, which is why they are separate keys rather than one +shared pair. + +To change one: + +```sql +-- from the postgres container +INSERT INTO app_settings (key, value) VALUES ('learning.search_limit', '20') + ON CONFLICT (key) DO UPDATE SET value = EXCLUDED.value; +``` + +Remember the ceiling: setting `learning.search_limit` above `RERANKER_TOP_K` +changes nothing. Raise the reranker cap first. + +## Reading what actually happened + +The MCP logs every search and what survived reranking: + +```bash +docker logs mcp-server-mcp-1 --since 10m 2>&1 | grep -E "reranked search|before reranking|unverified" +# Milvus reranked search: user=..., limit=60, score_threshold=0.0, doc_type=file +# Milvus candidate retrieval returned 600 results before reranking +# Returning 12 unverified reranked results +``` + +Note `limit=60` for a request of 30: `semantic.py` asks the algorithm for +`limit × 2` and trims after verification. + +Generation responses carry the same fact, so a caller never has to guess whether +a resource was grounded: + +```json +"grounding": { "used": true, "count": 12, "reason": null } +``` + +`used: false` with a `reason` means the resource was written from the model +alone — retrieval never fails a generation, because ungrounded material is a far +better outcome than an error page. The Learning screen and My Resources both +show this, so ungrounded output is never presented as grounded. + +## A caution on raising these + +Context is not free and more is not automatically better. + +* The prompt has to fit the model's window. 12 excerpts at 2500 characters is + about 23k characters (~6k tokens); 30 at 2500 is about 57k (~14k). Overflow + does not error — it truncates, and truncation lands in the middle of the + excerpt block, which is the worst place to lose source material. If a resource + starts ignoring obvious material, lower `context_chars` before suspecting the + model. +* Every excerpt past the reranker's confident set is a near-miss. Ten strong + excerpts beat thirty mediocre ones for a model trying to write accurately. +* The reranker is billed per call and scales with candidates, not results. + `RERANKER_FETCH_MULTIPLIER` is the cost lever, not `RERANKER_TOP_K`. + +## Where each number is read + +| Number | Read by | File | +|---|---|---| +| `RERANKER_TOP_K` | clinical-assist | `clinical_assist/search/reranker.py` | +| `RERANKER_FETCH_MULTIPLIER` | clinical-assist | `clinical_assist/search/milvus_reranked.py` | +| `clinical_assistant.*` | ped-ai | `src/routes/clinicalAssistant.js` | +| `learning.*` | ped-ai | `src/utils/learningRetrieval.js` | diff --git a/public/components/my-resources.html b/public/components/my-resources.html new file mode 100644 index 00000000..1a681af2 --- /dev/null +++ b/public/components/my-resources.html @@ -0,0 +1,69 @@ +
+
+

My Resources

+ Private to you +
+
+

+ Teaching material you generate for yourself — a deck for tomorrow's session, a + handout, a summary. Nobody else sees these. Published Learning Hub content is separate + and stays with the moderators. +

+ +
+ + +
+ +
+ +
+ + + +
+
+ +
+ Clinical library +
+ +

+ Searches the corpus for your topic and writes from those excerpts, preferring them + over the model's own recall, and ends with a References section. Turn this off for a + topic the library does not cover. +

+
+
+ +
+ + +
+ +
+ + +
+
+
+ +
+
+

Library

+ +
+
+
diff --git a/public/index.html b/public/index.html index 85eedc46..4da90009 100644 --- a/public/index.html +++ b/public/index.html @@ -266,6 +266,10 @@ Diagrams +