feat: image fallback chains for every workflow, and a library worth looking at
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 55s
Forgejo Docker Build / Root app tests (push) Successful in 45s
Forgejo Android APK / Build signed APK (push) Successful in 2m5s
Forgejo Docker Build / Build Docker image (push) Successful in 19s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 55s
Forgejo Docker Build / Root app tests (push) Successful in 45s
Forgejo Android APK / Build signed APK (push) Successful in 2m5s
Forgejo Docker Build / Build Docker image (push) Successful in 19s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s
**Fallbacks.** One image model meant a refusal, a rate limit or a model the gateway had since dropped ended as a missing picture. Every workflow now tries its model, then each fallback in order, stopping at the first that produces an image. Primary plus two, capped: each hop is a paid request, and a chain long enough to need a cap is long enough to surprise someone. My Resources previously had no fallback at all — only the Clinical Assistant did, and only one. That is backwards: a missing figure is most visible in a deck, where it leaves a hole in a slide. The retry rule is now a classifier that says *why*, rather than a boolean. Transient faults, a 404 for a model the gateway does not have, and a content refusal all move to the next model — a refusal because policy is a vendor decision, not a fact about the request. 401/403 stop immediately (one gateway, one set of credentials, the next model fails identically), as do 413 and any other 4xx, which are malformed everywhere. Refusals are recognised from the message: no provider sends a machine-readable reason and the status varies. Each hop re-leases the job, so a chain cannot outlive its claim and let a second worker repeat the same paid work, and the row records the model actually being paid for so a picture made by the third model is not attributed to the first. The old singular `fallback_image_model` is still read, so an existing configuration keeps working without anyone re-entering it. **Library.** Documents/Images tabs in My Resources, with a real grid: fixed aspect tiles so the rows line up whatever shape the pictures are, a source badge on the picture, two-line prompt, hover lift, shimmer skeletons while thumbnails land, and a lightbox that closes on Escape or the backdrop and restores focus. Actions are hidden on hover only behind `@media (hover:hover)` — hiding delete behind :hover would put it out of reach on touch and keyboard. Downloads go through privateImageBlob rather than a bare `<a download href>`: a mobile client's session is a bearer token an anchor cannot send, and these assets are served no-store on purpose. The gallery lives in My Resources only. Assistant images appear in it, which was the point; the assistant page does not grow a gallery of its own, and a test asserts no assistant module lists the endpoint. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
This commit is contained in:
parent
259b4858be
commit
03621752e8
10 changed files with 653 additions and 105 deletions
|
|
@ -272,6 +272,37 @@ tiles cost a few kB each rather than thirty full-size downloads. Every fetch
|
||||||
goes through `hydrateImage`, never a bare `src`: assets are served `no-store`
|
goes through `hydrateImage`, never a bare `src`: assets are served `no-store`
|
||||||
and a bare `src` would not carry the session on a mobile client.
|
and a bare `src` would not carry the session on a mobile client.
|
||||||
|
|
||||||
|
### Image model fallbacks
|
||||||
|
|
||||||
|
Every workflow tries its configured model first, then each fallback in order,
|
||||||
|
stopping at the first that produces an image. Primary plus two, capped — each
|
||||||
|
hop is a paid request. Set in **Admin → Image models**.
|
||||||
|
|
||||||
|
A fallback is only tried where another model has a real chance:
|
||||||
|
|
||||||
|
| Failure | Next model? | Why |
|
||||||
|
|---|---|---|
|
||||||
|
| Timeout, 429, 5xx, network fault | yes | The provider said "not now", not "not ever" |
|
||||||
|
| 404 — the gateway does not have that model | yes | A configuration mistake the next model rescues |
|
||||||
|
| A content refusal | yes | Policy is a vendor decision, not a fact about the request |
|
||||||
|
| 401 / 403 | **no** | One gateway, one set of credentials; the next model fails identically |
|
||||||
|
| 413 — too large | **no** | It is too large everywhere |
|
||||||
|
| Any other 4xx | **no** | Malformed is malformed everywhere |
|
||||||
|
| Cancelled, or shutting down | **no** | Never start more paid work |
|
||||||
|
|
||||||
|
A refusal is recognised from the message, because no provider sends a
|
||||||
|
machine-readable reason and the status varies — 400 from some, 422 from others.
|
||||||
|
|
||||||
|
Each hop re-leases the job, so a chain cannot outlive its claim and let a second
|
||||||
|
worker repeat the same paid work; if the lease has gone the attempt stops there
|
||||||
|
rather than paying again. The row records the model actually being paid for, so
|
||||||
|
a picture made by the third model is not attributed to the first, and every hop
|
||||||
|
is logged with the reason it moved on.
|
||||||
|
|
||||||
|
This used to be the Clinical Assistant alone, with one fallback. My Resources
|
||||||
|
had none at all — which is where a missing picture is most visible, because it
|
||||||
|
leaves a hole in a slide.
|
||||||
|
|
||||||
### Deleting
|
### Deleting
|
||||||
|
|
||||||
`DELETE /api/generated-images/:id` removes the bytes before the row, and refuses
|
`DELETE /api/generated-images/:id` removes the bytes before the row, and refuses
|
||||||
|
|
|
||||||
|
|
@ -177,7 +177,7 @@
|
||||||
<div id="mr-images-panel" role="tabpanel" aria-labelledby="tab-mr-images" hidden
|
<div id="mr-images-panel" role="tabpanel" aria-labelledby="tab-mr-images" hidden
|
||||||
style="padding:12px 16px;max-height:420px;overflow-y:auto;">
|
style="padding:12px 16px;max-height:420px;overflow-y:auto;">
|
||||||
<p id="mr-images-empty" style="margin:0;font-size:12px;color:var(--g500);">Loading…</p>
|
<p id="mr-images-empty" style="margin:0;font-size:12px;color:var(--g500);">Loading…</p>
|
||||||
<div id="mr-images-grid" style="display:grid;grid-template-columns:repeat(auto-fill,minmax(150px,1fr));gap:10px;"></div>
|
<div id="mr-images-grid" class="img-grid"></div>
|
||||||
<div style="display:flex;justify-content:center;padding-top:10px;">
|
<div style="display:flex;justify-content:center;padding-top:10px;">
|
||||||
<button id="btn-mr-images-more" class="btn-sm btn-ghost" type="button" hidden>Load more</button>
|
<button id="btn-mr-images-more" class="btn-sm btn-ghost" type="button" hidden>Load more</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
|
||||||
|
|
@ -1494,3 +1494,64 @@ button, a, .btn-sm, .btn-generate, .btn-send, .tab-btn, input, textarea, select,
|
||||||
@media (max-width:640px) {
|
@media (max-width:640px) {
|
||||||
.assistant-history .assistant-mode-switch { margin:6px 14px 10px; }
|
.assistant-history .assistant-mode-switch { margin:6px 14px 10px; }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* ── Image library ──────────────────────────────────────────────────────────
|
||||||
|
A grid of pictures, not a list of filenames: an image is recognised by
|
||||||
|
looking at it. Styles live here rather than inline so the hover and focus
|
||||||
|
states can exist at all — an inline style cannot express either. */
|
||||||
|
.img-grid{display:grid;grid-template-columns:repeat(auto-fill,minmax(168px,1fr));gap:12px;}
|
||||||
|
.img-tile{margin:0;display:flex;flex-direction:column;background:var(--white,#fff);
|
||||||
|
border:1px solid var(--g200);border-radius:10px;overflow:hidden;
|
||||||
|
transition:border-color .15s ease, box-shadow .15s ease, transform .15s ease;}
|
||||||
|
.img-tile:hover{border-color:var(--g300);box-shadow:0 6px 16px rgba(17,24,39,.08);transform:translateY(-1px);}
|
||||||
|
.img-tile:focus-within{border-color:var(--blue);box-shadow:0 0 0 3px var(--blue-light);}
|
||||||
|
|
||||||
|
/* A fixed ratio so the grid lines up whatever shape the pictures are, and a
|
||||||
|
neutral ground so a transparent PNG does not read as a hole in the page. */
|
||||||
|
.img-tile-frame{position:relative;aspect-ratio:4/3;background:var(--g100);
|
||||||
|
display:flex;align-items:center;justify-content:center;overflow:hidden;cursor:zoom-in;border:0;padding:0;width:100%;}
|
||||||
|
.img-tile-frame img{width:100%;height:100%;object-fit:cover;display:block;}
|
||||||
|
.img-tile-frame:focus-visible{outline:2px solid var(--blue);outline-offset:-2px;}
|
||||||
|
|
||||||
|
/* The source badge sits on the picture: it is one word, and a row of its own
|
||||||
|
would cost more space than it is worth. */
|
||||||
|
.img-tile-badge{position:absolute;top:8px;left:8px;padding:2px 7px;border-radius:999px;
|
||||||
|
font-size:10px;font-weight:600;letter-spacing:.02em;background:rgba(17,24,39,.72);color:#fff;
|
||||||
|
backdrop-filter:blur(2px);}
|
||||||
|
.img-tile-body{padding:9px 10px 10px;display:flex;flex-direction:column;gap:6px;min-width:0;}
|
||||||
|
.img-tile-prompt{font-size:11.5px;line-height:1.4;color:var(--g700);
|
||||||
|
display:-webkit-box;-webkit-line-clamp:2;-webkit-box-orient:vertical;overflow:hidden;}
|
||||||
|
.img-tile-meta{font-size:10.5px;color:var(--g500);font-variant-numeric:tabular-nums;}
|
||||||
|
|
||||||
|
/* Actions appear on hover on a pointer device, and are always present for
|
||||||
|
keyboard and touch — hiding them behind hover would put delete out of reach. */
|
||||||
|
.img-tile-actions{display:flex;gap:6px;margin-top:2px;opacity:1;transition:opacity .15s ease;}
|
||||||
|
@media (hover:hover){
|
||||||
|
.img-tile-actions{opacity:0;}
|
||||||
|
.img-tile:hover .img-tile-actions,
|
||||||
|
.img-tile:focus-within .img-tile-actions{opacity:1;}
|
||||||
|
}
|
||||||
|
.img-tile-action{border:1px solid var(--g200);background:var(--white,#fff);color:var(--g600);
|
||||||
|
font:inherit;font-size:11px;padding:3px 9px;border-radius:6px;cursor:pointer;}
|
||||||
|
.img-tile-action:hover{background:var(--g50);border-color:var(--g300);}
|
||||||
|
.img-tile-action.danger{color:var(--red);}
|
||||||
|
.img-tile-action.danger:hover{background:var(--red-light);border-color:var(--red);}
|
||||||
|
.img-tile-action[disabled]{opacity:.5;cursor:default;}
|
||||||
|
|
||||||
|
/* Skeletons while the thumbnails arrive, so the grid does not reflow under the
|
||||||
|
reader as each one lands. */
|
||||||
|
.img-tile-frame.loading{background:linear-gradient(90deg,var(--g100) 25%,var(--g200) 37%,var(--g100) 63%);
|
||||||
|
background-size:400% 100%;animation:img-shimmer 1.4s ease infinite;}
|
||||||
|
@keyframes img-shimmer{0%{background-position:100% 0}100%{background-position:0 0}}
|
||||||
|
@media (prefers-reduced-motion:reduce){
|
||||||
|
.img-tile,.img-tile-actions{transition:none}
|
||||||
|
.img-tile-frame.loading{animation:none}
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Full-size view. */
|
||||||
|
.img-lightbox{position:fixed;inset:0;z-index:9999;background:rgba(17,24,39,.82);
|
||||||
|
display:flex;flex-direction:column;align-items:center;justify-content:center;gap:12px;padding:24px;}
|
||||||
|
.img-lightbox img{max-width:100%;max-height:calc(100vh - 140px);object-fit:contain;
|
||||||
|
border-radius:8px;box-shadow:0 16px 48px rgba(0,0,0,.45);background:var(--g100);}
|
||||||
|
.img-lightbox-bar{display:flex;gap:8px;align-items:center;flex-wrap:wrap;justify-content:center;max-width:640px;}
|
||||||
|
.img-lightbox-caption{color:#e5e7eb;font-size:12px;line-height:1.5;text-align:center;}
|
||||||
|
|
|
||||||
|
|
@ -1,8 +1,21 @@
|
||||||
// One image model per workflow: Clinical Assistant and Learning, each a dropdown of discovered models.
|
// One image model per workflow, plus the models to try when it fails.
|
||||||
|
//
|
||||||
|
// A single model means a refusal, a rate limit or a model the gateway has since
|
||||||
|
// dropped ends as a missing picture. The chain is primary first, then each
|
||||||
|
// fallback in order, and it stops at the first one that produces an image —
|
||||||
|
// every hop being a paid request is why it is capped rather than open-ended.
|
||||||
import { imageJson } from '../generatedImages.js';
|
import { imageJson } from '../generatedImages.js';
|
||||||
|
|
||||||
|
const WORKFLOWS = [
|
||||||
|
{ key: 'clinical_assistant', label: 'Clinical Assistant' },
|
||||||
|
{ key: 'learning_hub', label: 'Learning Hub' },
|
||||||
|
{ key: 'my_resources', label: 'My Resources' }
|
||||||
|
];
|
||||||
|
|
||||||
let loading = false;
|
let loading = false;
|
||||||
let loaded = false;
|
let loaded = false;
|
||||||
let clinicalSelect; let learningSelect; let clinicalBudget; let learningBudget; let save; let status;
|
let controls = {};
|
||||||
|
let save; let status; let maxModels = 3;
|
||||||
|
|
||||||
async function load() {
|
async function load() {
|
||||||
const root = document.getElementById('workflow-image-settings');
|
const root = document.getElementById('workflow-image-settings');
|
||||||
|
|
@ -14,50 +27,122 @@ async function load() {
|
||||||
imageJson('/api/admin/image-settings'),
|
imageJson('/api/admin/image-settings'),
|
||||||
imageJson('/api/admin/config/image-models/discover').catch(() => ({ success: false }))
|
imageJson('/api/admin/config/image-models/discover').catch(() => ({ success: false }))
|
||||||
]);
|
]);
|
||||||
|
maxModels = Number(settings.maxModels) || 3;
|
||||||
|
const discovered = Array.isArray(models.models) ? models.models : [];
|
||||||
root.textContent = '';
|
root.textContent = '';
|
||||||
|
|
||||||
const form = document.createElement('form');
|
const form = document.createElement('form');
|
||||||
const heading = document.createElement('h4'); heading.textContent = 'Image models';
|
const heading = document.createElement('h4');
|
||||||
clinicalSelect = makeSelect([settings.workflows.clinical_assistant.model], models.models);
|
heading.textContent = 'Image models';
|
||||||
learningSelect = makeSelect([settings.workflows.learning_hub.model], models.models);
|
const note = document.createElement('p');
|
||||||
clinicalBudget = makeBudget(settings.workflows.clinical_assistant.budget);
|
note.style.cssText = 'margin:2px 0 10px;font-size:12px;color:var(--g500);';
|
||||||
learningBudget = makeBudget(settings.workflows.learning_hub.budget);
|
note.textContent = 'Each workflow tries its model first, then its fallbacks in order, ' +
|
||||||
save = document.createElement('button'); save.type = 'submit'; save.className = 'btn-sm btn-primary'; save.textContent = 'Save image settings';
|
'stopping at the first that produces an image. A fallback is only tried when another ' +
|
||||||
|
'model has a real chance — a refusal, a rate limit, a provider fault, or a model the ' +
|
||||||
|
'gateway does not have. Bad credentials and a malformed request stop immediately.';
|
||||||
|
form.append(heading, note);
|
||||||
|
|
||||||
|
controls = {};
|
||||||
|
WORKFLOWS.forEach(workflow => {
|
||||||
|
const saved = settings.workflows[workflow.key] || {};
|
||||||
|
const group = document.createElement('fieldset');
|
||||||
|
group.style.cssText = 'border:1px solid var(--g200);border-radius:8px;padding:12px;margin:0 0 12px;';
|
||||||
|
const legend = document.createElement('legend');
|
||||||
|
legend.style.cssText = 'font-size:12px;font-weight:600;color:var(--g700);padding:0 6px;';
|
||||||
|
legend.textContent = workflow.label;
|
||||||
|
group.appendChild(legend);
|
||||||
|
|
||||||
|
// My Resources picks its model per request from the roster, so only the
|
||||||
|
// Learning Hub and the Assistant name a primary here.
|
||||||
|
const primary = makeSelect([saved.model], discovered);
|
||||||
|
primary.disabled = workflow.key === 'my_resources';
|
||||||
|
group.appendChild(row(workflow.key === 'my_resources' ? 'Model (set per request)' : 'Model', primary));
|
||||||
|
|
||||||
|
const fallbacks = [];
|
||||||
|
for (let i = 0; i < maxModels - 1; i++) {
|
||||||
|
const select = makeSelect([(saved.fallbacks || [])[i] || ''], discovered, true);
|
||||||
|
fallbacks.push(select);
|
||||||
|
group.appendChild(row('Fallback ' + (i + 1), select));
|
||||||
|
}
|
||||||
|
|
||||||
|
const budget = makeBudget(saved.budget);
|
||||||
|
group.appendChild(row('Image input budget', budget));
|
||||||
|
|
||||||
|
controls[workflow.key] = { primary, fallbacks, budget };
|
||||||
|
form.appendChild(group);
|
||||||
|
});
|
||||||
|
|
||||||
|
save = document.createElement('button');
|
||||||
|
save.type = 'submit'; save.className = 'btn-sm btn-primary';
|
||||||
|
save.textContent = 'Save image settings';
|
||||||
status = document.createElement('p'); status.setAttribute('role', 'status');
|
status = document.createElement('p'); status.setAttribute('role', 'status');
|
||||||
const row1 = document.createElement('div'); row1.className = 'admin-row';
|
form.append(save, status);
|
||||||
const l1 = document.createElement('label'); l1.className = 'admin-row-label'; l1.textContent = 'Clinical Assistant image model'; row1.append(l1, clinicalSelect);
|
|
||||||
const row2 = document.createElement('div'); row2.className = 'admin-row';
|
|
||||||
const l2 = document.createElement('label'); l2.className = 'admin-row-label'; l2.textContent = 'Clinical image input budget'; row2.append(l2, clinicalBudget);
|
|
||||||
const row3 = document.createElement('div'); row3.className = 'admin-row';
|
|
||||||
const l3 = document.createElement('label'); l3.className = 'admin-row-label'; l3.textContent = 'Learning image model'; row3.append(l3, learningSelect);
|
|
||||||
const row4 = document.createElement('div'); row4.className = 'admin-row';
|
|
||||||
const l4 = document.createElement('label'); l4.className = 'admin-row-label'; l4.textContent = 'Learning image input budget'; row4.append(l4, learningBudget);
|
|
||||||
form.append(heading, row1, row2, row3, row4, save, status);
|
|
||||||
form.onsubmit = async e => {
|
form.onsubmit = async e => {
|
||||||
e.preventDefault(); if (save.disabled) return; save.disabled = true;
|
e.preventDefault();
|
||||||
|
if (save.disabled) return;
|
||||||
|
save.disabled = true;
|
||||||
try {
|
try {
|
||||||
await imageJson('/api/admin/image-settings/clinical_assistant', { method: 'PUT', body: JSON.stringify({ model: clinicalSelect.value, budget: Number(clinicalBudget.value) }) });
|
for (const workflow of WORKFLOWS) {
|
||||||
await imageJson('/api/admin/image-settings/learning_hub', { method: 'PUT', body: JSON.stringify({ model: learningSelect.value, budget: Number(learningBudget.value) }) });
|
const control = controls[workflow.key];
|
||||||
|
const body = {
|
||||||
|
budget: Number(control.budget.value),
|
||||||
|
fallbacks: control.fallbacks.map(s => s.value).filter(Boolean)
|
||||||
|
};
|
||||||
|
// Only the workflows that own a primary send one; My Resources would
|
||||||
|
// be overwriting a per-request choice with a form field.
|
||||||
|
if (workflow.key !== 'my_resources') body.model = control.primary.value;
|
||||||
|
await imageJson('/api/admin/image-settings/' + workflow.key,
|
||||||
|
{ method: 'PUT', body: JSON.stringify(body) });
|
||||||
|
}
|
||||||
status.textContent = 'Saved. New jobs use these settings; existing jobs are unchanged.';
|
status.textContent = 'Saved. New jobs use these settings; existing jobs are unchanged.';
|
||||||
} catch (error) { status.textContent = error.message + ' Nothing was saved.'; } finally { save.disabled = false; }
|
} catch (error) {
|
||||||
|
status.textContent = error.message + ' Nothing was saved.';
|
||||||
|
} finally { save.disabled = false; }
|
||||||
};
|
};
|
||||||
root.append(form);
|
root.append(form);
|
||||||
loaded = true;
|
loaded = true;
|
||||||
} catch (_) { /* Next tab entry retries; no drafts are touched. */ }
|
} catch (_) { /* Next tab entry retries; no drafts are touched. */ }
|
||||||
finally { loading = false; }
|
finally { loading = false; }
|
||||||
}
|
}
|
||||||
function makeSelect(knownIds, models) {
|
|
||||||
const select = document.createElement('select'); select.className = 'admin-control';
|
function row(labelText, control) {
|
||||||
const ids = new Set([...(Array.isArray(knownIds) ? knownIds : []), ...((Array.isArray(models) ? models : []).map(m => m && m.id).filter(Boolean))]);
|
const wrap = document.createElement('div');
|
||||||
[...ids].sort().forEach(id => { const o = document.createElement('option'); o.value = id; o.textContent = id; select.appendChild(o); });
|
wrap.className = 'admin-row';
|
||||||
if (knownIds[0]) select.value = knownIds[0];
|
const label = document.createElement('label');
|
||||||
|
label.className = 'admin-row-label';
|
||||||
|
label.textContent = labelText;
|
||||||
|
wrap.append(label, control);
|
||||||
|
return wrap;
|
||||||
|
}
|
||||||
|
|
||||||
|
function makeSelect(knownIds, models, optional) {
|
||||||
|
const select = document.createElement('select');
|
||||||
|
select.className = 'admin-control';
|
||||||
|
if (optional) {
|
||||||
|
const none = document.createElement('option');
|
||||||
|
none.value = ''; none.textContent = 'None';
|
||||||
|
select.appendChild(none);
|
||||||
|
}
|
||||||
|
const ids = new Set([...(Array.isArray(knownIds) ? knownIds : []).filter(Boolean),
|
||||||
|
...((Array.isArray(models) ? models : []).map(m => m && m.id).filter(Boolean))]);
|
||||||
|
[...ids].sort().forEach(id => {
|
||||||
|
const option = document.createElement('option');
|
||||||
|
option.value = id; option.textContent = id;
|
||||||
|
select.appendChild(option);
|
||||||
|
});
|
||||||
|
select.value = knownIds[0] || '';
|
||||||
return select;
|
return select;
|
||||||
}
|
}
|
||||||
|
|
||||||
function makeBudget(value) {
|
function makeBudget(value) {
|
||||||
const input = document.createElement('input'); input.className = 'admin-control';
|
const input = document.createElement('input');
|
||||||
input.type = 'number'; input.min = '1000'; input.max = '32000'; input.required = true; input.value = value;
|
input.className = 'admin-control';
|
||||||
|
input.type = 'number'; input.min = '1000'; input.max = '32000'; input.required = true;
|
||||||
|
input.value = value;
|
||||||
return input;
|
return input;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function initImageSettings() {
|
export function initImageSettings() {
|
||||||
document.addEventListener('tabChanged', e => { if (e.detail?.tab === 'admin') load(); });
|
document.addEventListener('tabChanged', e => { if (e.detail?.tab === 'admin') load(); });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -331,53 +331,71 @@
|
||||||
// Built as elements, never innerHTML: the caption is a model-written prompt.
|
// Built as elements, never innerHTML: the caption is a model-written prompt.
|
||||||
function imageTile(image, m) {
|
function imageTile(image, m) {
|
||||||
var tile = document.createElement('figure');
|
var tile = document.createElement('figure');
|
||||||
tile.style.cssText = 'margin:0;border:1px solid var(--g200);border-radius:8px;overflow:hidden;' +
|
tile.className = 'img-tile';
|
||||||
'display:flex;flex-direction:column;background:var(--white);';
|
|
||||||
|
// The picture is the control: clicking it opens the full size. A button
|
||||||
|
// rather than a div, so it is reachable and announced without inventing
|
||||||
|
// keyboard handling.
|
||||||
|
var frame = document.createElement('button');
|
||||||
|
frame.type = 'button';
|
||||||
|
frame.className = 'img-tile-frame loading';
|
||||||
|
frame.title = 'Open full size';
|
||||||
|
|
||||||
var frame = document.createElement('div');
|
|
||||||
frame.style.cssText = 'aspect-ratio:4/3;background:var(--g100);display:flex;align-items:center;justify-content:center;overflow:hidden;';
|
|
||||||
var img = document.createElement('img');
|
var img = document.createElement('img');
|
||||||
img.alt = image.prompt ? 'Generated illustration: ' + image.prompt.slice(0, 80) : 'Generated illustration';
|
img.alt = image.prompt ? 'Generated illustration: ' + image.prompt.slice(0, 80) : 'Generated illustration';
|
||||||
img.setAttribute('data-image-thumb', '256');
|
img.setAttribute('data-image-thumb', '256');
|
||||||
img.style.cssText = 'width:100%;height:100%;object-fit:cover;display:block;';
|
img.loading = 'lazy';
|
||||||
frame.appendChild(img);
|
frame.appendChild(img);
|
||||||
m.hydrateImage(img, image.imageUrl).catch(function () {
|
frame.addEventListener('click', function () { openImage(image, m); });
|
||||||
img.remove();
|
|
||||||
frame.textContent = 'Unavailable';
|
|
||||||
frame.style.fontSize = '11px';
|
|
||||||
frame.style.color = 'var(--g500)';
|
|
||||||
});
|
|
||||||
|
|
||||||
var caption = document.createElement('figcaption');
|
var badge = document.createElement('span');
|
||||||
caption.style.cssText = 'padding:8px;display:flex;flex-direction:column;gap:6px;min-width:0;';
|
badge.className = 'img-tile-badge';
|
||||||
|
badge.textContent = image.source;
|
||||||
|
frame.appendChild(badge);
|
||||||
|
|
||||||
var text = document.createElement('div');
|
m.hydrateImage(img, image.imageUrl)
|
||||||
text.style.cssText = 'font-size:11px;color:var(--g600);line-height:1.35;max-height:3.4em;overflow:hidden;';
|
.then(function () { frame.classList.remove('loading'); })
|
||||||
text.textContent = image.prompt || 'No description recorded';
|
.catch(function () {
|
||||||
|
frame.classList.remove('loading');
|
||||||
|
img.remove();
|
||||||
|
var gone = document.createElement('span');
|
||||||
|
gone.style.cssText = 'font-size:11px;color:var(--g500);';
|
||||||
|
gone.textContent = 'Unavailable';
|
||||||
|
frame.appendChild(gone);
|
||||||
|
});
|
||||||
|
|
||||||
|
var body = document.createElement('div');
|
||||||
|
body.className = 'img-tile-body';
|
||||||
|
|
||||||
|
var prompt = document.createElement('figcaption');
|
||||||
|
prompt.className = 'img-tile-prompt';
|
||||||
|
prompt.textContent = image.prompt || 'No description recorded';
|
||||||
|
prompt.title = image.prompt || '';
|
||||||
|
|
||||||
var meta = document.createElement('div');
|
var meta = document.createElement('div');
|
||||||
meta.style.cssText = 'font-size:10px;color:var(--g500);display:flex;gap:6px;flex-wrap:wrap;';
|
meta.className = 'img-tile-meta';
|
||||||
meta.textContent = image.source + ' · ' + new Date(image.createdAt).toLocaleDateString();
|
meta.textContent = new Date(image.createdAt).toLocaleDateString(undefined,
|
||||||
|
{ year: 'numeric', month: 'short', day: 'numeric' }) +
|
||||||
|
(image.bytes ? ' · ' + Math.max(1, Math.round(image.bytes / 1024)) + ' kB' : '');
|
||||||
|
|
||||||
var actions = document.createElement('div');
|
var actions = document.createElement('div');
|
||||||
actions.style.cssText = 'display:flex;gap:6px;';
|
actions.className = 'img-tile-actions';
|
||||||
var open = document.createElement('button');
|
|
||||||
open.type = 'button';
|
var download = document.createElement('button');
|
||||||
open.className = 'btn-sm btn-ghost';
|
download.type = 'button';
|
||||||
open.style.cssText = 'font-size:11px;padding:3px 8px;';
|
download.className = 'img-tile-action';
|
||||||
open.textContent = 'Open';
|
download.textContent = 'Download';
|
||||||
open.addEventListener('click', function () { openImage(image, m); });
|
download.addEventListener('click', function () { saveImage(image, m, download); });
|
||||||
|
|
||||||
var remove = document.createElement('button');
|
var remove = document.createElement('button');
|
||||||
remove.type = 'button';
|
remove.type = 'button';
|
||||||
remove.className = 'btn-sm btn-ghost';
|
remove.className = 'img-tile-action danger';
|
||||||
remove.style.cssText = 'font-size:11px;padding:3px 8px;color:var(--red);';
|
|
||||||
remove.textContent = 'Delete';
|
remove.textContent = 'Delete';
|
||||||
remove.addEventListener('click', function () { deleteImage(image, tile, remove); });
|
remove.addEventListener('click', function () { deleteImage(image, tile, remove); });
|
||||||
|
|
||||||
actions.append(open, remove);
|
actions.append(download, remove);
|
||||||
caption.append(text, meta, actions);
|
body.append(prompt, meta, actions);
|
||||||
tile.append(frame, caption);
|
tile.append(frame, body);
|
||||||
return tile;
|
return tile;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -386,26 +404,83 @@
|
||||||
// no-store on purpose.
|
// no-store on purpose.
|
||||||
function openImage(image, m) {
|
function openImage(image, m) {
|
||||||
var overlay = document.createElement('div');
|
var overlay = document.createElement('div');
|
||||||
overlay.style.cssText = 'position:fixed;inset:0;background:rgba(0,0,0,.72);z-index:9999;' +
|
overlay.className = 'img-lightbox';
|
||||||
'display:flex;align-items:center;justify-content:center;padding:24px;';
|
overlay.setAttribute('role', 'dialog');
|
||||||
|
overlay.setAttribute('aria-modal', 'true');
|
||||||
|
overlay.setAttribute('aria-label', 'Generated illustration');
|
||||||
|
|
||||||
var full = document.createElement('img');
|
var full = document.createElement('img');
|
||||||
full.alt = image.prompt || 'Generated illustration';
|
full.alt = image.prompt || 'Generated illustration';
|
||||||
full.style.cssText = 'max-width:100%;max-height:100%;object-fit:contain;border-radius:6px;';
|
|
||||||
overlay.appendChild(full);
|
var bar = document.createElement('div');
|
||||||
overlay.addEventListener('click', function () { overlay.remove(); });
|
bar.className = 'img-lightbox-bar';
|
||||||
document.addEventListener('keydown', function escape(e) {
|
var caption = document.createElement('p');
|
||||||
if (e.key !== 'Escape') return;
|
caption.className = 'img-lightbox-caption';
|
||||||
|
caption.textContent = image.prompt || 'No description recorded';
|
||||||
|
|
||||||
|
var close = document.createElement('button');
|
||||||
|
close.type = 'button';
|
||||||
|
close.className = 'img-tile-action';
|
||||||
|
close.textContent = 'Close';
|
||||||
|
|
||||||
|
var save = document.createElement('button');
|
||||||
|
save.type = 'button';
|
||||||
|
save.className = 'img-tile-action';
|
||||||
|
save.textContent = 'Download';
|
||||||
|
save.addEventListener('click', function () { saveImage(image, m, save); });
|
||||||
|
|
||||||
|
bar.append(save, close);
|
||||||
|
overlay.append(full, caption, bar);
|
||||||
|
|
||||||
|
// Escape and a click on the backdrop both close it; a click on the picture
|
||||||
|
// or the buttons does not.
|
||||||
|
function dismiss() {
|
||||||
overlay.remove();
|
overlay.remove();
|
||||||
document.removeEventListener('keydown', escape);
|
document.removeEventListener('keydown', onKey);
|
||||||
});
|
if (lastFocus && lastFocus.focus) lastFocus.focus();
|
||||||
|
}
|
||||||
|
function onKey(e) { if (e.key === 'Escape') dismiss(); }
|
||||||
|
var lastFocus = document.activeElement;
|
||||||
|
overlay.addEventListener('click', function (e) { if (e.target === overlay) dismiss(); });
|
||||||
|
close.addEventListener('click', dismiss);
|
||||||
|
document.addEventListener('keydown', onKey);
|
||||||
|
|
||||||
document.body.appendChild(overlay);
|
document.body.appendChild(overlay);
|
||||||
|
close.focus();
|
||||||
m.hydrateImage(full, image.imageUrl).catch(function () {
|
m.hydrateImage(full, image.imageUrl).catch(function () {
|
||||||
full.remove();
|
full.remove();
|
||||||
overlay.textContent = 'That image could not be loaded.';
|
caption.textContent = 'That image could not be loaded.';
|
||||||
overlay.style.color = 'var(--white)';
|
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Fetched, then saved from memory. A plain <a download href> would work in a
|
||||||
|
// browser carrying the session cookie and fail on a mobile client, whose
|
||||||
|
// session is a bearer token an <a> cannot send — and these assets are served
|
||||||
|
// no-store precisely so they are not sitting in a cache to be linked to.
|
||||||
|
function saveImage(image, m, button) {
|
||||||
|
var label = button.textContent;
|
||||||
|
button.disabled = true;
|
||||||
|
button.textContent = 'Saving…';
|
||||||
|
m.privateImageBlob(image.imageUrl)
|
||||||
|
.then(function (blob) {
|
||||||
|
var url = URL.createObjectURL(blob);
|
||||||
|
var link = document.createElement('a');
|
||||||
|
link.href = url;
|
||||||
|
link.download = 'illustration-' + image.id.slice(0, 8) + '.' +
|
||||||
|
({ 'image/png': 'png', 'image/jpeg': 'jpg', 'image/webp': 'webp' })[image.mime] || 'png';
|
||||||
|
document.body.appendChild(link);
|
||||||
|
link.click();
|
||||||
|
link.remove();
|
||||||
|
// Revoked on the next frame: revoking immediately races the download in
|
||||||
|
// some browsers.
|
||||||
|
setTimeout(function () { URL.revokeObjectURL(url); }, 1000);
|
||||||
|
})
|
||||||
|
.catch(function () {
|
||||||
|
if (typeof showToast === 'function') showToast('That image could not be downloaded.', 'error');
|
||||||
|
})
|
||||||
|
.finally(function () { button.disabled = false; button.textContent = label; });
|
||||||
|
}
|
||||||
|
|
||||||
function deleteImage(image, tile, button) {
|
function deleteImage(image, tile, button) {
|
||||||
// Asked, because it removes the picture from every deck that used it. The
|
// Asked, because it removes the picture from every deck that used it. The
|
||||||
// deck keeps working; the figure is simply no longer there.
|
// deck keeps working; the figure is simply no longer there.
|
||||||
|
|
|
||||||
|
|
@ -147,26 +147,66 @@ learningRouter.post('/jobs', async (req, res) => {
|
||||||
learningRouter.get('/jobs/:id', async (req, res) => {
|
learningRouter.get('/jobs/:id', async (req, res) => {
|
||||||
try { res.json(await images.service().get(req.params.id, req.user.id, 'learning_hub')); } catch (e) { fail(res, e); }
|
try { res.json(await images.service().get(req.params.id, req.user.id, 'learning_hub')); } catch (e) { fail(res, e); }
|
||||||
});
|
});
|
||||||
|
const IMAGE_WORKFLOWS = ['clinical_assistant', 'learning_hub', 'my_resources'];
|
||||||
|
|
||||||
|
// The old single key, still read so an existing deployment keeps its fallback
|
||||||
|
// without anyone re-entering it. Writing through the new key supersedes it.
|
||||||
|
async function fallbackModels(workflow) {
|
||||||
|
const list = String(await db.getSetting(workflow + '.fallback_image_models') || '').trim();
|
||||||
|
if (list) return list.split(',').map(s => s.trim()).filter(Boolean);
|
||||||
|
const legacy = String(await db.getSetting(workflow + '.fallback_image_model') || '').trim();
|
||||||
|
return legacy ? [legacy] : [];
|
||||||
|
}
|
||||||
|
|
||||||
router.get('/admin/image-settings', adminMiddleware, async (req, res) => {
|
router.get('/admin/image-settings', adminMiddleware, async (req, res) => {
|
||||||
try {
|
try {
|
||||||
const workflows = {};
|
const workflows = {};
|
||||||
for (const workflow of ['clinical_assistant', 'learning_hub']) workflows[workflow] = {
|
for (const workflow of IMAGE_WORKFLOWS) workflows[workflow] = {
|
||||||
model: await db.getSetting(workflow + '.image_model') || '',
|
model: await db.getSetting(workflow + '.image_model') || '',
|
||||||
|
fallbacks: await fallbackModels(workflow),
|
||||||
budget: images.budgetLimit(await db.getSetting(workflow + '.image_budget')), unit: 'UTF-16 code units'
|
budget: images.budgetLimit(await db.getSetting(workflow + '.image_budget')), unit: 'UTF-16 code units'
|
||||||
};
|
};
|
||||||
res.json({ success: true, workflows });
|
res.json({ success: true, workflows, maxModels: images.MAX_IMAGE_MODELS });
|
||||||
} catch (e) { fail(res, e); }
|
} catch (e) { fail(res, e); }
|
||||||
});
|
});
|
||||||
|
const MODEL_ID = /^[a-zA-Z0-9_.:/-]{1,200}$/;
|
||||||
|
|
||||||
router.put('/admin/image-settings/:workflow', adminMiddleware, async (req, res) => {
|
router.put('/admin/image-settings/:workflow', adminMiddleware, async (req, res) => {
|
||||||
try {
|
try {
|
||||||
const workflow = req.params.workflow;
|
const workflow = req.params.workflow;
|
||||||
if (!['clinical_assistant', 'learning_hub'].includes(workflow)) throw images.failure(404, 'Workflow not found');
|
if (!IMAGE_WORKFLOWS.includes(workflow)) throw images.failure(404, 'Workflow not found');
|
||||||
const budget = images.budgetLimit(req.body.budget);
|
const budget = images.budgetLimit(req.body.budget);
|
||||||
if (workflow === 'learning_hub' && (typeof req.body.model !== 'string' || !/^[a-zA-Z0-9_.:/-]{1,200}$/.test(req.body.model))) throw images.failure(400, 'Enter an image model ID enabled at the configured image gateway');
|
if (workflow === 'learning_hub' && (typeof req.body.model !== 'string' || !MODEL_ID.test(req.body.model))) throw images.failure(400, 'Enter an image model ID enabled at the configured image gateway');
|
||||||
|
|
||||||
|
// Fallbacks are optional, and only sent when the caller means to change
|
||||||
|
// them: an absent field leaves the saved chain alone rather than clearing it.
|
||||||
|
let fallbacks = null;
|
||||||
|
if (req.body.fallbacks !== undefined) {
|
||||||
|
const list = Array.isArray(req.body.fallbacks) ? req.body.fallbacks
|
||||||
|
: String(req.body.fallbacks || '').split(',');
|
||||||
|
fallbacks = list.map(id => String(id).trim()).filter(Boolean);
|
||||||
|
for (const id of fallbacks) {
|
||||||
|
if (!MODEL_ID.test(id)) throw images.failure(400, 'Not a model ID: ' + id.slice(0, 60));
|
||||||
|
}
|
||||||
|
// The primary counts toward the cap, and naming it again would pay twice
|
||||||
|
// for the same refusal.
|
||||||
|
const primary = workflow === 'learning_hub' ? req.body.model
|
||||||
|
: String(await db.getSetting(workflow + '.image_model') || '');
|
||||||
|
fallbacks = fallbacks.filter((id, i) => id !== primary && fallbacks.indexOf(id) === i);
|
||||||
|
if (fallbacks.length > images.MAX_IMAGE_MODELS - 1) {
|
||||||
|
throw images.failure(400, 'At most ' + (images.MAX_IMAGE_MODELS - 1) +
|
||||||
|
' fallback models; each one is a paid retry');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
const client = await db.pool.connect();
|
const client = await db.pool.connect();
|
||||||
try {
|
try {
|
||||||
await client.query('BEGIN');
|
await client.query('BEGIN');
|
||||||
for (const [key, value] of Object.entries(workflow === 'learning_hub' ? { image_model: req.body.model, image_budget: budget } : { image_budget: budget })) {
|
const changes = workflow === 'learning_hub'
|
||||||
|
? { image_model: req.body.model, image_budget: budget }
|
||||||
|
: { image_budget: budget };
|
||||||
|
if (fallbacks !== null) changes.fallback_image_models = fallbacks.join(',');
|
||||||
|
for (const [key, value] of Object.entries(changes)) {
|
||||||
await client.query('INSERT INTO app_settings(key,value) VALUES($1,$2) ON CONFLICT(key) DO UPDATE SET value=$2,updated_at=NOW()', [workflow + '.' + key, String(value)]);
|
await client.query('INSERT INTO app_settings(key,value) VALUES($1,$2) ON CONFLICT(key) DO UPDATE SET value=$2,updated_at=NOW()', [workflow + '.' + key, String(value)]);
|
||||||
}
|
}
|
||||||
await client.query('COMMIT');
|
await client.query('COMMIT');
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,12 @@
|
||||||
const crypto = require('crypto');
|
const crypto = require('crypto');
|
||||||
const { DEFAULT_IMAGE_BEHAVIOR, imagePromptForCanvas } = require('./clinicalPrompts');
|
const { DEFAULT_IMAGE_BEHAVIOR, imagePromptForCanvas } = require('./clinicalPrompts');
|
||||||
const storageUtil = require('./generatedImageStorage');
|
const storageUtil = require('./generatedImageStorage');
|
||||||
|
// fileLog rather than logger: logger requires the database at module load, and
|
||||||
|
// this module is exercised by tests that never open one.
|
||||||
|
const fileLog = require('./fileLog');
|
||||||
|
// Primary plus two. More is a bill, not a safety net: each hop is a paid request
|
||||||
|
// and a chain long enough to be worth capping is long enough to surprise someone.
|
||||||
|
const MAX_IMAGE_MODELS = 3;
|
||||||
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/;
|
||||||
const failure = (statusCode, message) => Object.assign(new Error(message), { statusCode });
|
const failure = (statusCode, message) => Object.assign(new Error(message), { statusCode });
|
||||||
const workflows = ['clinical_assistant', 'learning_hub', 'my_resources'];
|
const workflows = ['clinical_assistant', 'learning_hub', 'my_resources'];
|
||||||
|
|
@ -30,14 +36,57 @@ function isDefiniteImageError(err) {
|
||||||
function isAbortImageError(err, signal) {
|
function isAbortImageError(err, signal) {
|
||||||
return !!(signal && signal.aborted) || !!(err && (err.name === 'AbortError' || err.code === 'ERR_CANCELED'));
|
return !!(signal && signal.aborted) || !!(err && (err.name === 'AbortError' || err.code === 'ERR_CANCELED'));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// A model that declined on content grounds. Providers disagree about the status
|
||||||
|
// for this — some send 400, some 422 — and none of them send a machine-readable
|
||||||
|
// reason, so the message is what is left to read. Worth separating because it is
|
||||||
|
// the one 4xx where a different model genuinely may succeed: policy is a vendor
|
||||||
|
// decision, not a fact about the request.
|
||||||
|
var REFUSAL = /(safety|safety_?system|content[_ ]?polic|moderation|blocked|flagged|violat|not allowed|cannot generate|refus|prohibit|inappropriate|sensitive)/i;
|
||||||
|
function isContentRefusal(err) {
|
||||||
|
var status = err && err.response ? Number(err.response.status) : NaN;
|
||||||
|
if (!Number.isFinite(status) || status < 400 || status >= 500) return false;
|
||||||
|
var body = err && err.response && err.response.data;
|
||||||
|
var text = [err && err.message,
|
||||||
|
typeof body === 'string' ? body : body && JSON.stringify(body)].filter(Boolean).join(' ');
|
||||||
|
return REFUSAL.test(text);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Why a failure is or is not worth handing to the next model. One function, so
|
||||||
|
// the reasoning is in one place and is reportable rather than implied by a
|
||||||
|
// boolean.
|
||||||
|
//
|
||||||
|
// retry the next model has a real chance: transient faults, a model the
|
||||||
|
// gateway does not have, and a refusal — another vendor's policy is
|
||||||
|
// not this one's
|
||||||
|
// stop nothing downstream can help: the caller went away, we are shutting
|
||||||
|
// down, the credentials are wrong (every model shares them), or the
|
||||||
|
// request itself is malformed or too large
|
||||||
|
//
|
||||||
|
// 408 is Request Timeout and 429 is Too Many Requests; both are the server
|
||||||
|
// saying "not now", not "not ever", which is why they never counted as definite.
|
||||||
|
function classifyImageFailure(err, signal, stopping) {
|
||||||
|
if (stopping) return { retry: false, reason: 'shutting down' };
|
||||||
|
if (isAbortImageError(err, signal)) return { retry: false, reason: 'the request was cancelled' };
|
||||||
|
|
||||||
|
var status = err && err.response ? Number(err.response.status) : NaN;
|
||||||
|
if (!Number.isFinite(status)) return { retry: true, reason: 'network or timeout' };
|
||||||
|
if (status === 401 || status === 403) {
|
||||||
|
// Same gateway, same credentials: the next model fails identically.
|
||||||
|
return { retry: false, reason: 'the gateway rejected our credentials' };
|
||||||
|
}
|
||||||
|
if (status === 413) return { retry: false, reason: 'the request is too large' };
|
||||||
|
if (status === 404) return { retry: true, reason: 'the gateway does not have that model' };
|
||||||
|
if (status === 408 || status === 429) return { retry: true, reason: 'the provider was busy' };
|
||||||
|
if (status >= 500) return { retry: true, reason: 'the provider failed' };
|
||||||
|
if (isContentRefusal(err)) return { retry: true, reason: 'the model declined the prompt' };
|
||||||
|
return { retry: false, reason: 'the provider rejected the request' };
|
||||||
|
}
|
||||||
|
|
||||||
function shouldRetryImageFallback(state) {
|
function shouldRetryImageFallback(state) {
|
||||||
state = state || {};
|
state = state || {};
|
||||||
if (state.workflow !== 'clinical_assistant') return false;
|
|
||||||
if (!state.fallback || state.fallback === state.jobModel) return false;
|
if (!state.fallback || state.fallback === state.jobModel) return false;
|
||||||
if (state.stopping) return false;
|
return classifyImageFailure(state.error, state.signal, state.stopping).retry;
|
||||||
if (isAbortImageError(state.error, state.signal)) return false;
|
|
||||||
if (isDefiniteImageError(state.error)) return false;
|
|
||||||
return true;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Previews are generated once and stored beside the original, so a 56px tile
|
// Previews are generated once and stored beside the original, so a 56px tile
|
||||||
|
|
@ -156,24 +205,67 @@ function createImageService({ db, storage, generate = provider, encryption = req
|
||||||
await client.query('COMMIT'); return result.rows[0];
|
await client.query('COMMIT'); return result.rows[0];
|
||||||
} catch (e) { await client.query('ROLLBACK').catch(() => {}); throw e; } finally { client.release(); }
|
} catch (e) { await client.query('ROLLBACK').catch(() => {}); throw e; } finally { client.release(); }
|
||||||
}
|
}
|
||||||
async function generateWithFallback(job, prompt, signal) {
|
// The models to try, in order, for this workflow: the configured one first,
|
||||||
|
// then its fallbacks. Every workflow has this now — it used to be the Clinical
|
||||||
|
// Assistant alone, so a My Resources figure that failed simply had no picture,
|
||||||
|
// which is the case where a missing figure is most visible.
|
||||||
|
async function modelChain(workflow, primary) {
|
||||||
|
var chain = [primary];
|
||||||
try {
|
try {
|
||||||
return await generate(job, prompt, signal);
|
// The plural key is the current one; the old singular is still read so a
|
||||||
} catch (err) {
|
// deployment that configured a fallback before this keeps it without
|
||||||
var fallback = '';
|
// anyone re-entering it.
|
||||||
if (job.workflow === 'clinical_assistant' && !stopping && !isAbortImageError(err, signal) && !isDefiniteImageError(err)) {
|
const row = await db.query(
|
||||||
try {
|
'SELECT key, value FROM app_settings WHERE key = ANY($1::text[])',
|
||||||
const row = await db.query('SELECT value FROM app_settings WHERE key = $1', ['clinical_assistant.fallback_image_model']);
|
[[workflow + '.fallback_image_models', workflow + '.fallback_image_model']]);
|
||||||
fallback = String(row.rows[0] && row.rows[0].value || '');
|
const byKey = {};
|
||||||
} catch (_) { fallback = ''; }
|
row.rows.forEach(function (r) { byKey[r.key] = String(r.value || ''); });
|
||||||
|
var configured = (byKey[workflow + '.fallback_image_models'] || '').trim()
|
||||||
|
|| (byKey[workflow + '.fallback_image_model'] || '').trim();
|
||||||
|
configured.split(',')
|
||||||
|
.map(function (id) { return id.trim(); })
|
||||||
|
.filter(Boolean)
|
||||||
|
.forEach(function (id) { if (chain.indexOf(id) === -1) chain.push(id); });
|
||||||
|
} catch (_) { /* no fallbacks configured is the normal case */ }
|
||||||
|
return chain.slice(0, MAX_IMAGE_MODELS);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Walk the chain until one produces a picture. Each hop re-leases the job, so
|
||||||
|
// a chain cannot outlive its claim and let a second worker start the same paid
|
||||||
|
// work; if the lease has gone, the attempt stops there rather than paying
|
||||||
|
// again. The error that ends the chain is the one the caller sees, and every
|
||||||
|
// hop is recorded with the reason it moved on.
|
||||||
|
async function generateWithFallback(job, prompt, signal) {
|
||||||
|
const chain = await modelChain(job.workflow, job.model);
|
||||||
|
let lastError = null;
|
||||||
|
|
||||||
|
for (let attempt = 0; attempt < chain.length; attempt++) {
|
||||||
|
const model = chain[attempt];
|
||||||
|
if (attempt > 0) {
|
||||||
|
const lease = await db.query(
|
||||||
|
"UPDATE generated_image_jobs SET lease_until=NOW()+interval '3 minutes' WHERE id=$1 AND lease_token=$2 AND stage='generating' RETURNING id",
|
||||||
|
[job.id, job.lease_token]);
|
||||||
|
if (!lease.rows.length) throw lastError;
|
||||||
|
// The row records which model is actually being paid for, so a picture
|
||||||
|
// made by the third model is not attributed to the first.
|
||||||
|
await db.query('UPDATE generated_image_jobs SET model=$3,updated_at=NOW() WHERE id=$1 AND lease_token=$2',
|
||||||
|
[job.id, job.lease_token, model]).catch(function () {});
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
return await generate(Object.assign({}, job, { model: model }), prompt, signal);
|
||||||
|
} catch (err) {
|
||||||
|
lastError = err;
|
||||||
|
const verdict = classifyImageFailure(err, signal, stopping);
|
||||||
|
const next = chain[attempt + 1];
|
||||||
|
fileLog.write(next && verdict.retry ? 'warn' : 'error',
|
||||||
|
'[generated-images] ' + model + ' failed: ' + verdict.reason +
|
||||||
|
(next && verdict.retry ? '; trying ' + next : '; giving up'),
|
||||||
|
{ jobId: job.id, workflow: job.workflow, attempt: attempt + 1, of: chain.length,
|
||||||
|
status: err && err.response ? err.response.status : null });
|
||||||
|
if (!verdict.retry || !next) throw err;
|
||||||
}
|
}
|
||||||
if (!shouldRetryImageFallback({ workflow: job.workflow, fallback: fallback, jobModel: job.model, stopping: stopping, error: err, signal: signal })) throw err;
|
|
||||||
console.warn('[generated-images] image generation failed; retrying once with the fallback image model', { jobId: job.id, model: job.model, fallbackModel: fallback });
|
|
||||||
const lease = await db.query("UPDATE generated_image_jobs SET lease_until=NOW()+interval '3 minutes' WHERE id=$1 AND lease_token=$2 AND stage='generating' RETURNING id",
|
|
||||||
[job.id, job.lease_token]);
|
|
||||||
if (!lease.rows.length) throw err;
|
|
||||||
return await generate(Object.assign({}, job, { model: fallback }), prompt, signal);
|
|
||||||
}
|
}
|
||||||
|
throw lastError;
|
||||||
}
|
}
|
||||||
|
|
||||||
async function tick() {
|
async function tick() {
|
||||||
|
|
@ -303,4 +395,4 @@ function requestKey(body) {
|
||||||
}
|
}
|
||||||
return crypto.createHash('sha256').update(JSON.stringify(body)).digest('hex');
|
return crypto.createHash('sha256').update(JSON.stringify(body)).digest('hex');
|
||||||
}
|
}
|
||||||
module.exports = { createImageService, service, budgetLimit, THUMB_WIDTHS, thumbWidth, args, imageContext, IMAGE_OUTPUT_RULE, publicJob, requestKey, UUID, failure, isDefiniteImageError, isAbortImageError, shouldRetryImageFallback };
|
module.exports = { createImageService, service, budgetLimit, THUMB_WIDTHS, MAX_IMAGE_MODELS, classifyImageFailure, isContentRefusal, thumbWidth, args, imageContext, IMAGE_OUTPUT_RULE, publicJob, requestKey, UUID, failure, isDefiniteImageError, isAbortImageError, shouldRetryImageFallback };
|
||||||
|
|
|
||||||
|
|
@ -377,12 +377,15 @@ test('assistant config pending, rejected, HTTP failure and malformed payloads ne
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
test('single image-model dropdown keeps saved selection through discovery failures and saves both workflows', async t => {
|
test('image-model dropdowns keep saved selections through discovery failures and save every workflow', async t => {
|
||||||
const pending = [];
|
const pending = [];
|
||||||
const ui = await browser(t, 'admin', url => {
|
const ui = await browser(t, 'admin', url => {
|
||||||
if (url === '/api/admin/config') return json(assistantConfig());
|
if (url === '/api/admin/config') return json(assistantConfig());
|
||||||
if (url === '/api/models') throw Error('chat discovery offline');
|
if (url === '/api/models') throw Error('chat discovery offline');
|
||||||
if (url === '/api/admin/image-settings') return json({ success: true, workflows: { clinical_assistant: { model: 'saved-image', budget: 32000 }, learning_hub: { model: 'saved-image', budget: 32000 } } });
|
if (url === '/api/admin/image-settings') return json({ success: true, maxModels: 3, workflows: {
|
||||||
|
clinical_assistant: { model: 'saved-image', fallbacks: ['saved-backup'], budget: 32000 },
|
||||||
|
learning_hub: { model: 'saved-image', fallbacks: [], budget: 32000 },
|
||||||
|
my_resources: { model: '', fallbacks: [], budget: 32000 } } });
|
||||||
if (url.endsWith('/image-models/discover')) return new Promise((resolve, reject) => { pending.push({ resolve, reject }); });
|
if (url.endsWith('/image-models/discover')) return new Promise((resolve, reject) => { pending.push({ resolve, reject }); });
|
||||||
});
|
});
|
||||||
await tick(); await tick();
|
await tick(); await tick();
|
||||||
|
|
@ -391,12 +394,23 @@ test('single image-model dropdown keeps saved selection through discovery failur
|
||||||
const select = ui.document.querySelector('#workflow-image-settings select');
|
const select = ui.document.querySelector('#workflow-image-settings select');
|
||||||
assert.ok(select, 'image-model dropdown rendered');
|
assert.ok(select, 'image-model dropdown rendered');
|
||||||
assert.equal(select.value, 'saved-image', 'saved selection kept while discovery failed');
|
assert.equal(select.value, 'saved-image', 'saved selection kept while discovery failed');
|
||||||
ui.document.querySelector('#workflow-image-settings form button').click(); await tick();
|
// The saved fallback survives a discovery failure the same way the primary
|
||||||
|
// does: it is seeded into the list rather than only offered from discovery.
|
||||||
|
const clinical = ui.document.querySelectorAll('#workflow-image-settings fieldset')[0];
|
||||||
|
assert.equal(clinical.querySelectorAll('select')[1].value, 'saved-backup', 'saved fallback kept');
|
||||||
|
|
||||||
|
ui.document.querySelector('#workflow-image-settings form button[type="submit"]').click();
|
||||||
|
// One await per workflow, so the queue needs draining more than once.
|
||||||
|
for (let i = 0; i < 6; i++) await tick();
|
||||||
const puts = () => ui.calls.filter(c => c.options.method === 'PUT' && c.url.includes('/api/admin/image-settings/'));
|
const puts = () => ui.calls.filter(c => c.options.method === 'PUT' && c.url.includes('/api/admin/image-settings/'));
|
||||||
assert.equal(puts().length, 2);
|
assert.equal(puts().length, 3, 'every workflow is saved, My Resources included');
|
||||||
for (const workflow of ['clinical_assistant', 'learning_hub']) {
|
assert.deepEqual(puts().find(c => c.url.endsWith('clinical_assistant')).body,
|
||||||
assert.deepEqual(puts().find(c => c.url.endsWith(workflow)).body, { model: 'saved-image', budget: 32000 });
|
{ model: 'saved-image', budget: 32000, fallbacks: ['saved-backup'] });
|
||||||
}
|
assert.deepEqual(puts().find(c => c.url.endsWith('learning_hub')).body,
|
||||||
|
{ model: 'saved-image', budget: 32000, fallbacks: [] });
|
||||||
|
// My Resources chooses its model per request, so the form must not send one.
|
||||||
|
assert.deepEqual(puts().find(c => c.url.endsWith('my_resources')).body,
|
||||||
|
{ budget: 32000, fallbacks: [] });
|
||||||
assert.equal(select.value, 'saved-image', 'save never reverts the selection');
|
assert.equal(select.value, 'saved-image', 'save never reverts the selection');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
|
||||||
100
test/image-fallback-chain.test.js
Normal file
100
test/image-fallback-chain.test.js
Normal file
|
|
@ -0,0 +1,100 @@
|
||||||
|
// An image model that fails should hand the job to the next one — for every
|
||||||
|
// feature, not only the Clinical Assistant, and for the failures where another
|
||||||
|
// model genuinely has a chance.
|
||||||
|
const test = require('node:test');
|
||||||
|
const assert = require('node:assert/strict');
|
||||||
|
const { classifyImageFailure, isContentRefusal, MAX_IMAGE_MODELS,
|
||||||
|
shouldRetryImageFallback } = require('../src/utils/generatedImages');
|
||||||
|
|
||||||
|
const httpError = (status, body) => Object.assign(new Error(typeof body === 'string' ? body : 'request failed'),
|
||||||
|
{ response: { status, data: body } });
|
||||||
|
|
||||||
|
test('the provider saying "not now" is always worth another model', () => {
|
||||||
|
// 408 is Request Timeout, 429 is Too Many Requests. Both are a server saying
|
||||||
|
// not now, never not ever.
|
||||||
|
for (const status of [408, 429, 500, 502, 503, 504]) {
|
||||||
|
assert.equal(classifyImageFailure(httpError(status), null, false).retry, true, 'status ' + status);
|
||||||
|
}
|
||||||
|
// No response at all: a socket hang-up or a DNS failure.
|
||||||
|
assert.equal(classifyImageFailure(new Error('socket hang up'), null, false).retry, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('bad credentials stop the chain, because every model shares them', () => {
|
||||||
|
for (const status of [401, 403]) {
|
||||||
|
const verdict = classifyImageFailure(httpError(status), null, false);
|
||||||
|
assert.equal(verdict.retry, false, 'status ' + status);
|
||||||
|
assert.match(verdict.reason, /credentials/);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a model the gateway does not have moves on rather than failing the job', () => {
|
||||||
|
// This is a configuration mistake, and the next model is exactly the thing
|
||||||
|
// that rescues it.
|
||||||
|
const verdict = classifyImageFailure(httpError(404), null, false);
|
||||||
|
assert.equal(verdict.retry, true);
|
||||||
|
assert.match(verdict.reason, /does not have that model/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a content refusal tries the next model — policy is a vendor decision', () => {
|
||||||
|
const refusals = [
|
||||||
|
httpError(400, { error: { message: 'Your request was rejected by our safety system' } }),
|
||||||
|
httpError(400, 'content_policy_violation'),
|
||||||
|
httpError(422, { message: 'The prompt was flagged as sensitive' }),
|
||||||
|
httpError(400, { error: 'This request was blocked by moderation' })
|
||||||
|
];
|
||||||
|
for (const err of refusals) {
|
||||||
|
assert.equal(isContentRefusal(err), true, err.message);
|
||||||
|
assert.equal(classifyImageFailure(err, null, false).retry, true);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a malformed request is not retried, because it is malformed everywhere', () => {
|
||||||
|
const verdict = classifyImageFailure(httpError(400, { error: 'size must be one of 1024x1024' }), null, false);
|
||||||
|
assert.equal(verdict.retry, false);
|
||||||
|
assert.match(verdict.reason, /rejected the request/);
|
||||||
|
assert.equal(classifyImageFailure(httpError(413), null, false).retry, false, 'too large everywhere too');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a cancelled request and a shutdown never start more paid work', () => {
|
||||||
|
assert.equal(classifyImageFailure(new Error('x'), { aborted: true }, false).retry, false);
|
||||||
|
assert.equal(classifyImageFailure(Object.assign(new Error('x'), { name: 'AbortError' }), null, false).retry, false);
|
||||||
|
assert.equal(classifyImageFailure(httpError(503), null, true).retry, false, 'shutting down');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('every workflow gets fallbacks now, not the Clinical Assistant alone', () => {
|
||||||
|
// A My Resources figure that failed used to have no second chance at all,
|
||||||
|
// which is the case where a missing picture is most visible — a slide with a
|
||||||
|
// hole in it.
|
||||||
|
for (const workflow of ['clinical_assistant', 'my_resources', 'learning_hub']) {
|
||||||
|
assert.equal(shouldRetryImageFallback({
|
||||||
|
workflow, fallback: 'model-b', jobModel: 'model-a', error: httpError(503)
|
||||||
|
}), true, workflow);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
test('a fallback identical to the model that just failed is not a fallback', () => {
|
||||||
|
assert.equal(shouldRetryImageFallback({
|
||||||
|
workflow: 'my_resources', fallback: 'model-a', jobModel: 'model-a', error: httpError(503)
|
||||||
|
}), false);
|
||||||
|
assert.equal(shouldRetryImageFallback({
|
||||||
|
workflow: 'my_resources', fallback: '', jobModel: 'model-a', error: httpError(503)
|
||||||
|
}), false);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the chain is capped, because every hop is a paid request', () => {
|
||||||
|
assert.equal(MAX_IMAGE_MODELS, 3);
|
||||||
|
const lib = require('fs').readFileSync(require('path').join(__dirname, '..', 'src/utils/generatedImages.js'), 'utf8');
|
||||||
|
assert.match(lib, /chain\.slice\(0, MAX_IMAGE_MODELS\)/);
|
||||||
|
// Duplicates collapse: naming the primary again as a fallback would pay twice
|
||||||
|
// for the same refusal.
|
||||||
|
assert.match(lib, /if \(chain\.indexOf\(id\) === -1\) chain\.push\(id\)/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('each hop re-leases, and records which model is actually being paid for', () => {
|
||||||
|
const lib = require('fs').readFileSync(require('path').join(__dirname, '..', 'src/utils/generatedImages.js'), 'utf8');
|
||||||
|
const fn = lib.slice(lib.indexOf('async function generateWithFallback'));
|
||||||
|
assert.match(fn.slice(0, 1600), /lease_until=NOW\(\)\+interval '3 minutes'/,
|
||||||
|
'a chain must not outlive its claim and let a second worker repeat the work');
|
||||||
|
assert.match(fn.slice(0, 1600), /UPDATE generated_image_jobs SET model=\$3/,
|
||||||
|
'a picture made by the third model must not be attributed to the first');
|
||||||
|
});
|
||||||
|
|
@ -82,7 +82,7 @@ test('the grid asks for the stored preview, not the original', () => {
|
||||||
test('the caption is a model-written prompt and never reaches the page as HTML', () => {
|
test('the caption is a model-written prompt and never reaches the page as HTML', () => {
|
||||||
const ui = read('public/js/myResources.js');
|
const ui = read('public/js/myResources.js');
|
||||||
const tile = ui.slice(ui.indexOf('function imageTile'), ui.indexOf('function openImage'));
|
const tile = ui.slice(ui.indexOf('function imageTile'), ui.indexOf('function openImage'));
|
||||||
assert.match(tile, /text\.textContent = image\.prompt/);
|
assert.match(tile, /prompt\.textContent = image\.prompt/);
|
||||||
assert.doesNotMatch(tile, /innerHTML/);
|
assert.doesNotMatch(tile, /innerHTML/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|
@ -91,3 +91,53 @@ test('the images tab loads on first open, not on page load', () => {
|
||||||
const ui = read('public/js/myResources.js');
|
const ui = read('public/js/myResources.js');
|
||||||
assert.match(ui, /if \(!docs && !imagesLoaded\) loadImages\(true\)/);
|
assert.match(ui, /if \(!docs && !imagesLoaded\) loadImages\(true\)/);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
test('the library is in My Resources and never in the Clinical Assistant', () => {
|
||||||
|
// Assistant images appear here — that was the point — but the assistant page
|
||||||
|
// does not grow a gallery of its own.
|
||||||
|
const ui = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/js/myResources.js'), 'utf8');
|
||||||
|
assert.match(ui, /function loadImages/);
|
||||||
|
const assistantFiles = require('fs').readdirSync(require('path').join(__dirname, '..', 'public/js/assistant'));
|
||||||
|
for (const file of assistantFiles) {
|
||||||
|
const src = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/js/assistant', file), 'utf8');
|
||||||
|
assert.doesNotMatch(src, /\/api\/generated-images\?/, file + ' must not list the library');
|
||||||
|
}
|
||||||
|
const component = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/components/my-resources.html'), 'utf8');
|
||||||
|
assert.match(component, /id="mr-images-panel"/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('downloading goes through the authenticated blob, not a bare link', () => {
|
||||||
|
// A mobile client's session is a bearer token an <a> cannot send, and the
|
||||||
|
// asset is served no-store on purpose.
|
||||||
|
const ui = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/js/myResources.js'), 'utf8');
|
||||||
|
const save = ui.slice(ui.indexOf('function saveImage'), ui.indexOf('function deleteImage'));
|
||||||
|
assert.match(save, /m\.privateImageBlob\(image\.imageUrl\)/);
|
||||||
|
assert.match(save, /URL\.revokeObjectURL/, 'the object URL is released');
|
||||||
|
assert.doesNotMatch(save, /href = image\.downloadUrl/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the tile actions stay reachable without a pointer', () => {
|
||||||
|
// Hiding delete behind :hover puts it out of reach on touch and keyboard.
|
||||||
|
const css = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/css/styles.css'), 'utf8');
|
||||||
|
const block = css.slice(css.indexOf('.img-tile-actions'));
|
||||||
|
assert.match(block.slice(0, 400), /@media \(hover:hover\)/, 'only hidden where hover exists');
|
||||||
|
assert.match(css, /\.img-tile:focus-within \.img-tile-actions/);
|
||||||
|
assert.match(css, /prefers-reduced-motion/);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the picture itself is a button, so opening it needs no invented key handling', () => {
|
||||||
|
const ui = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/js/myResources.js'), 'utf8');
|
||||||
|
const tile = ui.slice(ui.indexOf('function imageTile'), ui.indexOf('function openImage'));
|
||||||
|
assert.match(tile, /frame\.type = 'button'/);
|
||||||
|
assert.match(tile, /loading = 'lazy'/, 'a grid of thumbnails should not all fetch at once');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('the lightbox closes on Escape and on the backdrop, and restores focus', () => {
|
||||||
|
const ui = require('fs').readFileSync(require('path').join(__dirname, '..', 'public/js/myResources.js'), 'utf8');
|
||||||
|
const box = ui.slice(ui.indexOf('function openImage'), ui.indexOf('function saveImage'));
|
||||||
|
assert.match(box, /aria-modal/);
|
||||||
|
assert.match(box, /e\.key === 'Escape'/);
|
||||||
|
assert.match(box, /if \(e\.target === overlay\) dismiss\(\)/, 'a click on the picture must not close it');
|
||||||
|
assert.match(box, /lastFocus\.focus\(\)/);
|
||||||
|
assert.match(box, /removeEventListener\('keydown', onKey\)/, 'no listener left behind');
|
||||||
|
});
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue