feat: optional web search, admin-enabled and off by default
Some checks failed
Forgejo Android APK / Root app tests (push) Successful in 47s
Forgejo Docker Build / Root app tests (push) Successful in 47s
Forgejo Android APK / Build signed APK (push) Successful in 2m9s
Forgejo Docker Build / Build Docker image (push) Successful in 11s
Forgejo Docker Build / Deploy to the host (push) Failing after 0s

The one feature here that sends text outside the building, so the defaults are
the careful ones: disabled unless an administrator turns it on, opt-in per
generation even then, and the option is hidden entirely rather than shown as
something a user can tick and be refused.

Only the search query leaves. Library excerpts, the generated resource and
anything about the user never do. Both screens say so plainly, because a topic
typed while drafting clinical material can carry clinical detail and the
provider keeps its own logs.

Four providers behind one shape, so swapping changes nothing downstream: Tavily,
Serper over Google, Brave, and SearXNG — the only one where the query does not
reach a commercial third party at all, which is why it is worth supporting even
though it needs somewhere to run.

The tool description says when NOT to search, because a model handed a search
tool will reach for it constantly: not for settled clinical knowledge, which is
what the indexed library is for, and one search per resource. That last one is
enforced in the route with toolChoice: 'none' on the continuation rather than
trusted to the model.

A failed search never fails a generation — same contract as corpus retrieval.
The resource is written without it and the response says what was searched for
and what came back, so a query that left the network is visible rather than
silent.

The API key is masked on read and preserved when the field is left blank, the
handling the OIDC client secret already gets, so changing provider cannot
silently wipe a working key.

Verified on the running instance: with nothing configured, webSearchAvailable is
false, and a request asking for it anyway is ignored rather than honoured.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
This commit is contained in:
Daniel 2026-09-11 15:47:06 +02:00
parent 7eca509b02
commit 571a013d29
9 changed files with 543 additions and 6 deletions

View file

@ -416,6 +416,68 @@
</div>
</div>
<!-- ── Web search ─────────────────────────────────────────────── -->
<div class="card">
<div class="card-header">
<h3><i class="fas fa-globe"></i> Web Search</h3>
<span style="font-size:12px;color:var(--g500);">Off by default</span>
</div>
<div style="padding:16px;display:flex;flex-direction:column;gap:14px;">
<p style="margin:0;padding:8px 10px;background:rgba(224,142,69,0.10);border:1px solid var(--orange, #e08e45);border-radius:6px;font-size:12px;color:var(--g700);">
<strong>This sends text outside the building.</strong> When a resource author turns
it on, the model may send a short search query to the provider below. Only the query
leaves &mdash; never library excerpts, never the generated content, never anything
about the user &mdash; but a query written while drafting clinical material can still
carry clinical detail, and the provider keeps its own logs. SearXNG is the only
option here that you host yourself.
</p>
<div class="admin-row">
<strong class="admin-row-label">Enabled</strong>
<div style="flex:1;min-width:0;">
<select id="ws-enabled" class="admin-control" style="max-width:200px;">
<option value="false">Disabled</option>
<option value="true">Enabled</option>
</select>
</div>
</div>
<div class="admin-row">
<label for="ws-provider" class="admin-row-label">Provider</label>
<div style="flex:1;min-width:0;">
<select id="ws-provider" class="admin-control" style="max-width:260px;">
<option value="tavily">Tavily &mdash; built for AI, clean extracts</option>
<option value="serper">Serper &mdash; Google results</option>
<option value="brave">Brave Search API</option>
<option value="searxng">SearXNG &mdash; self-hosted, no third party</option>
</select>
</div>
</div>
<div class="admin-row">
<label for="ws-api-key" class="admin-row-label">API key</label>
<div style="flex:1;display:flex;flex-direction:column;gap:4px;min-width:0;">
<input id="ws-api-key" type="password" class="admin-control" placeholder="Leave blank to keep the current key" autocomplete="off">
<p style="margin:0;font-size:12px;color:var(--g500);">Not needed for SearXNG. Shown masked once saved.</p>
</div>
</div>
<div class="admin-row">
<label for="ws-base-url" class="admin-row-label">SearXNG URL</label>
<div style="flex:1;display:flex;flex-direction:column;gap:4px;min-width:0;">
<input id="ws-base-url" type="text" class="admin-control" placeholder="https://searxng.example.com">
<p style="margin:0;font-size:12px;color:var(--g500);">Only used when the provider is SearXNG. Its <code>/search?format=json</code> endpoint must be enabled.</p>
</div>
</div>
<div style="border-top:1px solid var(--g100);padding-top:12px;display:flex;align-items:center;gap:8px;flex-wrap:wrap;">
<button id="btn-save-websearch" class="btn-sm btn-primary" type="button"><i class="fas fa-floppy-disk"></i> Save web search settings</button>
<button id="btn-test-websearch" class="btn-sm btn-ghost" type="button"><i class="fas fa-vial"></i> Test</button>
<span id="ws-status" role="status" style="font-size:12px;color:var(--g500);"></span>
</div>
</div>
</div>
<!-- ── Citation quality ───────────────────────────────────────── -->
<div class="card">
<div class="card-header">

View file

@ -73,6 +73,22 @@
</div>
</div>
<div class="admin-row" id="mr-web-row" hidden>
<strong class="admin-row-label">Web search</strong>
<div style="flex:1;display:flex;flex-direction:column;gap:4px;min-width:0;">
<label style="display:flex;align-items:center;gap:8px;font-size:13px;">
<input type="checkbox" id="mr-web-search">
Let the model look something up online
</label>
<p style="margin:0;font-size:12px;color:var(--g500);">
For something newer than the library &mdash; a guideline published this year, a recent
approval. <strong>The search query leaves this network</strong> and reaches the provider
your administrator configured; the library excerpts and the resource itself never do.
Do not put anything identifying in the topic if you turn this on.
</p>
</div>
</div>
<div class="admin-row" style="align-items:flex-start;">
<label for="mr-refinement" class="admin-row-label">Instructions <span style="color:var(--g400);font-weight:400;">(optional)</span></label>
<textarea id="mr-refinement" class="admin-control" style="min-height:70px;resize:vertical;font-family:inherit;" placeholder="e.g. for FY1s, case-based, emphasise red flags"></textarea>

View file

@ -85,6 +85,7 @@ function adminTabActive() {
if (el) el.textContent = stats.todayApiCalls !== undefined ? stats.todayApiCalls : '—';
updateRegStatus(settings.registrationEnabled !== false);
loadWebSearch();
})
.catch(function(err) { console.error('[Admin] Settings load failed:', err); });
}
@ -342,6 +343,8 @@ function adminTabActive() {
// Save buttons
if (e.target.closest('#btn-save-announcement')) saveAnnouncement();
if (e.target.closest('#btn-save-flags')) saveFlags();
if (e.target.closest('#btn-save-websearch')) saveWebSearch();
if (e.target.closest('#btn-test-websearch')) testWebSearch();
if (e.target.closest('#btn-save-email')) saveEmail();
if (e.target.closest('#btn-test-email')) sendTestEmail();
if (e.target.closest('#btn-save-smtp')) saveSmtp();
@ -413,6 +416,71 @@ function adminTabActive() {
}).catch(function() { showToast('Save failed', 'error'); });
}
// ---- WEB SEARCH ----
// Its own endpoints rather than the generic setter: the key is masked on read
// and a blank field means "keep what is there", so changing the provider does
// not silently wipe a working key.
function loadWebSearch() {
fetch('/api/admin/websearch', { headers: getAuthHeaders() })
.then(function(r) { return r.json(); })
.then(function(data) {
if (!data || !data.success) return;
var cfg = data.config || {};
var set = function(id, value) { var el = document.getElementById(id); if (el) el.value = value || ''; };
set('ws-enabled', cfg['websearch.enabled'] === 'true' ? 'true' : 'false');
set('ws-provider', cfg['websearch.provider'] || 'tavily');
set('ws-base-url', cfg['websearch.base_url']);
var key = document.getElementById('ws-api-key');
if (key) key.placeholder = cfg['websearch.api_key'] || 'Leave blank to keep the current key';
})
.catch(function() {});
}
function saveWebSearch() {
var status = document.getElementById('ws-status');
if (status) status.textContent = 'Saving...';
fetch('/api/admin/websearch', {
method: 'PUT', headers: getAuthHeaders(),
body: JSON.stringify({
enabled: (document.getElementById('ws-enabled') || {}).value,
provider: (document.getElementById('ws-provider') || {}).value,
apiKey: (document.getElementById('ws-api-key') || {}).value,
baseUrl: (document.getElementById('ws-base-url') || {}).value
})
})
.then(function(r) { return r.json(); })
.then(function(data) {
if (!data.success) throw new Error(data.error || 'Save failed');
if (status) status.textContent = 'Saved ' + new Date().toLocaleTimeString() + '.';
var key = document.getElementById('ws-api-key');
if (key) key.value = '';
showToast('Web search settings saved', 'success');
loadWebSearch();
})
.catch(function(err) {
if (status) status.textContent = 'Not saved.';
showToast(err.message, 'error');
});
}
function testWebSearch() {
var status = document.getElementById('ws-status');
if (status) status.textContent = 'Searching...';
fetch('/api/admin/websearch/test', {
method: 'POST', headers: getAuthHeaders(), body: JSON.stringify({})
})
.then(function(r) { return r.json(); })
.then(function(data) {
if (!status) return;
status.textContent = data.success
? data.count + ' results from ' + data.provider + ' — ' +
(data.sample || []).map(function(x) { return x.title; }).slice(0, 2).join('; ')
: 'No results: ' + (data.reason || 'unknown');
status.style.color = data.success ? 'var(--green)' : 'var(--red)';
})
.catch(function(err) { if (status) { status.textContent = err.message; status.style.color = 'var(--red)'; } });
}
// ---- FEATURE FLAGS ----
function saveFlags() {

View file

@ -60,6 +60,10 @@
if (modelRow) modelRow.hidden = models.length < 2;
var imagesRow = document.getElementById('mr-images-row');
if (imagesRow) imagesRow.hidden = !data.imagesAvailable;
// Hidden entirely unless an administrator enabled it, so the option
// never appears as something a user could turn on and be refused.
var webRow = document.getElementById('mr-web-row');
if (webRow) webRow.hidden = !data.webSearchAvailable;
})
.catch(function () { /* the defaults still work without this */ });
}
@ -99,7 +103,8 @@
refinement: (document.getElementById('mr-refinement') || {}).value || '',
useCorpus: corpusBox && corpusBox.checked === false ? 'false' : 'true',
model: (document.getElementById('mr-model') || {}).value || '',
withImages: (document.getElementById('mr-with-images') || {}).checked ? 'true' : 'false'
withImages: (document.getElementById('mr-with-images') || {}).checked ? 'true' : 'false',
withWebSearch: (document.getElementById('mr-web-search') || {}).checked ? 'true' : 'false'
})
})
.then(function (r) { return r.json(); })
@ -112,6 +117,13 @@
? 'Saved. Written from ' + g.count + ' library excerpt' + (g.count === 1 ? '' : 's') + '.'
: 'Saved. Not grounded' + (g.reason ? ' — ' + g.reason : '') + '; written from the model alone.',
g.used ? 'good' : null);
// Say what was searched for. A query that left the network is worth
// showing plainly rather than leaving someone to wonder.
if (data.webSearch && typeof showToast === 'function') {
showToast(data.webSearch.count
? 'Searched the web for "' + data.webSearch.query + '" — ' + data.webSearch.count + ' results used.'
: 'Web search found nothing for "' + data.webSearch.query + '".', 'info');
}
if ((data.imageJobs || []).length && typeof showToast === 'function') {
showToast('An illustration is being generated; it will appear in your image history.', 'info');
}

View file

@ -969,4 +969,66 @@ router.put('/config/:key(*)', async function(req, res) {
} catch (e) { res.status(500).json({ error: 'Request failed' }); }
});
// ── Web search ──────────────────────────────────────────────
// Its own routes rather than the generic config setter, because the key must be
// masked on read and preserved when the field is left blank — the same handling
// the OIDC client secret gets.
var WEBSEARCH_KEYS = ['websearch.enabled', 'websearch.provider', 'websearch.api_key', 'websearch.base_url'];
router.get('/websearch', adminMiddleware, async function (req, res) {
try {
var out = {};
for (var i = 0; i < WEBSEARCH_KEYS.length; i++) {
out[WEBSEARCH_KEYS[i]] = await db.getSetting(WEBSEARCH_KEYS[i], '') || '';
}
// Never send the key back. Enough tail to recognise which one is set.
if (out['websearch.api_key']) {
out['websearch.api_key'] = '••••••••' + out['websearch.api_key'].slice(-4);
}
res.json({ success: true, config: out });
} catch (err) {
logger.error('GET /admin/websearch', err.message);
res.status(500).json({ error: 'Could not load web search settings' });
}
});
router.put('/websearch', adminMiddleware, async function (req, res) {
try {
var providers = require('../utils/webSearch').PROVIDERS;
var provider = String(req.body.provider || 'tavily');
if (providers.indexOf(provider) === -1) return res.status(400).json({ error: 'Unknown provider' });
await db.setSetting('websearch.enabled', String(req.body.enabled) === 'true' ? 'true' : 'false');
await db.setSetting('websearch.provider', provider);
await db.setSetting('websearch.base_url', String(req.body.baseUrl || '').trim().slice(0, 500));
// A blank field means "leave it alone", so editing the provider does not
// silently wipe the key that was already working.
var key = String(req.body.apiKey || '').trim();
if (key && key.indexOf('•') === -1) await db.setSetting('websearch.api_key', key.slice(0, 400));
res.json({ success: true });
} catch (err) {
logger.error('PUT /admin/websearch', err.message);
res.status(500).json({ error: 'Could not save web search settings' });
}
});
router.post('/websearch/test', adminMiddleware, async function (req, res) {
try {
var webSearch = require('../utils/webSearch');
var found = await webSearch.search(String(req.body.query || 'paediatric bronchiolitis guideline'));
res.json({
success: !found.reason,
provider: found.provider || null,
count: found.results.length,
reason: found.reason || null,
sample: found.results.slice(0, 3).map(function (r) { return { title: r.title, url: r.url }; })
});
} catch (err) {
res.status(502).json({ success: false, reason: err.message });
}
});
module.exports = router;

View file

@ -21,6 +21,7 @@ var { authMiddleware } = require('../middleware/auth');
var { callAI } = require('../utils/ai');
var learningRetrieval = require('../utils/learningRetrieval');
var imageTool = require('../utils/imageTool');
var webSearch = require('../utils/webSearch');
var documentExport = require('../utils/documentExport');
// Scoped to this router's own prefix. Mounted on /api, a bare
@ -109,7 +110,8 @@ router.get('/my-resources/options', async function (req, res) {
success: true,
models: models.allowed,
defaultModel: models.configured,
imagesAvailable: Boolean(await db.getSetting('clinical_assistant.image_model', ''))
imagesAvailable: Boolean(await db.getSetting('clinical_assistant.image_model', '')),
webSearchAvailable: await webSearch.isAvailable()
});
} catch (err) {
console.error('[my-resources] options:', err.message);
@ -147,14 +149,43 @@ router.post('/my-resources/generate', async function (req, res) {
// author ticked the box, because a model handed a drawing tool will find a
// reason to use it, and most teaching material does not want one.
var wantsImages = String(req.body.withImages) === 'true' || req.body.withImages === true;
// Off unless asked for, and unavailable unless an administrator enabled it.
// This is the only path here that sends text outside the building.
var wantsWeb = (String(req.body.withWebSearch) === 'true' || req.body.withWebSearch === true)
&& await webSearch.isAvailable();
var imageModel = wantsImages ? String(await db.getSetting('clinical_assistant.image_model', '') || '') : '';
var model = await resolveModel(req.body.model);
var messages = [{ role: 'user', content: prompt }];
var options = { model: model, temperature: 0.3 };
var ai = await callAI(messages, wantsImages && imageModel
? Object.assign({}, options, { tools: imageTool.tools })
: options);
// Tools the model may reach for on this generation, and only these.
var tools = [];
if (wantsImages && imageModel) tools = tools.concat(imageTool.tools);
if (wantsWeb) tools = tools.concat(webSearch.tools);
var ai = await callAI(messages, tools.length ? Object.assign({}, options, { tools: tools }) : options);
// A web_search call is answered here and the model asked to continue, so
// the search result reaches the resource as material rather than as a tool
// transcript. One round only: a model allowed to keep searching will.
var webUsed = null;
if (wantsWeb && ai.toolCalls && ai.toolCalls.length) {
var call = ai.toolCalls.find(function (c) {
return c && c.function && c.function.name === 'web_search';
});
if (call) {
var args = {};
try { args = JSON.parse(call.function.arguments || '{}'); } catch (e) { args = {}; }
var found = await webSearch.search(args.query);
webUsed = { query: String(args.query || ''), count: found.results.length, reason: found.reason };
ai = await callAI(messages.concat([
{ role: 'assistant', content: null, tool_calls: [call] },
{ role: 'tool', tool_call_id: call.id, content: found.results.length
? webSearch.formatForPrompt(found.results)
: 'No results. Write the resource without web material.' }
]), Object.assign({}, options, { tools: tools, toolChoice: 'none' }));
}
}
// The same dispatcher the assistant uses, so an image generated here is
// owned, queued and rendered exactly as one generated there.
@ -181,6 +212,7 @@ router.post('/my-resources/generate', async function (req, res) {
markdown: markdown,
grounding: { used: Boolean(corpus.context), count: corpus.sources.length, reason: corpus.reason || null },
imageJobs: ai.imageJobs || [],
webSearch: webUsed,
model: ai && ai.model
});
} catch (err) {

161
src/utils/webSearch.js Normal file
View file

@ -0,0 +1,161 @@
// ============================================================
// WEB SEARCH
// Optional, admin-enabled, and off by default.
//
// This is the one feature here that sends text outside the building. A topic
// someone types while writing teaching material can carry clinical detail, and
// a search provider is a third party with its own logging and retention. So:
// disabled unless an administrator turns it on, opt-in per generation even
// then, and only the search query leaves — never the corpus excerpts, never the
// generated content, never anything about the user.
//
// Four providers, one shape. Tavily and Brave are answer-oriented APIs, Serper
// fronts Google, and SearXNG is self-hosted — the only one where the query does
// not reach a commercial third party at all, which is why it is worth having
// even though it needs somewhere to run.
// ============================================================
var db = require('../db/database');
var PROVIDERS = ['tavily', 'serper', 'brave', 'searxng'];
var MAX_RESULTS = 8;
var TIMEOUT_MS = 15000;
async function settings() {
return {
enabled: String(await db.getSetting('websearch.enabled', 'false')) === 'true',
provider: String(await db.getSetting('websearch.provider', 'tavily') || 'tavily'),
apiKey: String(await db.getSetting('websearch.api_key', '') || ''),
// SearXNG has no key; it needs somewhere to reach instead.
baseUrl: String(await db.getSetting('websearch.base_url', '') || '')
};
}
async function isAvailable() {
var s = await settings();
if (!s.enabled || PROVIDERS.indexOf(s.provider) === -1) return false;
return s.provider === 'searxng' ? Boolean(s.baseUrl) : Boolean(s.apiKey);
}
function clip(text, n) {
return String(text || '').replace(/\s+/g, ' ').trim().slice(0, n);
}
// One shape out of every provider, so the prompt never has to know which one is
// configured and swapping provider changes nothing downstream.
function normalize(items) {
return (items || []).slice(0, MAX_RESULTS).map(function (item) {
return {
title: clip(item.title, 200) || 'Untitled',
url: clip(item.url, 500),
snippet: clip(item.snippet, 1200)
};
}).filter(function (r) { return r.url; });
}
var ADAPTERS = {
async tavily(query, s) {
var r = await fetch('https://api.tavily.com/search', {
method: 'POST',
headers: { 'Content-Type': 'application/json', Authorization: 'Bearer ' + s.apiKey },
body: JSON.stringify({ query: query, max_results: MAX_RESULTS, search_depth: 'basic' }),
signal: AbortSignal.timeout(TIMEOUT_MS)
});
if (!r.ok) throw new Error('Tavily returned ' + r.status);
var data = await r.json();
return normalize((data.results || []).map(function (x) {
return { title: x.title, url: x.url, snippet: x.content };
}));
},
async serper(query, s) {
var r = await fetch('https://google.serper.dev/search', {
method: 'POST',
headers: { 'Content-Type': 'application/json', 'X-API-KEY': s.apiKey },
body: JSON.stringify({ q: query, num: MAX_RESULTS }),
signal: AbortSignal.timeout(TIMEOUT_MS)
});
if (!r.ok) throw new Error('Serper returned ' + r.status);
var data = await r.json();
return normalize((data.organic || []).map(function (x) {
return { title: x.title, url: x.link, snippet: x.snippet };
}));
},
async brave(query, s) {
var url = 'https://api.search.brave.com/res/v1/web/search?count=' + MAX_RESULTS +
'&q=' + encodeURIComponent(query);
var r = await fetch(url, {
headers: { Accept: 'application/json', 'X-Subscription-Token': s.apiKey },
signal: AbortSignal.timeout(TIMEOUT_MS)
});
if (!r.ok) throw new Error('Brave returned ' + r.status);
var data = await r.json();
return normalize(((data.web || {}).results || []).map(function (x) {
return { title: x.title, url: x.url, snippet: x.description };
}));
},
async searxng(query, s) {
var base = s.baseUrl.replace(/\/+$/, '');
var r = await fetch(base + '/search?format=json&q=' + encodeURIComponent(query), {
headers: { Accept: 'application/json' },
signal: AbortSignal.timeout(TIMEOUT_MS)
});
if (!r.ok) throw new Error('SearXNG returned ' + r.status);
var data = await r.json();
return normalize((data.results || []).map(function (x) {
return { title: x.title, url: x.url, snippet: x.content };
}));
}
};
/**
* Run one search.
*
* Never throws: a failed search must not fail a generation, exactly as a failed
* corpus retrieval does not. The caller is told what happened so it can say so.
*/
async function search(query) {
var text = String(query || '').trim().slice(0, 400);
if (!text) return { results: [], reason: 'empty query' };
try {
var s = await settings();
if (!s.enabled) return { results: [], reason: 'web search is disabled' };
var adapter = ADAPTERS[s.provider];
if (!adapter) return { results: [], reason: 'unknown provider: ' + s.provider };
if (s.provider === 'searxng' ? !s.baseUrl : !s.apiKey) {
return { results: [], reason: s.provider + ' is not configured' };
}
var results = await adapter(text, s);
if (!results.length) return { results: [], reason: 'no results' };
return { results: results, reason: null, provider: s.provider };
} catch (e) {
return { results: [], reason: e.message || 'search failed' };
}
}
// The tool a model may call. Described tightly: a model given a search tool will
// reach for it constantly unless told when not to.
var tools = [{
type: 'function',
function: {
name: 'web_search',
description: 'Search the public web for current information — a guideline published after your training, a drug approval, an outbreak. ' +
'Do not use it for settled clinical knowledge, which the indexed library and your own training already cover. ' +
'One search per resource at most. Never send patient details or anything identifying.',
parameters: {
type: 'object', additionalProperties: false,
properties: { query: { type: 'string', minLength: 3, maxLength: 400 } },
required: ['query']
}
}
}];
function formatForPrompt(results) {
return results.map(function (r, i) {
return '(' + (i + 1) + ') ' + r.title + '\n' + r.url + '\n' + r.snippet;
}).join('\n\n');
}
module.exports = { search, isAvailable, settings, tools, formatForPrompt, PROVIDERS, MAX_RESULTS };

View file

@ -132,7 +132,9 @@ test('illustration is opt-in, and reuses the assistants image tool', () => {
// A model handed a drawing tool will find a reason to use it, so the tool is
// only offered when the author asked for one.
assert.match(route, /var wantsImages = String\(req\.body\.withImages\) === 'true'/);
assert.match(route, /wantsImages && imageModel\s*\n?\s*\? Object\.assign\(\{\}, options, \{ tools: imageTool\.tools \}\)/);
// Tools are assembled per generation: only what the author asked for.
assert.match(route, /if \(wantsImages && imageModel\) tools = tools\.concat\(imageTool\.tools\);/);
assert.match(route, /if \(wantsWeb\) tools = tools\.concat\(webSearch\.tools\);/);
// The same dispatcher the assistant uses, so an image made here is owned,
// queued and rendered identically to one made there.
assert.match(route, /imageTool\.dispatch\(ai, \{/);

122
test/web-search.test.js Normal file
View file

@ -0,0 +1,122 @@
const test = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const vm = require('node:vm');
const root = path.join(__dirname, '..');
const read = file => fs.readFileSync(path.join(root, file), 'utf8');
function load(settings, fetchImpl) {
const module = { exports: {} };
vm.runInNewContext(read('src/utils/webSearch.js'), {
module, exports: module.exports, console: { warn() {}, info() {} },
fetch: fetchImpl, AbortSignal: { timeout: () => null },
require(name) {
if (name === '../db/database') return { getSetting: async (k, d) => (k in settings ? settings[k] : d) };
throw new Error('unexpected import: ' + name);
}
});
return module.exports;
}
const ON = {
'websearch.enabled': 'true', 'websearch.provider': 'tavily', 'websearch.api_key': 'k', 'websearch.base_url': ''
};
test('web search is off until an administrator turns it on', async () => {
// This is the only path that sends text outside the building, so the default
// has to be the safe one and nothing should be able to flip it implicitly.
const off = load({}, async () => { throw new Error('must not be called'); });
assert.equal(await off.isAvailable(), false);
const out = await off.search('anything');
assert.equal(out.results.length, 0);
assert.match(out.reason, /disabled/);
// Enabled but unconfigured is still unavailable — no silent half-state.
const noKey = load({ 'websearch.enabled': 'true', 'websearch.provider': 'tavily' },
async () => { throw new Error('must not be called'); });
assert.equal(await noKey.isAvailable(), false);
assert.match((await noKey.search('x')).reason, /not configured/);
// SearXNG needs a URL rather than a key, and is judged on that.
const searx = load({ 'websearch.enabled': 'true', 'websearch.provider': 'searxng', 'websearch.base_url': 'https://s.example' },
async () => ({ ok: true, json: async () => ({ results: [] }) }));
assert.equal(await searx.isAvailable(), true);
});
test('every provider comes back in the same shape', async () => {
const cases = [
['tavily', { results: [{ title: 'T', url: 'https://a', content: 'snippet a' }] }],
['serper', { organic: [{ title: 'T', link: 'https://a', snippet: 'snippet a' }] }],
['brave', { web: { results: [{ title: 'T', url: 'https://a', description: 'snippet a' }] } }],
['searxng', { results: [{ title: 'T', url: 'https://a', content: 'snippet a' }] }]
];
for (const [provider, payload] of cases) {
const lib = load(
{ 'websearch.enabled': 'true', 'websearch.provider': provider, 'websearch.api_key': 'k', 'websearch.base_url': 'https://s.example' },
async () => ({ ok: true, status: 200, json: async () => payload }));
const out = await lib.search('bronchiolitis');
assert.equal(out.results.length, 1, provider + ' returned a result');
assert.deepEqual(Object.keys(out.results[0]).sort(), ['snippet', 'title', 'url'],
provider + ' normalises to one shape');
assert.equal(out.provider, provider);
}
});
test('a failed search never fails the generation', async () => {
// Same contract as corpus retrieval: the resource is written without it, and
// the caller is told why rather than shown an error page.
const lib = load(ON, async () => { throw new Error('provider unreachable'); });
const out = await lib.search('bronchiolitis');
assert.equal(out.results.length, 0);
assert.match(out.reason, /provider unreachable/);
const http = load(ON, async () => ({ ok: false, status: 429, json: async () => ({}) }));
assert.match((await http.search('x')).reason, /429/);
});
test('results are bounded, and a result with no URL is dropped', async () => {
const many = Array.from({ length: 40 }, (_, i) => ({ title: 'T' + i, url: 'https://a/' + i, content: 'x'.repeat(4000) }));
many.push({ title: 'no url', url: '', content: 'y' });
const lib = load(ON, async () => ({ ok: true, json: async () => ({ results: many }) }));
const out = await lib.search('bronchiolitis');
assert.equal(out.results.length, lib.MAX_RESULTS, 'capped');
assert.ok(out.results.every(r => r.url), 'nothing without a URL');
assert.ok(out.results.every(r => r.snippet.length <= 1200), 'snippets clipped');
});
test('the tool tells the model when NOT to search', () => {
const src = read('src/utils/webSearch.js');
// A model given a search tool will reach for it constantly unless told
// otherwise, and settled clinical knowledge is what the corpus is for.
assert.match(src, /Do not use it for settled clinical knowledge/);
assert.match(src, /One search per resource at most/);
assert.match(src, /Never send patient details or anything identifying/);
// One round only, enforced in the route rather than trusted to the model.
const route = read('src/routes/myResources.js');
assert.match(route, /toolChoice: 'none'/);
assert.match(route, /var wantsWeb = \(String\(req\.body\.withWebSearch\) === 'true'/);
assert.match(route, /&& await webSearch\.isAvailable\(\)/, 'and the server checks, not just the UI');
});
test('the key is masked on read and preserved when left blank', () => {
const admin = read('src/routes/adminConfig.js');
// Same handling the OIDC client secret gets.
assert.match(admin, /out\['websearch\.api_key'\] = '••••••••' \+ out\['websearch\.api_key'\]\.slice\(-4\)/);
// Changing the provider must not silently wipe a working key.
assert.match(admin, /if \(key && key\.indexOf\('•'\) === -1\) await db\.setSetting\('websearch\.api_key'/);
assert.match(admin, /if \(providers\.indexOf\(provider\) === -1\)/, 'and the provider is validated');
});
test('both screens say plainly that a query leaves the network', () => {
assert.match(read('public/components/admin.html'), /This sends text outside the building/);
assert.match(read('public/components/admin.html'), /SearXNG is the only\s*\n?\s*option here that you host yourself/);
const mine = read('public/components/my-resources.html');
assert.match(mine, /The search query leaves this network/);
assert.match(mine, /Do not put anything identifying in the topic/);
// And it is hidden entirely when unavailable, so nobody ticks a box that
// cannot work.
assert.match(read('public/js/myResources.js'), /if \(webRow\) webRow\.hidden = !data\.webSearchAvailable;/);
});