050a7d5241
Citation quality - A citation naming a source that never came back is never rendered as a link, so it appears as plain text and nobody learns it happened. It is now measured on the server, where the answer and the sources both exist, so it is seen whether or not a browser rendered it. - Four Prometheus counters feed a Grafana dashboard (Ped-AI Citation Quality): answers, citations written, answers affected, and individual unresolved markers. Only answers with at least one unresolved citation are stored, with the question and the titles retrieval returned, so an operator can judge whether retrieval came back thin or the model over-cited. Rows expire after 30 days: this is a quality signal, not a transcript log. - Both answer paths are covered. /chat/stream is normal; /chat is the fallback the client uses when streaming fails, so auditing only the first would have hidden exactly the answers produced under failure. - The tracker is resolved on demand and allowed to be absent. Seven test files load this route with a hand-built list of permitted imports, and adding a hard dependency would mean editing all seven — and the eighth written later would break. Observation must never be able to fail an answer, so a missing module simply means no tracking. - Metric registration reuses an already-registered counter, because this module can legitimately load twice in one process. SSO settings on mobile - Six rows were laid out inline: flex with a 160px label and an input that would not shrink, so on a phone the row was wider than the screen with nothing to scroll and no way to reach the rest. They use .admin-row now, which already stacks below 640px. Verified at 390px and 360px: nothing off-screen, no sideways overflow. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Dv6sqaY6Vq3ChZHMem3cnU
34 lines
1.5 KiB
JavaScript
34 lines
1.5 KiB
JavaScript
// Answers whose citations pointed at nothing.
|
|
//
|
|
// The Prometheus counters say how often it happens; this says what happened,
|
|
// so an admin can read the question and the sources and judge whether the
|
|
// retrieval came back thin or the model over-cited.
|
|
//
|
|
// Only answers with at least one unverifiable citation are stored — this is a
|
|
// quality signal, not a transcript log — and rows expire, because the question
|
|
// text is clinical material and should not accumulate indefinitely.
|
|
|
|
exports.up = pgm => {
|
|
pgm.sql(`
|
|
CREATE TABLE IF NOT EXISTS citation_audit (
|
|
id SERIAL PRIMARY KEY,
|
|
user_id INTEGER REFERENCES users(id) ON DELETE SET NULL,
|
|
question TEXT NOT NULL DEFAULT '',
|
|
cited_count INTEGER NOT NULL DEFAULT 0,
|
|
source_count INTEGER NOT NULL DEFAULT 0,
|
|
-- The numbers the model wrote that no source matched.
|
|
unverifiable INTEGER[] NOT NULL DEFAULT '{}',
|
|
-- Titles of what retrieval actually returned, so the two can be compared
|
|
-- without keeping the passages themselves.
|
|
source_titles TEXT[] NOT NULL DEFAULT '{}',
|
|
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
|
expires_at TIMESTAMPTZ NOT NULL DEFAULT NOW() + INTERVAL '30 days'
|
|
);
|
|
CREATE INDEX IF NOT EXISTS idx_citation_audit_created ON citation_audit(created_at DESC);
|
|
CREATE INDEX IF NOT EXISTS idx_citation_audit_expires ON citation_audit(expires_at);
|
|
`);
|
|
};
|
|
|
|
exports.down = pgm => {
|
|
pgm.sql('DROP TABLE IF EXISTS citation_audit;');
|
|
};
|