The query files encode the speaker into the text, so every retrieval query
arrived as "|user|: How many teams are in the NFL?". That reaches the
embedder, the BM25 query and the reranker's query.
Measured paired over 777 queries on four domains: stripping is worth +3.60pp
recall@5 with a reranker (94 queries better, 33 worse, 650 tied) and nothing
without one (40 better, 40 worse). A cross-encoder scores query against
document directly, so junk tokens on the query side hurt it where a
bag-of-words branch and a pooled embedding absorb them.
Confined to the retrieval query files: 208 of 208 in both lastturn and
rewrite carry it, while QA turn texts, answers and live questions carry none.
Changes retrieval scores for mtrag_clapnq, mtrag_clapnq_rewrite,
mtrag_federated and mtrag_pooled. The single-database direction is small and
signed: hybrid -0.36pp, vector -1.83pp, FTS +1.25pp, the branches moving
oppositely and nearly cancelling.
Claude-Session: https://claude.ai/code/session_01WhudUtZm6qqiuv8Y1sbwSc
IBM's MTRAG benchmark (ClapNQ domain, pinned repo SHA): retrieval with
Recall@k/nDCG@k against binary qrels, gold-prefix QA replaying reference
conversation prefixes as message history, and live-session replay
carrying the model's own answers and tool history across turns.
Corpus population gains a bounded, resumable batched ingest path.
ConversationInput case type with transcript rendering for the judge,
eligibility-aware citation scoring, refusal precision/recall via a
label-aware RefusalJudge, per-turn verdicts with judged-turn coverage,
and per-turn tool-traffic attributes counted from each turn's new
messages so the arrays survive prior-turn compaction.
