# What "close enough" means

AI Mode gives one of three answers, and which one is decided by a number rather
than by asking the model to work out its own situation. This is where that
number comes from, what it was measured against, and how to re-measure it.

## Why a number and not a longer prompt

Retrieval could not say "nothing". `hybrid_ids()` fuses a lexical ranker and a
semantic one by reciprocal rank, throws the distances away, and returns the
*union* — so the shortlist was never empty. A question about photosynthesis came
back with six paediatric sources and an instruction to answer only from them.

Adding scenarios to the prompt would have asked the model to classify, in prose,
a situation the data already knows. Classification-by-prose is the part that does
not work, and it is also the part that makes prompts long. So the branch lives in
code: `ai_mode_service.answer_mode()`.

## The three answers

| closeness | mode | what the learner gets |
|---|---|---|
| `>= STRONG_MATCH` | `sourced` | answered from the library, cited |
| `>= ADJACENT_MATCH` | `adjacent` | "nothing covers this directly; the closest is…", then an answer marking which parts came from where |
| below | `open` | one line saying the library does not cover it, then an answer from general knowledge, citing nothing |
| unmeasurable (`None`) | `sourced` | see below |

**Unmeasurable is not low.** No vector database, or a downed encoder, returns
`None`. Retrieval still found its rows by other means, so they are still cited;
dropping every citation because the ruler is missing is the worse failure.

## The numbers, and the measurement behind them

In `backend/app/services/ai_mode_service.py`:

```python
STRONG_MATCH = 0.55
ADJACENT_MATCH = 0.50
```

Measured on 2026-09-12, against the corpus with article bodies embedded — eight
clearly on-topic questions and eight clearly off-topic, scored with
`search_service.top_similarity()` over the article and section corpora:

| | range | examples |
|---|---|---|
| off-topic | **0.339 – 0.499** | the French revolution 0.339 · how do I bake sourdough 0.410 · quantum entanglement 0.448 · javascript closures 0.452 · tell me a joke 0.455 · what is a mortgage 0.462 · discuss love 0.491 · photosynthesis 0.499 |
| on-topic | **0.586 – 0.740** | neonatal jaundice phototherapy 0.586 · what causes croup 0.594 · febrile seizure workup 0.638 · Kawasaki disease 0.660 · testicular torsion 0.713 · bronchiolitis in an infant 0.716 · iron deficiency in a toddler 0.733 · posterior urethral valves 0.740 |

The thresholds sit in the gap between those two bands.

Worth noticing that **"discuss love" scores 0.491, alongside "tell me a joke"**.
It feels adjacent to a paediatrics library — attachment, behaviour — and it is
not: 0.49 is where anything written in English lands against any corpus. That is
the reading to keep in mind. A number in the 0.4s is noise, not a weak signal.

## Why these are not the reranker's score

Retrieval now has a cross-encoder in it (docs/reranking.md), which is a better
judge of a query-document pair than cosine distance is by a wide margin. The
three-state decision still does not go through it, on purpose.

The question here is *"is there anything in this library about this at all"*,
and that is a question about the corpus, not about a shortlist. `top_similarity`
answers it by scanning every embedded row in two corpora through the vector
index, in about 25 ms. A reranker can only score the candidates something else
already shortlisted, so a reranked closeness could not tell "the library does
not cover this" from "retrieval had a bad day" — and on this corpus it is
exactly the uncovered query where the cross-encoder is least trustworthy: asked
how croup is treated at home, with no croup treatment section in the library, it
promotes the *Treatment* section of whatever else is lying around.

There is also a failure argument. Order is a preference, so a reranker being
down costs a worse-ordered page and nothing else. Whether the answer *claims to
come from the library* is a promise, and putting a network hop in the path of a
promise means a proxy restart changes what the assistant asserts.

So: the cross-encoder decides the order of the shortlist; cosine decides what
the answer is allowed to say about it. The numbers below are unchanged and did
not need re-measuring, because nothing that feeds them changed.

## Why these are not `SEMANTIC_FLOOR`

`search_service.SEMANTIC_FLOOR` (0.45) decides what is worth putting in a list,
where a weak hit costs a reader one glance. These decide whether an answer
*claims to come from the library*, and a wrong claim there costs a learner their
trust in every other answer. Different jobs, different numbers, and tying them
together would mean one could not be tuned without moving the other.

## Re-measuring

Re-measure when the corpus changes size or subject — a much larger library
raises the floor for everything, because there is more for any query to be
vaguely near.

```
docker compose exec -T backend python -c "
from app.database import SessionLocal
from app.services.search_service import top_similarity
db = SessionLocal()
for q in ['the French revolution', 'tell me a joke', 'discuss love',
          'what causes croup', 'bronchiolitis in an infant']:
    print(f'{top_similarity(db, q):.3f}  {q}')
"
```

Pick your own on- and off-topic sets, run both, and put the thresholds in the
gap. If there is no gap, the embeddings are wrong before the thresholds are —
that is what happened before the article bodies were indexed, when 98% of the
corpus was embedded on its title and summary alone and every distance was
measuring the wrong thing.