docs: rerank, vision, uploads and the small-talk gate in the synopsis
Four things a later session would otherwise have to rediscover from the code, including the two whose whole point is that they run before the obvious place to look for them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TqXevQJhxFrM7jJg82cgZN
This commit is contained in:
parent
a020e2bb31
commit
7f5e691670
1 changed files with 8 additions and 1 deletions
|
|
@ -55,6 +55,8 @@ backend/app/
|
|||
services/
|
||||
ai_service.py — LLM calls, _proxy_model(), get_model_for_task() fallback chain
|
||||
extraction_modes.py — 6 quiz extraction modes + flashcard generation prompt
|
||||
search_service.py — Hybrid retrieval (BM25 + pgvector, RRF), rerank_ids()
|
||||
rerank_service.py — Cross-encoder rerank via the proxy's /v1/rerank
|
||||
vector_service.py — ChromaDB: store/query page chunks, LiteLLMEmbeddingFunction
|
||||
embedding_service.py — pgvector: embed questions for semantic search
|
||||
pdf_service.py — PyMuPDF: text extraction, image extraction with MD5 hash skip list
|
||||
|
|
@ -106,7 +108,12 @@ frontend/src/
|
|||
- **Tag filtering SQL**: `WHERE tag_id = ANY(:tag_ids) GROUP BY ... HAVING COUNT(DISTINCT tag_id) = :cnt` — AND logic across tags
|
||||
- **Multi-category filtering**: `category_ids` param (comma-separated), uses `.in_()` — OR logic within categories
|
||||
- **Job progress**: Celery tasks push steps to Redis lists (`extraction:steps:{job_id}`), frontend polls `GET /quizzes/job/{job_id}`
|
||||
- **Model selection**: Admin configures models per task (extraction, teach, tts, keyword, flashcard). `get_model_for_task(db, task)` returns (model_id, api_key) with fallback to `settings.LITELLM_MODEL`.
|
||||
- **Model selection**: Admin configures models per task (extraction, teach, article, tts, stt, keyword, flashcard, tool). `get_model_for_task(db, task)` returns (model_id, api_key) with fallback to `settings.LITELLM_MODEL`; `get_configured_model(db, task)` returns None instead of falling back, for work that must not run on a model nobody chose.
|
||||
- **Vision fallback**: `vision_service.image_context(db, images, model_id=...)` returns the message parts to splice into a user message. If the job's model is vision-capable it gets the images; if not, the `tool` model describes them and the primary reads the description. Capability comes from the proxy's `/model/info` `supports_vision` (cached in-process), and where that is absent from a one-off 8px probe (cached in Redis) — never from a hard-coded list of model names. No tool model configured raises `VisionUnavailable`.
|
||||
- **Reranking**: `search_service.rerank_ids(db, q, kind, ids)` reorders the first 50 of a fused ranking with a cross-encoder (`LITELLM_RERANK_MODEL`, default `cohere-rerank-v4.0-pro`, Redis override `settings:rerank_model`). It is a *permutation* — an unset, unreachable or malformed reranker returns the ids untouched, never fewer. Applied to question and article search, the test builder's description path, and the AI Mode shortlist; deliberately not to the typeahead, flashcard/media search, or the AI Mode closeness thresholds. Cached in Redis for a day, keyed on model + query + document text. See docs/reranking.md.
|
||||
- **Uploads**: `file_intake.read()` then `kind_of()` then `text_from()` — size, then type sniffed from the leading bytes, then text. PDF, DOCX and images only (2 MB); an extension is a claim, never the decision. Images are read by the `tool` model, and say so when none is configured.
|
||||
- **Small talk in AI Mode**: `ai_mode_service.is_small_talk()` runs *before* retrieval. A greeting scores 0.46–0.51 against a clinical corpus, which is either side of the adjacency threshold, so the closeness gate cannot be the guard. Mode "chat" means no shortlist, no citations, and no thread name.
|
||||
- **`==key points==`**: a remark plugin (`utils/keyPoints.js`) → `<mark class="key-point">`. Never rewrite the markdown string before parsing: highlights and the read-aloud cursor are offsets into the raw text.
|
||||
- **useEffect dependencies**: Use `.join(',')` on arrays to create a stable string key (e.g., `tagIdsKey`, `catIdsKey`)
|
||||
- **Admin page data refresh**: `loadData(false)` — the `false` param skips the loading spinner on re-fetch after actions
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue