519f2e572a
Search - Retrieval was hybrid in name only: the keyword filter was applied to the SQL query, so results were the *intersection* of the two rankers. A question that matched the meaning but not the literal string could never be returned. It is now a union, fused with Reciprocal Rank Fusion (a text rank and a cosine distance are not on comparable scales, so RRF uses only their orderings). - Added a generated `search_vector` tsvector + GIN index, so the lexical half is ranked full text rather than ILIKE substring matching. - Chose Postgres + pgvector over OpenSearch/Elasticsearch: a search cluster would add a second datastore to keep in sync and a JVM on this host, to replace an index Postgres maintains inside the same transaction. - Removed the keyword-only mode. It looks precise but silently drops the question that asks the same thing in different words. Embeddings — measured on 500 real questions, using each question's own explanation as a paraphrase query (known answer, no hand labelling): bge-small (local CPU, 384d) R@1 0.840 R@5 0.953 186ms/query bge-m3 (LiteLLM proxy, 1024d) R@1 0.847 R@5 0.973 93ms/query BGE-M3 wins on both quality and latency and needs no extra credential, since llm.danvics.com already serves `openrouter-bge-m3`. Three gaps this exposed, all fixed: - Nothing recorded which model produced a stored vector, so changing models silently mixed incomparable spaces. `embedding_model` / `embedded_at` now stamp every vector, `GET /admin/embedding/health` reports current vs stale vs missing, and regeneration defaults to stale-only. - The generator read the model from env while the stamp read a Redis override, so a vector could be labelled with a model that did not produce it. Both now resolve through one function, with a regression test. - Embedding at creation is best effort, and a failure left a question invisible to semantic search forever. `retry_missing_embeddings` runs every 15 minutes via Celery beat and backfills missing or stale rows. - Query embeddings are cached in Redis per model, so typing is not a network round-trip per keystroke. `dimensions` is only sent to OpenAI's embedding-3 family; BGE-M3 rejects it. Tests: 8 new backend tests (union not intersection, fusion ordering, per-ranker failure degradation, provenance stamping, stale/missing accounting, generator and stamp agreement). Full suites green: 95 backend, 127 frontend, build clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_014yhHB8Pc7oQqyqn2Vo9DXA
26 lines
902 B
Python
26 lines
902 B
Python
"""Record which model produced each stored embedding.
|
|
|
|
Without this a model change silently mixes vectors from different embedding
|
|
spaces, and cosine distance between them is meaningless.
|
|
|
|
Revision ID: r0d1e2f3a475
|
|
Revises: q9c0d1e2f364
|
|
"""
|
|
from alembic import op
|
|
|
|
revision = "r0d1e2f3a475"
|
|
down_revision = "q9c0d1e2f364"
|
|
branch_labels = None
|
|
depends_on = None
|
|
|
|
|
|
def upgrade():
|
|
op.execute("ALTER TABLE questions ADD COLUMN IF NOT EXISTS embedding_model VARCHAR(120)")
|
|
op.execute("ALTER TABLE questions ADD COLUMN IF NOT EXISTS embedded_at TIMESTAMP")
|
|
op.execute("CREATE INDEX IF NOT EXISTS ix_questions_embedding_model ON questions(embedding_model)")
|
|
|
|
|
|
def downgrade():
|
|
op.execute("DROP INDEX IF EXISTS ix_questions_embedding_model")
|
|
op.execute("ALTER TABLE questions DROP COLUMN IF EXISTS embedded_at")
|
|
op.execute("ALTER TABLE questions DROP COLUMN IF EXISTS embedding_model")
|