From 77da7aea6cce395f27c557470aacaf2a4a79a902 Mon Sep 17 00:00:00 2001
From: Daniel <daniel@danvics.com>
Date: Wed, 1 Apr 2026 12:57:28 +0200
Subject: [PATCH] Restore original verbatim extraction prompt; switch to Claude
 Opus 4.6
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Extraction prompt:
- Restored full original prompt with strict verbatim instructions
- Explanation field: must include full explanation paragraph + Critique
  verbatim + Content Specifications verbatim (no summarization)
- Correct answer: explicit step-by-step with example showing letter→full text
- Options: extract full text only, no letter prefix
- All 5 rules numbered and explicit

Model switch:
- Default extraction model changed from Claude Haiku 4.5 → Claude Opus 4.6
- Haiku tends to compress/summarize long explanation text
- Opus 4.6 follows verbatim instructions reliably and handles
  the structured PREP format (explanation + Critique + Content Specs)
  without losing content
- Haiku 4.5 still available in the Extraction Model dropdown for
  cost-sensitive use cases

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
---
 backend/app/services/ai_service.py | 47 ++++++++++++++++++------------
 1 file changed, 29 insertions(+), 18 deletions(-)

diff --git a/backend/app/services/ai_service.py b/backend/app/services/ai_service.py
index 7bed936..44a1d2d 100644
--- a/backend/app/services/ai_service.py
+++ b/backend/app/services/ai_service.py
@@ -16,34 +16,45 @@ def _proxy_model(model_id: str) -> str:
 
 EXTRACTION_PROMPT = """You are extracting questions from a PREP (Pediatric Review and Education Program) exam PDF.
 
-These PDFs have questions with a clinical vignette, followed by five answer options A-E.
-Some PDFs include "Correct Answer: X" or "Preferred Response: X" right after the options.
-Some PDFs have the correct answers only at the END of the document — in that case correct_answer will be null.
+These PDFs follow a strict format:
+1. A numbered question with a clinical vignette (patient scenario)
+2. Five answer options labeled A, B, C, D, E
+3. A line "Correct Answer: X" or "Preferred Response: X" where X is the letter of the correct option
+4. An explanation paragraph
+5. A "Critique:" section with detailed reasoning
+6. A "Content Specifications:" section listing the learning objectives
 
-Return ONLY a JSON object:
+Your task: extract every question and return ONLY a JSON object in this exact format:
 
 {{"questions": [
   {{
-    "item_number": "<item number like '193' or '21' — digits only, null if not found>",
-    "question_text": "<full question stem including the clinical vignette>",
+    "item_number": "<item number digits only, e.g. '193' — null if not found>",
+    "question_text": "<full question stem including any patient vignette>",
     "question_type": "mcq",
     "options": ["<option A text>", "<option B text>", "<option C text>", "<option D text>", "<option E text>"],
-    "correct_answer": "<full text of the correct option, NOT the letter — or null if not found>",
-    "explanation": "<explanation/critique/content specs if present, else empty string>",
+    "correct_answer": "<full text of the correct option, NOT the letter>",
+    "explanation": "<explanation paragraph>\\n\\nCritique: <critique section verbatim>\\n\\nContent Specifications: <content spec section verbatim>",
     "page_reference": {page_ref}
   }}
 ]}}
 
-RULES:
-- A new question starts when you see "Item NNN" or "ltem NNN" (OCR artifact for Item).
-- Extract question_text as the full vignette + the "Of the following..." or "Which of the following..." stem.
-- Options are labeled A. B. C. D. E. — extract just the text, not the letter.
-- If "Correct Answer: X" or "Preferred Response: X" appears after the options, resolve it to full option text.
-- CRITICAL: Content immediately after an image reference (e.g. "Item C123A", "Item C123B", figure captions,
-  table data) is NOT a new question. Skip it and wait for the next "Item NNN" number.
-- If no correct answer line is found for a question, set correct_answer to null — do NOT guess.
-- Do NOT omit questions — extract all questions found in the content, even if partial.
-- Return ONLY the JSON — no markdown, no preamble.
+CRITICAL RULES — follow exactly:
+1. CORRECT ANSWER: Find "Correct Answer: X" or "Preferred Response: X" after each question.
+   X is a letter (A–E). Look up the full text of that option and store it as correct_answer.
+   NEVER store just the letter. NEVER guess. If not found, set correct_answer to null.
+   Example: options=["alpha","beta","gamma","delta","epsilon"] and "Correct Answer: C" → correct_answer="gamma"
+
+2. EXPLANATION: Copy the explanation paragraph, then the Critique section, then Content Specifications
+   ALL VERBATIM — do NOT summarize, shorten, or paraphrase anything.
+   Format: "<explanation>\\n\\nCritique: <critique verbatim>\\n\\nContent Specifications: <specs verbatim>"
+
+3. QUESTIONS: Extract ALL questions, even if partially cut off.
+   A new question starts with "Item NNN" or "ltem NNN" (OCR artifact — lowercase l instead of I).
+   Content after an image reference (e.g. "Item C123A", figure captions) is NOT a new question.
+
+4. OPTIONS: Extract just the text for each option, without the letter prefix (A. B. C. D. E.).
+
+5. Return ONLY the JSON — no markdown fences, no explanation, no preamble.
 
 Content from page(s) {page_info}:
 {content}"""