From 081856c6b1f17be1a3baf80d17156de69ffc8836 Mon Sep 17 00:00:00 2001 From: Yiorgis Gozadinos Date: Sat, 25 Jul 2026 14:40:51 +0300 Subject: [PATCH] Instruct capabilities to judge user-attached images --- .../haiku/rag/capabilities/instructions/analysis.md | 4 ++++ haiku_rag_slim/haiku/rag/capabilities/instructions/rag.md | 4 ++++ 2 files changed, 8 insertions(+) diff --git a/haiku_rag_slim/haiku/rag/capabilities/instructions/analysis.md b/haiku_rag_slim/haiku/rag/capabilities/instructions/analysis.md index abeade74..92cbf702 100644 --- a/haiku_rag_slim/haiku/rag/capabilities/instructions/analysis.md +++ b/haiku_rag_slim/haiku/rag/capabilities/instructions/analysis.md @@ -92,6 +92,10 @@ Section tree derived from `heading_level`: `{"doc_id", "title", "tree": [...]}` ### Cross-referencing search results with items Search results include `doc_item_refs` (e.g. `["#/texts/48", "#/tables/0"]`) that correspond to `self_ref` values in `items.jsonl`. To find which section a hit lives in: locate the item by `self_ref`, take its line index, and walk `toc.json` to find the deepest node whose `item_range` contains that index. +## Questions with attached images + +The user may attach images to their question. An attached image is part of the question, not knowledge-base content. Search the knowledge base for the criteria, standards, or facts named in the question text, cite them, and apply them to the attached image. Never refuse merely because the image itself is not in the knowledge base. + ## Strategy 1. Search first. diff --git a/haiku_rag_slim/haiku/rag/capabilities/instructions/rag.md b/haiku_rag_slim/haiku/rag/capabilities/instructions/rag.md index 1ab0c122..93c2ccc1 100644 --- a/haiku_rag_slim/haiku/rag/capabilities/instructions/rag.md +++ b/haiku_rag_slim/haiku/rag/capabilities/instructions/rag.md @@ -21,6 +21,10 @@ Register the chunk IDs that ground your answer. Call this BEFORE writing your fi Use chunk_ids exactly as they appear in the search response — copy the full UUID verbatim. Do not abbreviate, paraphrase, or reconstruct chunk_ids from memory; the tool matches them as opaque strings. +## Questions with attached images + +The user may attach images to their question. An attached image is part of the question, not knowledge-base content. Search the knowledge base for the criteria, standards, or facts named in the question text, cite them, and apply them to the attached image. Never refuse merely because the image itself is not in the knowledge base. + ## How to answer questions 1. Call `rag_search` with relevant keywords from the question