diff --git a/TODO.md b/TODO.md index c853adea..633631eb 100644 --- a/TODO.md +++ b/TODO.md @@ -43,26 +43,27 @@ and green (674 tests, three consecutive clean runs). per request. Needs an admin setting for which collections to search, then fan-out and merge — `dedupeSources` in `src/utils/clinicalRetrieval.js` already merges and renumbers. See `clinical-assist/COLLECTIONS.md`. -- [ ] **Confirm mail indexing end to end.** Six separate breaks are fixed, and - mail is enabled (`BASIC_INDEXING_MAIL_ENABLED=true`), but no mail has been - indexed yet: the file pass was still working through 472 large PDFs (68 - done in six hours, some over 1,500 chunks each) and mail runs after it. - On 2026-09-11 the indexed folder was changed from `Documents` to - `Personal assistant`, which currently lists 0 files, so the file pass is - now trivial and mail should be reached on a scan cycle (every 300s). - Watch for `[SCAN-*] Mail messages: N seen, M queued`. - Note: the 68 files already indexed from `Documents` are still in the - collection. Say the word if they should be cleared. -- [ ] **`image-size` DoS advisory (high), no upstream fix.** Every published - version up to 2.0.2 is affected; `npm audit fix` only offers a breaking - downgrade of pptxgenjs. Not reachable here: the only `addImage` call - (`src/routes/learningAI.js`) is fed PNGs the app generated itself and - fetched with an ownership check, never an uploaded file. Uploads are - allowlisted to jpeg/png/gif and checked against their magic bytes. Revisit - when pptxgenjs ships a patched dependency. -- [ ] **`uuid` advisory (moderate) via gaxios via Google auth.** Not reachable: - the flaw needs v3/v5/v6 with an explicit buffer, and gaxios calls only - `v4()` for a multipart boundary. Forcing an override risks Vertex auth. +- [x] **Mail indexing works.** It was never reached: mail ran last, after nine + other sources, and Tables alone is thousands of rows at about a second + each. Mail leads now — it is the only bounded source (identities only, + capped by `BASIC_INDEXING_MAIL_MAX_MESSAGES`, bodies left to the + processor), so it cannot starve the others the way they starved it. + Messages are indexing. +- [x] **Mail attachments were never indexed.** An attachment's id is its index + within its message, so `/api/attachments/{id}` meant nothing and answered + 500 every time. Fixed to `/api/messages/{id}/attachment/{id}`, verified + live against a real message. +- [x] **The basic collection is renamed** `personal_assistant_bge_m3_1024` + (was `basic_bge_m3_1024_v2`), matching `mcp_bge_m3_1024` on the clinical + side. Milvus grants name the collection, so the rename revoked + basic_reader/basic_writer; `bootstrap_basic.py` restored them, but it + must be bind-mounted because the operator image ships an older copy. +- [ ] **The indexed folder is `Personal assistant`,** and the setting now takes + a comma-separated list (`Personal assistant,Clinical Notes`), each walked + recursively. Note the file reconciliation removed the chunks of the 68 + `Documents` files, since a complete listing is the deletion authority and + they are no longer under an indexed root. Entities went 27,930 -> ~9,700. + Re-add those files under an indexed folder if they are still wanted. ## Deployment readiness (CI/CD and Kubernetes)