Commit graph

5 commits

Author SHA1 Message Date
LuoXi
84d88f911c
perf: add reasoning_effort passthrough to avoid thinking-mode latency in local models (#1221)
* perf: add reasoning_effort passthrough to avoid thinking-mode latency

Reasoning models (Gemma 4, Qwen 3, etc.) default to thinking mode,
adding 10-40x latency for simple transcript cleanup. Pass through the
OpenAI-compatible reasoning_effort parameter so users can disable
thinking when speed matters more than deep reasoning.

- Add optional reasoning_effort field to ChatCompletionRequest
- Thread through send_chat_completion / send_chat_completion_with_schema
- New set_post_process_reasoning_effort Tauri command for persistence
- Dropdown in post-processing settings, Custom provider only
- i18n for en and ja

* refactor: simplify reasoning_effort to on/off toggle

Addresses review feedback on the reasoning_effort passthrough:
post-processing rarely benefits from reasoning, so a 5-way dropdown
(default/none/low/medium/high) is overkill. Collapse to a single
boolean toggle "Disable reasoning" that defaults to ON for Custom
providers (sends reasoning_effort: "none"). Cloud providers continue
to receive no reasoning_effort parameter.

- Replace post_process_reasoning_effort: Option<String> with
  post_process_disable_reasoning: bool (default true) in settings
- Map bool to Option<String> at the request site (true -> "none",
  false -> omit) so the API contract is unchanged
- Rename Tauri command to set_post_process_disable_reasoning(disable)
  so parameter naming matches the setting and avoids inversion bugs
- Swap the Dropdown for a ToggleSwitch in the Custom provider section
- Collapse five i18n strings to one label + description, and add the
  translation to all 18 additional locales so non-en/ja users don't
  fall back to English for this control

* refactor: remove disable-reasoning toggle, always disable for custom provider

Per PR #1221 discussion with @cjpais: since this is an alpha feature,
simplify the UI by always sending reasoning_effort: "none" for custom
provider post-processing. If this causes issues with specific provider
setups, we can revisit.

* disable for openrouter too

---------

Co-authored-by: CJ Pais <cj@cjpais.com>
2026-04-07 15:19:40 +08:00
Chirag Agggarwal
0cb8ab2162
feat: implement structured outputs for post-processing providers (#706)
* feat: implement structured outputs for Cerebras, OpenRouter, OpenAI, and Apple Intelligence

- Add structured output support with JSON schema in llm_client.rs
- Update actions.rs to use system prompt + user content approach
- Remove legacy ${output} variable substitution for supported providers
- Update Apple Intelligence Swift code to accept system prompts
- All providers now use consistent structured output format
- Remove duplicate check_apple_intelligence_availability function

* wip changes

* fix(structured-outputs): address PR #706 review comments

- Add settings migration to sync supports_structured_output field for existing providers
- Fix fallback behavior: structured output failures now fall through to legacy mode
- Clone api_key to prevent ownership issues in fallback path
- Clean up build_system_prompt(): remove  placeholder entirely
  (instead of replacing with 'the user's message' which reads awkwardly)
- Add warn import from log crate

* refactor(structured-outputs): apply best practice improvements

- Optimize settings migration: use single match instead of double iteration
- Add TRANSCRIPTION_FIELD constant to replace magic strings
- Keep Apple Intelligence behavior unchanged (no API fallback for privacy)

Addresses code review feedback on PR #706:
1. More efficient provider lookup in ensure_post_process_defaults()
2. Eliminates hardcoded 'transcription' string in JSON parsing
3. Maintains privacy-first approach for Apple Intelligence

* fix groq output

---------

Co-authored-by: CJ Pais <cj@cjpais.com>
2026-02-17 12:14:00 +08:00
Dmitry Gordin
5c5e428e06
fix: replace async-openai library with post request (#480)
* fix: replace async-openai library with post request to support custom providers

* move models call to llm_client too

* tweak post processing ui and add groq and cerebras

---------

Co-authored-by: CJ Pais <cj@cjpais.com>
2025-12-27 11:21:27 +07:00
CJ Pais
70ec82285a
format everything (#323) 2025-11-10 11:39:14 +08:00
Luis Eduardo
f7380fa69a
feat: LLM based post-processing (#222)
* Add LLM post-processing feature with settings management and UI integration

* Refactor LLM post-processing settings and prompt selection logic

* Rename CLAUDE.md to more standard AGENTS.md

* Replace Open Router with OpenAI Compatible endpoints for post-processing

* Fix post processing settings api styles

* refactor or something

* simplify

* cleanup a bit?

* remove useless code

* minor improvements

* fix laggy textarea

* Store post-processing results in db

* fix text color

* new default prompt

---------

Co-authored-by: CJ Pais <cj@cjpais.com>
2025-11-05 18:29:03 -08:00