a4999164ae
* feat(models): auto-discover custom Whisper models in models directory Enable automatic discovery of custom GGML-format Whisper models (.bin files) placed in the models directory, so users don't need to modify source code to use their own fine-tuned models. Backend changes: - Add discover_custom_whisper_models() to scan for .bin files - Generate display names from filenames (e.g., "my-model" → "My Model") - Skip predefined model filenames to avoid duplicates - Add 3 unit tests for discovery logic Frontend changes: - Split model dropdown into 3 sections: Custom, Downloaded, Downloadable - Add collapsible section for downloadable models to reduce clutter - Add max-height with scroll for long model lists - Add "Custom" badge for user-provided models * feat(models): make custom Whisper model discovery opt-in [why] Address maintainer concern about support burden from community models. Custom models should be a power-user feature, not enabled by default. [how] Add custom_models_enabled setting (default: false) in Debug settings. Discovery only runs when enabled. Models show "Not officially supported" messaging. Documentation updated with enable steps. * feat(models): add is_custom field and integrate custom models with new UI [why] PR review requested an explicit is_custom field instead of inferring custom status from url === null. Custom models also need proper integration with the new models settings page from PR #478. [how] - Add is_custom: bool to ModelInfo struct, set true on discovered models - Change discover_custom_whisper_models signature from &PathBuf to &Path - Use 0.0 sentinel scores so UI hides score bars for custom models - Add "Custom Models" section to ModelsSettings with 3-way model split - Show "Custom" badge in ModelCard and ModelDropdown - Hide language/translation tags when supported_languages is empty - Remove custom models from available_models on delete instead of just marking as not downloaded (they have no re-download URL) - Update tests with new fields and assertions * chore(i18n): add custom model translation keys to all locales [why] Custom model feature introduces 5 new translation keys that need to be present in all 15 non-English locales for CI to pass. [how] Add English placeholder values for: customModelDescription, modelSelector.custom, settings.models.customModels, settings.debug.customModels.label, settings.debug.customModels.description * fix(models): keep language filter visible when no models match [why] The language filter was inside a conditionally rendered section that disappeared when no downloaded models matched the selected language, leaving the user stuck with no way to change the filter. [how] Always render the "Your Models" header row with the language filter, only conditionally render the model cards below it. * docs: fix custom models section reference in README Model selector → Models settings page. * fix(models): apply custom models toggle immediately without restart [why] Two bugs reported in PR review: (1) app breaks on restart when a custom model was selected and the toggle is disabled — selected_model still points to the missing model, causing "Model not found" error. (2) Custom models remain visible in the UI after toggle-off because the in-memory model list is never updated and no event is emitted. [how] - Add remove_custom_models() and add_custom_models() to ModelManager for runtime mutation of the available_models mutex - On disable: reset selected_model to empty if it's a custom model, then remove custom models from the in-memory list - On enable: run discover_custom_whisper_models against the mutex - Emit model-state-changed event so the frontend refreshes immediately * chore(i18n): remove "restart required" from custom model descriptions Toggle now takes effect immediately, so the restart sentence is inaccurate. Updated all 16 locales and README instructions. * fix(models): clear stale model selection on startup [why] If a custom model file is deleted from disk while it's the selected model, the app gets stuck on "Loading..." forever on next launch because the model ID is not in available_models but auto_select_model_if_needed only checked for empty string. [how] Validate that selected_model exists in available_models before accepting it. If not found, clear the selection so auto-select picks a valid downloaded model. * remove toggle and clean up * format --------- Co-authored-by: CJ Pais <cj@cjpais.com> |
||
|---|---|---|
| .cargo | ||
| .github | ||
| .vscode | ||
| scripts | ||
| sponsor-images | ||
| src | ||
| src-tauri | ||
| tests | ||
| .gitignore | ||
| .prettierignore | ||
| .prettierrc | ||
| AGENTS.md | ||
| BUILD.md | ||
| bun.lock | ||
| CHANGELOG.md | ||
| CLAUDE.md | ||
| CONTRIBUTING.md | ||
| CONTRIBUTING_TRANSLATIONS.md | ||
| CRUSH.md | ||
| eslint.config.js | ||
| flake.lock | ||
| flake.nix | ||
| index.html | ||
| LICENSE | ||
| package.json | ||
| playwright.config.ts | ||
| README.md | ||
| tailwind.config.js | ||
| tsconfig.json | ||
| tsconfig.node.json | ||
| vite.config.ts | ||
Handy
A free, open source, and extensible speech-to-text application that works completely offline.
Handy is a cross-platform desktop application built with Tauri (Rust + React/TypeScript) that provides simple, privacy-focused speech transcription. Press a shortcut, speak, and have your words appear in any text field—all without sending your voice to the cloud.
Why Handy?
Handy was created to fill the gap for a truly open source, extensible speech-to-text tool. As stated on handy.computer:
- Free: Accessibility tooling belongs in everyone's hands, not behind a paywall
- Open Source: Together we can build further. Extend Handy for yourself and contribute to something bigger
- Private: Your voice stays on your computer. Get transcriptions without sending audio to the cloud
- Simple: One tool, one job. Transcribe what you say and put it into a text box
Handy isn't trying to be the best speech-to-text app—it's trying to be the most forkable one.
How It Works
- Press a configurable keyboard shortcut to start/stop recording (or use push-to-talk mode)
- Speak your words while the shortcut is active
- Release and Handy processes your speech using Whisper
- Get your transcribed text pasted directly into whatever app you're using
The process is entirely local:
- Silence is filtered using VAD (Voice Activity Detection) with Silero
- Transcription uses your choice of models:
- Whisper models (Small/Medium/Turbo/Large) with GPU acceleration when available
- Parakeet V3 - CPU-optimized model with excellent performance and automatic language detection
- Works on Windows, macOS, and Linux
Quick Start
Installation
- Download the latest release from the releases page or the website
- macOS: Also available via Homebrew cask:
brew install --cask handy
- macOS: Also available via Homebrew cask:
- Install the application
- Launch Handy and grant necessary system permissions (microphone, accessibility)
- Configure your preferred keyboard shortcuts in Settings
- Start transcribing!
Development Setup
For detailed build instructions including platform-specific requirements, see BUILD.md.
Architecture
Handy is built as a Tauri application combining:
- Frontend: React + TypeScript with Tailwind CSS for the settings UI
- Backend: Rust for system integration, audio processing, and ML inference
- Core Libraries:
whisper-rs: Local speech recognition with Whisper modelstranscription-rs: CPU-optimized speech recognition with Parakeet modelscpal: Cross-platform audio I/Ovad-rs: Voice Activity Detectionrdev: Global keyboard shortcuts and system eventsrubato: Audio resampling
Debug Mode
Handy includes an advanced debug mode for development and troubleshooting. Access it by pressing:
- macOS:
Cmd+Shift+D - Windows/Linux:
Ctrl+Shift+D
Known Issues & Current Limitations
This project is actively being developed and has some known issues. We believe in transparency about the current state:
Major Issues (Help Wanted)
Whisper Model Crashes:
- Whisper models crash on certain system configurations (Windows and Linux)
- Does not affect all systems - issue is configuration-dependent
- If you experience crashes and are a developer, please help to fix and provide debug logs!
Wayland Support (Linux):
- Limited support for Wayland display server
- Requires
wtypeordotoolfor text input to work correctly (see Linux Notes below for installation)
Linux Notes
Text Input Tools:
For reliable text input on Linux, install the appropriate tool for your display server:
| Display Server | Recommended Tool | Install Command |
|---|---|---|
| X11 | xdotool |
sudo apt install xdotool |
| Wayland | wtype |
sudo apt install wtype |
| Both | dotool |
sudo apt install dotool (requires input group) |
- X11: Install
xdotoolfor both direct typing and clipboard paste shortcuts - Wayland: Install
wtype(preferred) ordotoolfor text input to work correctly - dotool setup: Requires adding your user to the
inputgroup:sudo usermod -aG input $USER(then log out and back in)
Without these tools, Handy falls back to enigo which may have limited compatibility, especially on Wayland.
Other Notes:
-
The recording overlay is disabled by default on Linux (
Overlay Position: None) because certain compositors treat it as the active window. When the overlay is visible it can steal focus, which prevents Handy from pasting back into the application that triggered transcription. If you enable the overlay anyway, be aware that clipboard-based pasting might fail or end up in the wrong window. -
If you are having trouble with the app, running with the environment variable
WEBKIT_DISABLE_DMABUF_RENDERER=1may help -
You can manage global shortcuts outside of Handy and still control the app via signals. Sending
SIGUSR2to the Handy process toggles recording on/off, which lets Wayland window managers or other hotkey daemons keep ownership of keybindings. Example (Sway):bindsym $mod+o exec pkill -USR2 -n handypkillhere simply delivers the signal—it does not terminate the process.
Platform Support
- macOS (both Intel and Apple Silicon)
- x64 Windows
- x64 Linux
System Requirements/Recommendations
The following are recommendations for running Handy on your own machine. If you don't meet the system requirements, the performance of the application may be degraded. We are working on improving the performance across all kinds of computers and hardware.
For Whisper Models:
- macOS: M series Mac, Intel Mac
- Windows: Intel, AMD, or NVIDIA GPU
- Linux: Intel, AMD, or NVIDIA GPU
- Ubuntu 22.04, 24.04
For Parakeet V3 Model:
- CPU-only operation - runs on a wide variety of hardware
- Minimum: Intel Skylake (6th gen) or equivalent AMD processors
- Performance: ~5x real-time speed on mid-range hardware (tested on i5)
- Automatic language detection - no manual language selection required
Roadmap & Active Development
We're actively working on several features and improvements. Contributions and feedback are welcome!
In Progress
Debug Logging:
- Adding debug logging to a file to help diagnose issues
macOS Keyboard Improvements:
- Support for Globe key as transcription trigger
- A rewrite of global shortcut handling for MacOS, and potentially other OS's too.
Opt-in Analytics:
- Collect anonymous usage data to help improve Handy
- Privacy-first approach with clear opt-in
Settings Refactoring:
- Cleanup and refactor settings system which is becoming bloated and messy
- Implement better abstractions for settings management
Tauri Commands Cleanup:
- Abstract and organize Tauri command patterns
- Investigate tauri-specta for improved type safety and organization
Troubleshooting
Manual Model Installation (For Proxy Users or Network Restrictions)
If you're behind a proxy, firewall, or in a restricted network environment where Handy cannot download models automatically, you can manually download and install them. The URLs are publicly accessible from any browser.
Step 1: Find Your App Data Directory
- Open Handy settings
- Navigate to the About section
- Copy the "App Data Directory" path shown there, or use the shortcuts:
- macOS:
Cmd+Shift+Dto open debug menu - Windows/Linux:
Ctrl+Shift+Dto open debug menu
- macOS:
The typical paths are:
- macOS:
~/Library/Application Support/com.pais.handy/ - Windows:
C:\Users\{username}\AppData\Roaming\com.pais.handy\ - Linux:
~/.config/com.pais.handy/
Step 2: Create Models Directory
Inside your app data directory, create a models folder if it doesn't already exist:
# macOS/Linux
mkdir -p ~/Library/Application\ Support/com.pais.handy/models
# Windows (PowerShell)
New-Item -ItemType Directory -Force -Path "$env:APPDATA\com.pais.handy\models"
Step 3: Download Model Files
Download the models you want from below
Whisper Models (single .bin files):
- Small (487 MB):
https://blob.handy.computer/ggml-small.bin - Medium (492 MB):
https://blob.handy.computer/whisper-medium-q4_1.bin - Turbo (1600 MB):
https://blob.handy.computer/ggml-large-v3-turbo.bin - Large (1100 MB):
https://blob.handy.computer/ggml-large-v3-q5_0.bin
Parakeet Models (compressed archives):
- V2 (473 MB):
https://blob.handy.computer/parakeet-v2-int8.tar.gz - V3 (478 MB):
https://blob.handy.computer/parakeet-v3-int8.tar.gz
Step 4: Install Models
For Whisper Models (.bin files):
Simply place the .bin file directly into the models directory:
{app_data_dir}/models/
├── ggml-small.bin
├── whisper-medium-q4_1.bin
├── ggml-large-v3-turbo.bin
└── ggml-large-v3-q5_0.bin
For Parakeet Models (.tar.gz archives):
- Extract the
.tar.gzfile - Place the extracted directory into the
modelsfolder - The directory must be named exactly as follows:
- Parakeet V2:
parakeet-tdt-0.6b-v2-int8 - Parakeet V3:
parakeet-tdt-0.6b-v3-int8
- Parakeet V2:
Final structure should look like:
{app_data_dir}/models/
├── parakeet-tdt-0.6b-v2-int8/ (directory with model files inside)
│ ├── (model files)
│ └── (config files)
└── parakeet-tdt-0.6b-v3-int8/ (directory with model files inside)
├── (model files)
└── (config files)
Important Notes:
- For Parakeet models, the extracted directory name must match exactly as shown above
- Do not rename the
.binfiles for Whisper models—use the exact filenames from the download URLs - After placing the files, restart Handy to detect the new models
Step 5: Verify Installation
- Restart Handy
- Open Settings → Models
- Your manually installed models should now appear as "Downloaded"
- Select the model you want to use and test transcription
Custom Whisper Models
Handy can auto-discover custom Whisper GGML models placed in the models directory. This is useful for users who want to use fine-tuned or community models not included in the default model list.
How to use:
- Obtain a Whisper model in GGML
.binformat (e.g., from Hugging Face) - Place the
.binfile in yourmodelsdirectory (see paths above) - Restart Handy to discover the new model
- The model will appear in the "Custom Models" section of the Models settings page
Important:
- Community models are user-provided and may not receive troubleshooting assistance
- The model must be a valid Whisper GGML format (
.binfile) - Model name is derived from the filename (e.g.,
my-custom-model.bin→ "My Custom Model")
How to Contribute
- Check existing issues at github.com/cjpais/Handy/issues
- Fork the repository and create a feature branch
- Test thoroughly on your target platform
- Submit a pull request with clear description of changes
- Join the discussion - reach out at contact@handy.computer
The goal is to create both a useful tool and a foundation for others to build upon—a well-patterned, simple codebase that serves the community.
Sponsors
Related Projects
- Handy CLI - The original Python command-line version
- handy.computer - Project website with demos and documentation
License
MIT License - see LICENSE file for details.
Acknowledgments
- Whisper by OpenAI for the speech recognition model
- whisper.cpp and ggml for amazing cross-platform whisper inference/acceleration
- Silero for great lightweight VAD
- Tauri team for the excellent Rust-based app framework
- Community contributors helping make Handy better
"Your search for the right speech-to-text tool can end here—not because Handy is perfect, but because you can make it perfect for you."