171 lines
6.6 KiB
Markdown
171 lines
6.6 KiB
Markdown
# Handy
|
|
|
|
[]([https://discord.gg/your-invite-link](https://discord.gg/WVBeWsNXK4))
|
|
|
|
**A free, open source, and extensible speech-to-text application that works completely offline.**
|
|
|
|
Handy is a cross-platform desktop application built with Tauri (Rust + React/TypeScript) that provides simple, privacy-focused speech transcription. Press a shortcut, speak, and have your words appear in any text field—all without sending your voice to the cloud.
|
|
|
|
## Why Handy?
|
|
|
|
Handy was created to fill the gap for a truly open source, extensible speech-to-text tool. As stated on [handy.computer](https://handy.computer):
|
|
|
|
- **Free**: Accessibility tooling belongs in everyone's hands, not behind a paywall
|
|
- **Open Source**: Together we can build further. Extend Handy for yourself and contribute to something bigger
|
|
- **Private**: Your voice stays on your computer. Get transcriptions without sending audio to the cloud
|
|
- **Simple**: One tool, one job. Transcribe what you say and put it into a text box
|
|
|
|
Handy isn't trying to be the best speech-to-text app—it's trying to be the most forkable one.
|
|
|
|
## How It Works
|
|
|
|
1. **Press** a configurable keyboard shortcut to start/stop recording (or use push-to-talk mode)
|
|
2. **Speak** your words while the shortcut is active
|
|
3. **Release** and Handy processes your speech using Whisper
|
|
4. **Get** your transcribed text pasted directly into whatever app you're using
|
|
|
|
The process is entirely local:
|
|
- Silence is filtered using VAD (Voice Activity Detection) with Silero
|
|
- Transcription uses Whisper Small model with GPU acceleration when available
|
|
- Works on Windows, macOS, and Linux
|
|
|
|
## Quick Start
|
|
|
|
### Installation
|
|
|
|
1. Download the latest release from the [releases page](https://github.com/cjpais/Handy/releases) or the [website](https://handy.computer)
|
|
2. Install the application following platform-specific instructions
|
|
3. Launch Handy and grant necessary system permissions (microphone, accessibility)
|
|
4. Configure your preferred keyboard shortcuts in Settings
|
|
5. Start transcribing!
|
|
|
|
### Development Setup
|
|
|
|
**Prerequisites:**
|
|
- [Rust](https://rustup.rs/) (latest stable)
|
|
- [Bun](https://bun.sh/) package manager
|
|
- Platform-specific requirements:
|
|
- **macOS**: Xcode Command Line Tools
|
|
- **Windows**: Microsoft C++ Build Tools
|
|
- **Linux**: Build essentials, ALSA development libraries
|
|
|
|
**Getting Started:**
|
|
|
|
```bash
|
|
# Clone the repository
|
|
git clone git@github.com:cjpais/Handy.git
|
|
cd Handy
|
|
|
|
# Install dependencies
|
|
bun install
|
|
|
|
# Run in development mode
|
|
bun run tauri dev
|
|
# if it fails with cmake error on MacOS, try
|
|
CMAKE_POLICY_VERSION_MINIMUM=3.5 bun run tauri dev
|
|
|
|
# Build for production
|
|
bun run tauri build
|
|
```
|
|
|
|
**Model Files Setup:**
|
|
|
|
For development, you need to download the required model files:
|
|
|
|
1. Create the models directory inside the resources folder:
|
|
```bash
|
|
mkdir -p src-tauri/resources/models
|
|
```
|
|
|
|
2. Download the required VAD model for development:
|
|
```bash
|
|
# Download Silero VAD model (required for voice activity detection)
|
|
curl -o src-tauri/resources/models/silero_vad_v4.onnx https://blob.handy.computer/silero_vad_v4.onnx
|
|
```
|
|
|
|
**Note:** Whisper models are no longer bundled with the app. Users will download their preferred model (Small, Medium, Turbo, or Large) from within the app on first run.
|
|
|
|
**Whisper Models:**
|
|
|
|
The app now supports dynamic model downloading and switching:
|
|
- **Small**: Fast, good for most use cases
|
|
- **Medium**: Better accuracy, balanced performance
|
|
- **Turbo**: Optimized large model with improved speed
|
|
- **Large**: Highest accuracy, slower processing
|
|
|
|
Users can download and switch between models directly from the app's settings interface. No models are bundled with the app, reducing the initial download size.
|
|
|
|
## Architecture
|
|
|
|
Handy is built as a Tauri application combining:
|
|
|
|
- **Frontend**: React + TypeScript with Tailwind CSS for the settings UI
|
|
- **Backend**: Rust for system integration, audio processing, and ML inference
|
|
- **Core Libraries**:
|
|
- `whisper-rs`: Local speech recognition with Whisper models
|
|
- `cpal`: Cross-platform audio I/O
|
|
- `vad-rs`: Voice Activity Detection
|
|
- `rdev`: Global keyboard shortcuts and system events
|
|
- `rubato`: Audio resampling
|
|
|
|
## Known Issues & Current Limitations
|
|
|
|
This project is actively being developed and has some [known issues](https://github.com/cjpais/Handy/issues). We believe in transparency about the current state:
|
|
|
|
### Platform Support
|
|
- **Apple Silicon Macs**
|
|
- **x64 Windows**
|
|
- **x64 Linux**
|
|
|
|
### Active Issues
|
|
- Paste functionality occasionally produces just 'v' instead of full text on macOS
|
|
- VAD filter sometimes includes trailing "thank you" in transcriptions
|
|
- Transcription end-cutting due to potential threading issues
|
|
- Microphone remains active for optimal latency (design choice under discussion)
|
|
|
|
## Contributing
|
|
|
|
We're actively seeking contributors! Priority areas include:
|
|
|
|
### High Priority
|
|
1. **Cross-platform support** - Windows and Linux compatibility
|
|
2. **Code quality improvements** - Better error handling, architecture refinements
|
|
3. **Bug fixes** - Address the known issues listed above
|
|
4. **Performance optimization** - Reduce latency, improve resource usage
|
|
|
|
### Feature Requests
|
|
- Configurable microphone selection
|
|
- Multiple STT model options (beyond Whisper Small)
|
|
- Modifier-only key bindings
|
|
- Enhanced VAD configuration
|
|
|
|
### How to Contribute
|
|
|
|
1. **Check existing issues** at [github.com/cjpais/Handy/issues](https://github.com/cjpais/Handy/issues)
|
|
2. **Fork the repository** and create a feature branch
|
|
3. **Test thoroughly** on your target platform
|
|
4. **Submit a pull request** with clear description of changes
|
|
5. **Join the discussion** - reach out at [contact@handy.computer](mailto:contact@handy.computer)
|
|
|
|
The goal is to create both a useful tool and a foundation for others to build upon—a well-patterned, simple codebase that serves the community.
|
|
|
|
## Related Projects
|
|
|
|
- **[Handy CLI](https://github.com/cjpais/handy-cli)** - The original Python command-line version
|
|
- **[handy.computer](https://handy.computer)** - Project website with demos and documentation
|
|
|
|
## License
|
|
|
|
MIT License - see [LICENSE](LICENSE) file for details.
|
|
|
|
## Acknowledgments
|
|
|
|
- **Whisper** by OpenAI for the speech recognition model
|
|
- **whisper.cpp and ggml** for amazing cross-platform whisper inference/acceleration
|
|
- **Silero** for great lightweight VAD
|
|
- **Tauri** team for the excellent Rust-based app framework
|
|
- **Community contributors** helping make Handy better
|
|
|
|
---
|
|
|
|
*"Your search for the right speech-to-text tool can end here—not because Handy is perfect, but because you can make it perfect for you."*
|