Orpheus-FASTAPI v1.3.0: Multilingual TTS with 24 Voices Across 8 Languages
This major update brings comprehensive multilingual support to Orpheus-FASTAPI: • Added 16 new voice actors across 7 languages (French, German, Korean, Hindi, Mandarin, Spanish, Italian) • Released 6 language-specific optimized models for superior pronunciation and fluency • Enhanced UI with dynamic language selector and voice filtering • Updated Docker Compose workflow to simplify language-specific model deployment • Reorganized code architecture with language-aware voice mappings All voices support the same emotion tags and audio quality features as the original English voices, maintaining compatibility with existing applications while expanding international usability.
This commit is contained in:
Lex-au
parent
f49e6b6a37
commit
5ead1e4b89
6 changed files with 248 additions and 27 deletions
69
README.md
69
README.md
|
|
@ -4,10 +4,30 @@
|
|||
|
||||
[](https://github.com/Lex-au/Orpheus-FastAPI/blob/main/LICENSE.txt)
|
||||
|
||||
High-performance Text-to-Speech server with OpenAI-compatible API, 8 voices, emotion tags, and modern web UI. Optimized for RTX GPUs.
|
||||
High-performance Text-to-Speech server with OpenAI-compatible API, multilingual support with 24 voices, emotion tags, and modern web UI. Optimized for RTX GPUs.
|
||||
|
||||
## Changelog
|
||||
|
||||
**v1.3.0** (2025-04-18)
|
||||
- 🌐 Added comprehensive multilingual support with 16 new voice actors across 7 languages
|
||||
- 🗣️ New voice actors include:
|
||||
- French: pierre, amelie, marie
|
||||
- German: jana, thomas, max
|
||||
- Korean: 유나, 준서
|
||||
- Hindi: ऋतिका
|
||||
- Mandarin: 长乐, 白芷
|
||||
- Spanish: javi, sergio, maria
|
||||
- Italian: pietro, giulia, carlo
|
||||
- 🔄 Enhanced UI with dynamic language selection and voice filtering
|
||||
- 🚀 Released language-specific optimized models:
|
||||
- [Italian & Spanish Model](https://huggingface.co/lex-au/Orpheus-3b-Italian_Spanish-FT-Q8_0.gguf)
|
||||
- [Korean Model](https://huggingface.co/lex-au/Orpheus-3b-Korean-FT-Q8_0.gguf)
|
||||
- [French Model](https://huggingface.co/lex-au/Orpheus-3b-French-FT-Q8_0.gguf)
|
||||
- [Hindi Model](https://huggingface.co/lex-au/Orpheus-3b-Hindi-FT-Q8_0.gguf)
|
||||
- [Mandarin Model](https://huggingface.co/lex-au/Orpheus-3b-Chinese-FT-Q8_0.gguf)
|
||||
- [German Model](https://huggingface.co/lex-au/Orpheus-3b-German-FT-Q8_0.gguf)
|
||||
- 🐳 Docker Compose users: To use a language-specific model, edit the `.env` file before installation and change `ORPHEUS_MODEL_NAME` to match the desired model repo ID (e.g., `Orpheus-3b-French-FT-Q8_0.gguf`)
|
||||
|
||||
**v1.2.0** (2025-04-12)
|
||||
- ❤️ Added optional Docker Compose support with GPU-enabled `llama.cpp` server and Orpheus-FastAPI integration
|
||||
- 🐳 Docker implementation contributed by [@richardr1126](https://github.com/richardr1126) – huge thanks for the clean setup and orchestration work!
|
||||
|
|
@ -49,7 +69,7 @@ Listen to sample outputs with different voices and emotions:
|
|||
- **OpenAI API Compatible**: Drop-in replacement for OpenAI's `/v1/audio/speech` endpoint
|
||||
- **Modern Web Interface**: Clean, responsive UI with waveform visualization
|
||||
- **High Performance**: Optimized for RTX GPUs with parallel processing
|
||||
- **Multiple Voices**: 8 different voice options with different characteristics
|
||||
- **Multilingual Support**: 24 different voices across 8 languages (English, French, German, Korean, Hindi, Mandarin, Spanish, Italian)
|
||||
- **Emotion Tags**: Support for laughter, sighs, and other emotional expressions
|
||||
- **Unlimited Audio Length**: Generate audio of any length through intelligent batching
|
||||
- **Smooth Transitions**: Crossfaded audio segments for seamless listening experience
|
||||
|
|
@ -88,14 +108,23 @@ Orpheus-FastAPI/
|
|||
The docker compose file orchestrates the Orpheus-FastAPI for audio and a llama.cpp inference server for the base model token generation. The GGUF model is downloaded with the model-init service.
|
||||
|
||||
```bash
|
||||
cp .env.example .env # Nothing needs to be changed, but the file is required
|
||||
cp .env.example .env # Create your .env file from the example
|
||||
copy .env.example .env # For Windows CMD
|
||||
```
|
||||
|
||||
For multilingual models, edit the `.env` file and change the model name:
|
||||
```
|
||||
# Change this line in .env to use a language-specific model
|
||||
ORPHEUS_MODEL_NAME=Orpheus-3b-French-FT-Q8_0.gguf # Example for French
|
||||
```
|
||||
|
||||
Then start the services:
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
The system will automatically download the specified model from Hugging Face before starting the service.
|
||||
|
||||
### FastAPI Service Native Installation
|
||||
|
||||
1. Clone the repository:
|
||||
|
|
@ -194,6 +223,7 @@ curl -X POST http://localhost:5005/speak \
|
|||
|
||||
### Available Voices
|
||||
|
||||
#### English
|
||||
- `tara`: Female, conversational, clear
|
||||
- `leah`: Female, warm, gentle
|
||||
- `jess`: Female, energetic, youthful
|
||||
|
|
@ -203,6 +233,37 @@ curl -X POST http://localhost:5005/speak \
|
|||
- `zac`: Male, enthusiastic, dynamic
|
||||
- `zoe`: Female, calm, soothing
|
||||
|
||||
#### French
|
||||
- `pierre`: Male, sophisticated
|
||||
- `amelie`: Female, elegant
|
||||
- `marie`: Female, spirited
|
||||
|
||||
#### German
|
||||
- `jana`: Female, clear
|
||||
- `thomas`: Male, authoritative
|
||||
- `max`: Male, energetic
|
||||
|
||||
#### Korean
|
||||
- `유나`: Female, melodic
|
||||
- `준서`: Male, confident
|
||||
|
||||
#### Hindi
|
||||
- `ऋतिका`: Female, expressive
|
||||
|
||||
#### Mandarin
|
||||
- `长乐`: Female, gentle
|
||||
- `白芷`: Female, clear
|
||||
|
||||
#### Spanish
|
||||
- `javi`: Male, warm
|
||||
- `sergio`: Male, professional
|
||||
- `maria`: Female, friendly
|
||||
|
||||
#### Italian
|
||||
- `pietro`: Male, passionate
|
||||
- `giulia`: Female, expressive
|
||||
- `carlo`: Male, refined
|
||||
|
||||
### Emotion Tags
|
||||
|
||||
You can insert emotion tags into your text to add expressiveness:
|
||||
|
|
@ -286,7 +347,7 @@ You can easily integrate this TTS solution with [OpenWebUI](https://github.com/o
|
|||
3. Change TTS from Web API to OpenAI
|
||||
4. Set APIBASE URL to your server address (e.g., `http://localhost:5005/v1`)
|
||||
5. API Key can be set to "not-needed"
|
||||
6. Set TTS Voice to one of the available voices: `tara`, `leah`, `jess`, `leo`, `dan`, `mia`, `zac`, or `zoe`
|
||||
6. Set TTS Voice to any of the available voices (e.g., `tara`, `pierre`, `jana`, `유나`, etc.)
|
||||
7. Set TTS Model to `tts-1`
|
||||
|
||||
### External Inference Server
|
||||
|
|
|
|||
25
app.py
25
app.py
|
|
@ -51,7 +51,7 @@ from fastapi.templating import Jinja2Templates
|
|||
from pydantic import BaseModel
|
||||
import json
|
||||
|
||||
from tts_engine import generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE
|
||||
from tts_engine import generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE, VOICE_TO_LANGUAGE, AVAILABLE_LANGUAGES
|
||||
|
||||
# Create FastAPI app
|
||||
app = FastAPI(
|
||||
|
|
@ -189,7 +189,12 @@ async def root(request: Request):
|
|||
"""Redirect to web UI"""
|
||||
return templates.TemplateResponse(
|
||||
"tts.html",
|
||||
{"request": request, "voices": AVAILABLE_VOICES}
|
||||
{
|
||||
"request": request,
|
||||
"voices": AVAILABLE_VOICES,
|
||||
"VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
|
||||
"AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
|
||||
}
|
||||
)
|
||||
|
||||
@app.get("/web/", response_class=HTMLResponse)
|
||||
|
|
@ -199,7 +204,13 @@ async def web_ui(request: Request):
|
|||
config = get_current_config()
|
||||
return templates.TemplateResponse(
|
||||
"tts.html",
|
||||
{"request": request, "voices": AVAILABLE_VOICES, "config": config}
|
||||
{
|
||||
"request": request,
|
||||
"voices": AVAILABLE_VOICES,
|
||||
"config": config,
|
||||
"VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
|
||||
"AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
|
||||
}
|
||||
)
|
||||
|
||||
@app.get("/get_config")
|
||||
|
|
@ -301,7 +312,9 @@ async def generate_from_web(
|
|||
{
|
||||
"request": request,
|
||||
"error": "Please enter some text.",
|
||||
"voices": AVAILABLE_VOICES
|
||||
"voices": AVAILABLE_VOICES,
|
||||
"VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
|
||||
"AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
|
||||
}
|
||||
)
|
||||
|
||||
|
|
@ -334,7 +347,9 @@ async def generate_from_web(
|
|||
"voice": voice,
|
||||
"output_file": output_path,
|
||||
"generation_time": generation_time,
|
||||
"voices": AVAILABLE_VOICES
|
||||
"voices": AVAILABLE_VOICES,
|
||||
"VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
|
||||
"AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
|
||||
}
|
||||
)
|
||||
|
||||
|
|
|
|||
BIN
docs/TaraLongGeneration.mp3
Normal file
BIN
docs/TaraLongGeneration.mp3
Normal file
Binary file not shown.
|
|
@ -155,25 +155,72 @@
|
|||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Language selection -->
|
||||
<div class="mb-6">
|
||||
<label class="block text-sm font-medium text-white mb-2">Language</label>
|
||||
<div class="flex flex-wrap gap-3">
|
||||
{% for language in AVAILABLE_LANGUAGES %}
|
||||
<div class="language-option {% if language == 'english' %}active{% endif %}" data-language="{{ language }}">
|
||||
<input type="radio" name="language" value="{{ language }}" class="hidden" {% if language == 'english' %}checked{% endif %}>
|
||||
<span class="px-3 py-1 rounded-full bg-dark-700 hover:bg-dark-600 text-white text-sm cursor-pointer">{{ language|capitalize }}</span>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Voice selection -->
|
||||
<div class="mb-6">
|
||||
<label class="block text-sm font-medium text-white mb-2">Voice</label>
|
||||
<div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
|
||||
{% for voice_option in voices %}
|
||||
<div class="voice-card {% if voice_option == DEFAULT_VOICE %}active{% endif %}" data-voice="{{ voice_option }}">
|
||||
<div class="voice-card {% if voice_option == DEFAULT_VOICE %}active{% endif %}"
|
||||
data-voice="{{ voice_option }}"
|
||||
data-language="{{ VOICE_TO_LANGUAGE[voice_option] }}"
|
||||
style="display: {% if VOICE_TO_LANGUAGE[voice_option] != 'english' %}none{% endif %}">
|
||||
<input type="radio" name="voice" value="{{ voice_option }}" class="hidden" {% if voice_option == DEFAULT_VOICE %}checked{% endif %}>
|
||||
<div class="flex items-center mb-2">
|
||||
<span class="font-medium text-white">{{ voice_option|capitalize }}</span>
|
||||
</div>
|
||||
<div class="text-xs text-dark-300">
|
||||
{% if voice_option == "tara" %}Female, conversational, clear
|
||||
{% elif voice_option == "leah" %}Female, warm, gentle
|
||||
{% elif voice_option == "jess" %}Female, energetic, youthful
|
||||
{% elif voice_option == "leo" %}Male, authoritative, deep
|
||||
{% elif voice_option == "dan" %}Male, friendly, casual
|
||||
{% elif voice_option == "mia" %}Female, professional, articulate
|
||||
{% elif voice_option == "zac" %}Male, enthusiastic, dynamic
|
||||
{% elif voice_option == "zoe" %}Female, calm, soothing
|
||||
{% if voice_option == "tara" %}Female, English, conversational, clear
|
||||
{% elif voice_option == "leah" %}Female, English, warm, gentle
|
||||
{% elif voice_option == "jess" %}Female, English, energetic, youthful
|
||||
{% elif voice_option == "leo" %}Male, English, authoritative, deep
|
||||
{% elif voice_option == "dan" %}Male, English, friendly, casual
|
||||
{% elif voice_option == "mia" %}Female, English, professional, articulate
|
||||
{% elif voice_option == "zac" %}Male, English, enthusiastic, dynamic
|
||||
{% elif voice_option == "zoe" %}Female, English, calm, soothing
|
||||
|
||||
<!-- French voices -->
|
||||
{% elif voice_option == "pierre" %}Male, French, sophisticated
|
||||
{% elif voice_option == "amelie" %}Female, French, elegant
|
||||
{% elif voice_option == "marie" %}Female, French, spirited
|
||||
|
||||
<!-- German voices -->
|
||||
{% elif voice_option == "jana" %}Female, German, clear
|
||||
{% elif voice_option == "thomas" %}Male, German, authoritative
|
||||
{% elif voice_option == "max" %}Male, German, energetic
|
||||
|
||||
<!-- Korean voices -->
|
||||
{% elif voice_option == "유나" %}Female, Korean, melodic
|
||||
{% elif voice_option == "준서" %}Male, Korean, confident
|
||||
|
||||
<!-- Hindi voice -->
|
||||
{% elif voice_option == "ऋतिका" %}Female, Hindi, expressive
|
||||
|
||||
<!-- Mandarin voices -->
|
||||
{% elif voice_option == "长乐" %}Female, Mandarin, gentle
|
||||
{% elif voice_option == "白芷" %}Female, Mandarin, clear
|
||||
|
||||
<!-- Spanish voices -->
|
||||
{% elif voice_option == "javi" %}Male, Spanish, warm
|
||||
{% elif voice_option == "sergio" %}Male, Spanish, professional
|
||||
{% elif voice_option == "maria" %}Female, Spanish, friendly
|
||||
|
||||
<!-- Italian voices -->
|
||||
{% elif voice_option == "pietro" %}Male, Italian, passionate
|
||||
{% elif voice_option == "giulia" %}Female, Italian, expressive
|
||||
{% elif voice_option == "carlo" %}Male, Italian, refined
|
||||
{% endif %}
|
||||
</div>
|
||||
</div>
|
||||
|
|
@ -404,20 +451,83 @@
|
|||
// Initialize char count
|
||||
charCount.textContent = textArea.value.length;
|
||||
|
||||
// Voice selection
|
||||
// Language selection
|
||||
const languageOptions = document.querySelectorAll('.language-option');
|
||||
const voiceCards = document.querySelectorAll('.voice-card');
|
||||
|
||||
languageOptions.forEach(option => {
|
||||
option.addEventListener('click', function() {
|
||||
// Unselect all language options
|
||||
languageOptions.forEach(o => o.classList.remove('active'));
|
||||
|
||||
// Select this language option
|
||||
this.classList.add('active');
|
||||
|
||||
// Check the radio button
|
||||
const radio = this.querySelector('input[type="radio"]');
|
||||
radio.checked = true;
|
||||
|
||||
// Get the selected language
|
||||
const selectedLanguage = this.getAttribute('data-language');
|
||||
|
||||
// Show/hide voice cards based on language
|
||||
let firstVisibleCard = null;
|
||||
|
||||
voiceCards.forEach(card => {
|
||||
const cardLanguage = card.getAttribute('data-language');
|
||||
if (cardLanguage === selectedLanguage) {
|
||||
card.style.display = '';
|
||||
if (!firstVisibleCard) {
|
||||
firstVisibleCard = card;
|
||||
}
|
||||
} else {
|
||||
card.style.display = 'none';
|
||||
// If this hidden card was selected, unselect it
|
||||
if (card.classList.contains('active')) {
|
||||
card.classList.remove('active');
|
||||
const cardRadio = card.querySelector('input[type="radio"]');
|
||||
if (cardRadio) {
|
||||
cardRadio.checked = false;
|
||||
}
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// If no voice card is selected for this language, select the first one
|
||||
if (firstVisibleCard && document.querySelectorAll(`.voice-card[data-language="${selectedLanguage}"].active:not([style*="display: none"])`).length === 0) {
|
||||
firstVisibleCard.classList.add('active');
|
||||
const firstVisibleRadio = firstVisibleCard.querySelector('input[type="radio"]');
|
||||
if (firstVisibleRadio) {
|
||||
firstVisibleRadio.checked = true;
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
// Voice card selection
|
||||
voiceCards.forEach(card => {
|
||||
card.addEventListener('click', function() {
|
||||
// Unselect all cards
|
||||
voiceCards.forEach(c => c.classList.remove('active'));
|
||||
// Only allow selection of cards that are visible (not display:none)
|
||||
if (this.style.display !== 'none') {
|
||||
// Unselect all cards with the same language
|
||||
const cardLanguage = this.getAttribute('data-language');
|
||||
document.querySelectorAll(`.voice-card[data-language="${cardLanguage}"]`).forEach(c => {
|
||||
c.classList.remove('active');
|
||||
const radio = c.querySelector('input[type="radio"]');
|
||||
if (radio) {
|
||||
radio.checked = false;
|
||||
}
|
||||
});
|
||||
|
||||
// Select this card
|
||||
this.classList.add('active');
|
||||
|
||||
// Check the radio button
|
||||
const radio = this.querySelector('input[type="radio"]');
|
||||
if (radio) {
|
||||
radio.checked = true;
|
||||
}
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
|
|
|
|||
|
|
@ -11,5 +11,7 @@ from .inference import (
|
|||
generate_speech_from_api,
|
||||
AVAILABLE_VOICES,
|
||||
DEFAULT_VOICE,
|
||||
VOICE_TO_LANGUAGE,
|
||||
AVAILABLE_LANGUAGES,
|
||||
list_available_voices
|
||||
)
|
||||
|
|
|
|||
|
|
@ -136,10 +136,43 @@ if not IS_RELOADER:
|
|||
# Parallel processing settings
|
||||
NUM_WORKERS = 4 if HIGH_END_GPU else 2
|
||||
|
||||
# Available voices based on the Orpheus-TTS repository
|
||||
AVAILABLE_VOICES = ["tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"]
|
||||
# Define voices by language
|
||||
ENGLISH_VOICES = ["tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"]
|
||||
FRENCH_VOICES = ["pierre", "amelie", "marie"]
|
||||
GERMAN_VOICES = ["jana", "thomas", "max"]
|
||||
KOREAN_VOICES = ["유나", "준서"]
|
||||
HINDI_VOICES = ["ऋतिका"]
|
||||
MANDARIN_VOICES = ["长乐", "白芷"]
|
||||
SPANISH_VOICES = ["javi", "sergio", "maria"]
|
||||
ITALIAN_VOICES = ["pietro", "giulia", "carlo"]
|
||||
|
||||
# Combined list for API compatibility
|
||||
AVAILABLE_VOICES = (
|
||||
ENGLISH_VOICES +
|
||||
FRENCH_VOICES +
|
||||
GERMAN_VOICES +
|
||||
KOREAN_VOICES +
|
||||
HINDI_VOICES +
|
||||
MANDARIN_VOICES +
|
||||
SPANISH_VOICES +
|
||||
ITALIAN_VOICES
|
||||
)
|
||||
DEFAULT_VOICE = "tara" # Best voice according to documentation
|
||||
|
||||
# Map voices to languages for the UI
|
||||
VOICE_TO_LANGUAGE = {}
|
||||
VOICE_TO_LANGUAGE.update({voice: "english" for voice in ENGLISH_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "french" for voice in FRENCH_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "german" for voice in GERMAN_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "korean" for voice in KOREAN_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "hindi" for voice in HINDI_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "mandarin" for voice in MANDARIN_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "spanish" for voice in SPANISH_VOICES})
|
||||
VOICE_TO_LANGUAGE.update({voice: "italian" for voice in ITALIAN_VOICES})
|
||||
|
||||
# Languages list for the UI
|
||||
AVAILABLE_LANGUAGES = ["english", "french", "german", "korean", "hindi", "mandarin", "spanish", "italian"]
|
||||
|
||||
# Import the unified token handling from speechpipe
|
||||
from .speechpipe import turn_token_into_id, CUSTOM_TOKEN_PREFIX
|
||||
|
||||
|
|
|
|||
Loading…
Reference in a new issue