diff --git a/README.md b/README.md index b50f4d9..6dc1fc8 100644 --- a/README.md +++ b/README.md @@ -4,10 +4,30 @@ [![GitHub](https://img.shields.io/github/license/Lex-au/Orpheus-FastAPI)](https://github.com/Lex-au/Orpheus-FastAPI/blob/main/LICENSE.txt) -High-performance Text-to-Speech server with OpenAI-compatible API, 8 voices, emotion tags, and modern web UI. Optimized for RTX GPUs. +High-performance Text-to-Speech server with OpenAI-compatible API, multilingual support with 24 voices, emotion tags, and modern web UI. Optimized for RTX GPUs. ## Changelog +**v1.3.0** (2025-04-18) +- 🌐 Added comprehensive multilingual support with 16 new voice actors across 7 languages +- 🗣️ New voice actors include: + - French: pierre, amelie, marie + - German: jana, thomas, max + - Korean: 유나, 준서 + - Hindi: ऋतिका + - Mandarin: 长乐, 白芷 + - Spanish: javi, sergio, maria + - Italian: pietro, giulia, carlo +- 🔄 Enhanced UI with dynamic language selection and voice filtering +- 🚀 Released language-specific optimized models: + - [Italian & Spanish Model](https://huggingface.co/lex-au/Orpheus-3b-Italian_Spanish-FT-Q8_0.gguf) + - [Korean Model](https://huggingface.co/lex-au/Orpheus-3b-Korean-FT-Q8_0.gguf) + - [French Model](https://huggingface.co/lex-au/Orpheus-3b-French-FT-Q8_0.gguf) + - [Hindi Model](https://huggingface.co/lex-au/Orpheus-3b-Hindi-FT-Q8_0.gguf) + - [Mandarin Model](https://huggingface.co/lex-au/Orpheus-3b-Chinese-FT-Q8_0.gguf) + - [German Model](https://huggingface.co/lex-au/Orpheus-3b-German-FT-Q8_0.gguf) +- 🐳 Docker Compose users: To use a language-specific model, edit the `.env` file before installation and change `ORPHEUS_MODEL_NAME` to match the desired model repo ID (e.g., `Orpheus-3b-French-FT-Q8_0.gguf`) + **v1.2.0** (2025-04-12) - ❤️ Added optional Docker Compose support with GPU-enabled `llama.cpp` server and Orpheus-FastAPI integration - 🐳 Docker implementation contributed by [@richardr1126](https://github.com/richardr1126) – huge thanks for the clean setup and orchestration work! @@ -49,7 +69,7 @@ Listen to sample outputs with different voices and emotions: - **OpenAI API Compatible**: Drop-in replacement for OpenAI's `/v1/audio/speech` endpoint - **Modern Web Interface**: Clean, responsive UI with waveform visualization - **High Performance**: Optimized for RTX GPUs with parallel processing -- **Multiple Voices**: 8 different voice options with different characteristics +- **Multilingual Support**: 24 different voices across 8 languages (English, French, German, Korean, Hindi, Mandarin, Spanish, Italian) - **Emotion Tags**: Support for laughter, sighs, and other emotional expressions - **Unlimited Audio Length**: Generate audio of any length through intelligent batching - **Smooth Transitions**: Crossfaded audio segments for seamless listening experience @@ -88,14 +108,23 @@ Orpheus-FastAPI/ The docker compose file orchestrates the Orpheus-FastAPI for audio and a llama.cpp inference server for the base model token generation. The GGUF model is downloaded with the model-init service. ```bash -cp .env.example .env # Nothing needs to be changed, but the file is required +cp .env.example .env # Create your .env file from the example copy .env.example .env # For Windows CMD ``` +For multilingual models, edit the `.env` file and change the model name: +``` +# Change this line in .env to use a language-specific model +ORPHEUS_MODEL_NAME=Orpheus-3b-French-FT-Q8_0.gguf # Example for French +``` + +Then start the services: ```bash docker compose up --build ``` +The system will automatically download the specified model from Hugging Face before starting the service. + ### FastAPI Service Native Installation 1. Clone the repository: @@ -194,6 +223,7 @@ curl -X POST http://localhost:5005/speak \ ### Available Voices +#### English - `tara`: Female, conversational, clear - `leah`: Female, warm, gentle - `jess`: Female, energetic, youthful @@ -203,6 +233,37 @@ curl -X POST http://localhost:5005/speak \ - `zac`: Male, enthusiastic, dynamic - `zoe`: Female, calm, soothing +#### French +- `pierre`: Male, sophisticated +- `amelie`: Female, elegant +- `marie`: Female, spirited + +#### German +- `jana`: Female, clear +- `thomas`: Male, authoritative +- `max`: Male, energetic + +#### Korean +- `유나`: Female, melodic +- `준서`: Male, confident + +#### Hindi +- `ऋतिका`: Female, expressive + +#### Mandarin +- `长乐`: Female, gentle +- `白芷`: Female, clear + +#### Spanish +- `javi`: Male, warm +- `sergio`: Male, professional +- `maria`: Female, friendly + +#### Italian +- `pietro`: Male, passionate +- `giulia`: Female, expressive +- `carlo`: Male, refined + ### Emotion Tags You can insert emotion tags into your text to add expressiveness: @@ -286,7 +347,7 @@ You can easily integrate this TTS solution with [OpenWebUI](https://github.com/o 3. Change TTS from Web API to OpenAI 4. Set APIBASE URL to your server address (e.g., `http://localhost:5005/v1`) 5. API Key can be set to "not-needed" -6. Set TTS Voice to one of the available voices: `tara`, `leah`, `jess`, `leo`, `dan`, `mia`, `zac`, or `zoe` +6. Set TTS Voice to any of the available voices (e.g., `tara`, `pierre`, `jana`, `유나`, etc.) 7. Set TTS Model to `tts-1` ### External Inference Server diff --git a/app.py b/app.py index 3bb7c61..1ca2108 100644 --- a/app.py +++ b/app.py @@ -51,7 +51,7 @@ from fastapi.templating import Jinja2Templates from pydantic import BaseModel import json -from tts_engine import generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE +from tts_engine import generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE, VOICE_TO_LANGUAGE, AVAILABLE_LANGUAGES # Create FastAPI app app = FastAPI( @@ -189,7 +189,12 @@ async def root(request: Request): """Redirect to web UI""" return templates.TemplateResponse( "tts.html", - {"request": request, "voices": AVAILABLE_VOICES} + { + "request": request, + "voices": AVAILABLE_VOICES, + "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE, + "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES + } ) @app.get("/web/", response_class=HTMLResponse) @@ -199,7 +204,13 @@ async def web_ui(request: Request): config = get_current_config() return templates.TemplateResponse( "tts.html", - {"request": request, "voices": AVAILABLE_VOICES, "config": config} + { + "request": request, + "voices": AVAILABLE_VOICES, + "config": config, + "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE, + "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES + } ) @app.get("/get_config") @@ -301,7 +312,9 @@ async def generate_from_web( { "request": request, "error": "Please enter some text.", - "voices": AVAILABLE_VOICES + "voices": AVAILABLE_VOICES, + "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE, + "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES } ) @@ -334,7 +347,9 @@ async def generate_from_web( "voice": voice, "output_file": output_path, "generation_time": generation_time, - "voices": AVAILABLE_VOICES + "voices": AVAILABLE_VOICES, + "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE, + "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES } ) diff --git a/docs/TaraLongGeneration.mp3 b/docs/TaraLongGeneration.mp3 new file mode 100644 index 0000000..ffd56b8 Binary files /dev/null and b/docs/TaraLongGeneration.mp3 differ diff --git a/templates/tts.html b/templates/tts.html index 3a5c6bf..3cf715c 100644 --- a/templates/tts.html +++ b/templates/tts.html @@ -155,25 +155,72 @@ + +
+ +
+ {% for language in AVAILABLE_LANGUAGES %} +
+ + {{ language|capitalize }} +
+ {% endfor %} +
+
+
{% for voice_option in voices %} -
+
{{ voice_option|capitalize }}
-
- {% if voice_option == "tara" %}Female, conversational, clear - {% elif voice_option == "leah" %}Female, warm, gentle - {% elif voice_option == "jess" %}Female, energetic, youthful - {% elif voice_option == "leo" %}Male, authoritative, deep - {% elif voice_option == "dan" %}Male, friendly, casual - {% elif voice_option == "mia" %}Female, professional, articulate - {% elif voice_option == "zac" %}Male, enthusiastic, dynamic - {% elif voice_option == "zoe" %}Female, calm, soothing +
+ {% if voice_option == "tara" %}Female, English, conversational, clear + {% elif voice_option == "leah" %}Female, English, warm, gentle + {% elif voice_option == "jess" %}Female, English, energetic, youthful + {% elif voice_option == "leo" %}Male, English, authoritative, deep + {% elif voice_option == "dan" %}Male, English, friendly, casual + {% elif voice_option == "mia" %}Female, English, professional, articulate + {% elif voice_option == "zac" %}Male, English, enthusiastic, dynamic + {% elif voice_option == "zoe" %}Female, English, calm, soothing + + + {% elif voice_option == "pierre" %}Male, French, sophisticated + {% elif voice_option == "amelie" %}Female, French, elegant + {% elif voice_option == "marie" %}Female, French, spirited + + + {% elif voice_option == "jana" %}Female, German, clear + {% elif voice_option == "thomas" %}Male, German, authoritative + {% elif voice_option == "max" %}Male, German, energetic + + + {% elif voice_option == "유나" %}Female, Korean, melodic + {% elif voice_option == "준서" %}Male, Korean, confident + + + {% elif voice_option == "ऋतिका" %}Female, Hindi, expressive + + + {% elif voice_option == "长乐" %}Female, Mandarin, gentle + {% elif voice_option == "白芷" %}Female, Mandarin, clear + + + {% elif voice_option == "javi" %}Male, Spanish, warm + {% elif voice_option == "sergio" %}Male, Spanish, professional + {% elif voice_option == "maria" %}Female, Spanish, friendly + + + {% elif voice_option == "pietro" %}Male, Italian, passionate + {% elif voice_option == "giulia" %}Female, Italian, expressive + {% elif voice_option == "carlo" %}Male, Italian, refined {% endif %}
@@ -404,20 +451,83 @@ // Initialize char count charCount.textContent = textArea.value.length; - // Voice selection + // Language selection + const languageOptions = document.querySelectorAll('.language-option'); const voiceCards = document.querySelectorAll('.voice-card'); - voiceCards.forEach(card => { - card.addEventListener('click', function() { - // Unselect all cards - voiceCards.forEach(c => c.classList.remove('active')); + languageOptions.forEach(option => { + option.addEventListener('click', function() { + // Unselect all language options + languageOptions.forEach(o => o.classList.remove('active')); - // Select this card + // Select this language option this.classList.add('active'); // Check the radio button const radio = this.querySelector('input[type="radio"]'); radio.checked = true; + + // Get the selected language + const selectedLanguage = this.getAttribute('data-language'); + + // Show/hide voice cards based on language + let firstVisibleCard = null; + + voiceCards.forEach(card => { + const cardLanguage = card.getAttribute('data-language'); + if (cardLanguage === selectedLanguage) { + card.style.display = ''; + if (!firstVisibleCard) { + firstVisibleCard = card; + } + } else { + card.style.display = 'none'; + // If this hidden card was selected, unselect it + if (card.classList.contains('active')) { + card.classList.remove('active'); + const cardRadio = card.querySelector('input[type="radio"]'); + if (cardRadio) { + cardRadio.checked = false; + } + } + } + }); + + // If no voice card is selected for this language, select the first one + if (firstVisibleCard && document.querySelectorAll(`.voice-card[data-language="${selectedLanguage}"].active:not([style*="display: none"])`).length === 0) { + firstVisibleCard.classList.add('active'); + const firstVisibleRadio = firstVisibleCard.querySelector('input[type="radio"]'); + if (firstVisibleRadio) { + firstVisibleRadio.checked = true; + } + } + }); + }); + + // Voice card selection + voiceCards.forEach(card => { + card.addEventListener('click', function() { + // Only allow selection of cards that are visible (not display:none) + if (this.style.display !== 'none') { + // Unselect all cards with the same language + const cardLanguage = this.getAttribute('data-language'); + document.querySelectorAll(`.voice-card[data-language="${cardLanguage}"]`).forEach(c => { + c.classList.remove('active'); + const radio = c.querySelector('input[type="radio"]'); + if (radio) { + radio.checked = false; + } + }); + + // Select this card + this.classList.add('active'); + + // Check the radio button + const radio = this.querySelector('input[type="radio"]'); + if (radio) { + radio.checked = true; + } + } }); }); diff --git a/tts_engine/__init__.py b/tts_engine/__init__.py index 7252e9c..88bc054 100644 --- a/tts_engine/__init__.py +++ b/tts_engine/__init__.py @@ -11,5 +11,7 @@ from .inference import ( generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE, + VOICE_TO_LANGUAGE, + AVAILABLE_LANGUAGES, list_available_voices ) diff --git a/tts_engine/inference.py b/tts_engine/inference.py index 7e1e501..b11817c 100644 --- a/tts_engine/inference.py +++ b/tts_engine/inference.py @@ -136,10 +136,43 @@ if not IS_RELOADER: # Parallel processing settings NUM_WORKERS = 4 if HIGH_END_GPU else 2 -# Available voices based on the Orpheus-TTS repository -AVAILABLE_VOICES = ["tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"] +# Define voices by language +ENGLISH_VOICES = ["tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"] +FRENCH_VOICES = ["pierre", "amelie", "marie"] +GERMAN_VOICES = ["jana", "thomas", "max"] +KOREAN_VOICES = ["유나", "준서"] +HINDI_VOICES = ["ऋतिका"] +MANDARIN_VOICES = ["长乐", "白芷"] +SPANISH_VOICES = ["javi", "sergio", "maria"] +ITALIAN_VOICES = ["pietro", "giulia", "carlo"] + +# Combined list for API compatibility +AVAILABLE_VOICES = ( + ENGLISH_VOICES + + FRENCH_VOICES + + GERMAN_VOICES + + KOREAN_VOICES + + HINDI_VOICES + + MANDARIN_VOICES + + SPANISH_VOICES + + ITALIAN_VOICES +) DEFAULT_VOICE = "tara" # Best voice according to documentation +# Map voices to languages for the UI +VOICE_TO_LANGUAGE = {} +VOICE_TO_LANGUAGE.update({voice: "english" for voice in ENGLISH_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "french" for voice in FRENCH_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "german" for voice in GERMAN_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "korean" for voice in KOREAN_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "hindi" for voice in HINDI_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "mandarin" for voice in MANDARIN_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "spanish" for voice in SPANISH_VOICES}) +VOICE_TO_LANGUAGE.update({voice: "italian" for voice in ITALIAN_VOICES}) + +# Languages list for the UI +AVAILABLE_LANGUAGES = ["english", "french", "german", "korean", "hindi", "mandarin", "spanish", "italian"] + # Import the unified token handling from speechpipe from .speechpipe import turn_token_into_id, CUSTOM_TOKEN_PREFIX