Orpheus-FASTAPI v1.3.0: Multilingual TTS with 24 Voices Across 8 Languages

This major update brings comprehensive multilingual support to Orpheus-FASTAPI:

• Added 16 new voice actors across 7 languages (French, German, Korean, Hindi, Mandarin, Spanish, Italian)
• Released 6 language-specific optimized models for superior pronunciation and fluency
• Enhanced UI with dynamic language selector and voice filtering
• Updated Docker Compose workflow to simplify language-specific model deployment
• Reorganized code architecture with language-aware voice mappings

All voices support the same emotion tags and audio quality features as the original English voices, maintaining compatibility with existing applications while expanding international usability.

README.md

@@ -4,10 +4,30 @@
 
 [![GitHub](https://img.shields.io/github/license/Lex-au/Orpheus-FastAPI)](https://github.com/Lex-au/Orpheus-FastAPI/blob/main/LICENSE.txt)
 
-High-performance Text-to-Speech server with OpenAI-compatible API, 8 voices, emotion tags, and modern web UI. Optimized for RTX GPUs.
+High-performance Text-to-Speech server with OpenAI-compatible API, multilingual support with 24 voices, emotion tags, and modern web UI. Optimized for RTX GPUs.
 
 ## Changelog
 
+**v1.3.0** (2025-04-18)
+- 🌐 Added comprehensive multilingual support with 16 new voice actors across 7 languages
+- 🗣️ New voice actors include:
+  - French: pierre, amelie, marie
+  - German: jana, thomas, max
+  - Korean: 유나, 준서
+  - Hindi: ऋतिका
+  - Mandarin: 长乐, 白芷
+  - Spanish: javi, sergio, maria
+  - Italian: pietro, giulia, carlo
+- 🔄 Enhanced UI with dynamic language selection and voice filtering
+- 🚀 Released language-specific optimized models:
+  - [Italian & Spanish Model](https://huggingface.co/lex-au/Orpheus-3b-Italian_Spanish-FT-Q8_0.gguf)
+  - [Korean Model](https://huggingface.co/lex-au/Orpheus-3b-Korean-FT-Q8_0.gguf)
+  - [French Model](https://huggingface.co/lex-au/Orpheus-3b-French-FT-Q8_0.gguf)
+  - [Hindi Model](https://huggingface.co/lex-au/Orpheus-3b-Hindi-FT-Q8_0.gguf)
+  - [Mandarin Model](https://huggingface.co/lex-au/Orpheus-3b-Chinese-FT-Q8_0.gguf)
+  - [German Model](https://huggingface.co/lex-au/Orpheus-3b-German-FT-Q8_0.gguf)
+- 🐳 Docker Compose users: To use a language-specific model, edit the `.env` file before installation and change `ORPHEUS_MODEL_NAME` to match the desired model repo ID (e.g., `Orpheus-3b-French-FT-Q8_0.gguf`)
+
 **v1.2.0** (2025-04-12)
 - ❤️ Added optional Docker Compose support with GPU-enabled `llama.cpp` server and Orpheus-FastAPI integration  
 - 🐳 Docker implementation contributed by [@richardr1126](https://github.com/richardr1126) – huge thanks for the clean setup and orchestration work!  
@@ -49,7 +69,7 @@ Listen to sample outputs with different voices and emotions:
 - **OpenAI API Compatible**: Drop-in replacement for OpenAI's `/v1/audio/speech` endpoint
 - **Modern Web Interface**: Clean, responsive UI with waveform visualization
 - **High Performance**: Optimized for RTX GPUs with parallel processing
-- **Multiple Voices**: 8 different voice options with different characteristics
+- **Multilingual Support**: 24 different voices across 8 languages (English, French, German, Korean, Hindi, Mandarin, Spanish, Italian)
 - **Emotion Tags**: Support for laughter, sighs, and other emotional expressions
 - **Unlimited Audio Length**: Generate audio of any length through intelligent batching
 - **Smooth Transitions**: Crossfaded audio segments for seamless listening experience
@@ -88,14 +108,23 @@ Orpheus-FastAPI/
 The docker compose file orchestrates the Orpheus-FastAPI for audio and a llama.cpp inference server for the base model token generation. The GGUF model is downloaded with the model-init service.
 
 ```bash
-cp .env.example .env # Nothing needs to be changed, but the file is required
+cp .env.example .env # Create your .env file from the example
 copy .env.example .env # For Windows CMD
 ```
 
+For multilingual models, edit the `.env` file and change the model name:
+```
+# Change this line in .env to use a language-specific model
+ORPHEUS_MODEL_NAME=Orpheus-3b-French-FT-Q8_0.gguf  # Example for French
+```
+
+Then start the services:
 ```bash
 docker compose up --build
 ```
 
+The system will automatically download the specified model from Hugging Face before starting the service.
+
 ### FastAPI Service Native Installation
 
 1. Clone the repository:
@@ -194,6 +223,7 @@ curl -X POST http://localhost:5005/speak \
 
 ### Available Voices
 
+#### English
 - `tara`: Female, conversational, clear
 - `leah`: Female, warm, gentle
 - `jess`: Female, energetic, youthful
@@ -203,6 +233,37 @@ curl -X POST http://localhost:5005/speak \
 - `zac`: Male, enthusiastic, dynamic
 - `zoe`: Female, calm, soothing
 
+#### French
+- `pierre`: Male, sophisticated
+- `amelie`: Female, elegant
+- `marie`: Female, spirited
+
+#### German
+- `jana`: Female, clear
+- `thomas`: Male, authoritative
+- `max`: Male, energetic
+
+#### Korean
+- `유나`: Female, melodic
+- `준서`: Male, confident
+
+#### Hindi
+- `ऋतिका`: Female, expressive
+
+#### Mandarin
+- `长乐`: Female, gentle
+- `白芷`: Female, clear
+
+#### Spanish
+- `javi`: Male, warm
+- `sergio`: Male, professional
+- `maria`: Female, friendly
+
+#### Italian
+- `pietro`: Male, passionate
+- `giulia`: Female, expressive
+- `carlo`: Male, refined
+
 ### Emotion Tags
 
 You can insert emotion tags into your text to add expressiveness:
@@ -286,7 +347,7 @@ You can easily integrate this TTS solution with [OpenWebUI](https://github.com/o
 3. Change TTS from Web API to OpenAI
 4. Set APIBASE URL to your server address (e.g., `http://localhost:5005/v1`)
 5. API Key can be set to "not-needed"
-6. Set TTS Voice to one of the available voices: `tara`, `leah`, `jess`, `leo`, `dan`, `mia`, `zac`, or `zoe`
+6. Set TTS Voice to any of the available voices (e.g., `tara`, `pierre`, `jana`, `유나`, etc.)
 7. Set TTS Model to `tts-1`
 
 ### External Inference Server

app.py

@@ -51,7 +51,7 @@ from fastapi.templating import Jinja2Templates
 from pydantic import BaseModel
 import json
 
-from tts_engine import generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE
+from tts_engine import generate_speech_from_api, AVAILABLE_VOICES, DEFAULT_VOICE, VOICE_TO_LANGUAGE, AVAILABLE_LANGUAGES
 
 # Create FastAPI app
 app = FastAPI(
@@ -189,7 +189,12 @@ async def root(request: Request):
     """Redirect to web UI"""
     return templates.TemplateResponse(
         "tts.html",
-        {"request": request, "voices": AVAILABLE_VOICES}
+        {
+            "request": request, 
+            "voices": AVAILABLE_VOICES,
+            "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
+            "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
+        }
     )
 
 @app.get("/web/", response_class=HTMLResponse)
@@ -199,7 +204,13 @@ async def web_ui(request: Request):
     config = get_current_config()
     return templates.TemplateResponse(
         "tts.html",
-        {"request": request, "voices": AVAILABLE_VOICES, "config": config}
+        {
+            "request": request, 
+            "voices": AVAILABLE_VOICES, 
+            "config": config,
+            "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
+            "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
+        }
     )
 
 @app.get("/get_config")
@@ -301,7 +312,9 @@ async def generate_from_web(
             {
                 "request": request,
                 "error": "Please enter some text.",
-                "voices": AVAILABLE_VOICES
+                "voices": AVAILABLE_VOICES,
+                "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
+                "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
             }
         )
     
@@ -334,7 +347,9 @@ async def generate_from_web(
             "voice": voice,
             "output_file": output_path,
             "generation_time": generation_time,
-            "voices": AVAILABLE_VOICES
+            "voices": AVAILABLE_VOICES,
+            "VOICE_TO_LANGUAGE": VOICE_TO_LANGUAGE,
+            "AVAILABLE_LANGUAGES": AVAILABLE_LANGUAGES
         }
     )
 

templates/tts.html

@@ -155,25 +155,72 @@
                 </div>
               </div>
 
+              <!-- Language selection -->
+              <div class="mb-6">
+                <label class="block text-sm font-medium text-white mb-2">Language</label>
+                <div class="flex flex-wrap gap-3">
+                  {% for language in AVAILABLE_LANGUAGES %}
+                  <div class="language-option {% if language == 'english' %}active{% endif %}" data-language="{{ language }}">
+                    <input type="radio" name="language" value="{{ language }}" class="hidden" {% if language == 'english' %}checked{% endif %}>
+                    <span class="px-3 py-1 rounded-full bg-dark-700 hover:bg-dark-600 text-white text-sm cursor-pointer">{{ language|capitalize }}</span>
+                  </div>
+                  {% endfor %}
+                </div>
+              </div>
+
               <!-- Voice selection -->
               <div class="mb-6">
                 <label class="block text-sm font-medium text-white mb-2">Voice</label>
                 <div class="grid grid-cols-1 md:grid-cols-2 lg:grid-cols-4 gap-4">
                   {% for voice_option in voices %}
-                  <div class="voice-card {% if voice_option == DEFAULT_VOICE %}active{% endif %}" data-voice="{{ voice_option }}">
+                  <div class="voice-card {% if voice_option == DEFAULT_VOICE %}active{% endif %}" 
+                       data-voice="{{ voice_option }}" 
+                       data-language="{{ VOICE_TO_LANGUAGE[voice_option] }}"
+                       style="display: {% if VOICE_TO_LANGUAGE[voice_option] != 'english' %}none{% endif %}">
                     <input type="radio" name="voice" value="{{ voice_option }}" class="hidden" {% if voice_option == DEFAULT_VOICE %}checked{% endif %}>
                     <div class="flex items-center mb-2">
                       <span class="font-medium text-white">{{ voice_option|capitalize }}</span>
                     </div>
                     <div class="text-xs text-dark-300">
-                      {% if voice_option == "tara" %}Female, conversational, clear
+                      {% if voice_option == "tara" %}Female, English, conversational, clear
-                      {% elif voice_option == "leah" %}Female, warm, gentle
+                      {% elif voice_option == "leah" %}Female, English, warm, gentle
-                      {% elif voice_option == "jess" %}Female, energetic, youthful
+                      {% elif voice_option == "jess" %}Female, English, energetic, youthful
-                      {% elif voice_option == "leo" %}Male, authoritative, deep
+                      {% elif voice_option == "leo" %}Male, English, authoritative, deep
-                      {% elif voice_option == "dan" %}Male, friendly, casual
+                      {% elif voice_option == "dan" %}Male, English, friendly, casual
-                      {% elif voice_option == "mia" %}Female, professional, articulate
+                      {% elif voice_option == "mia" %}Female, English, professional, articulate
-                      {% elif voice_option == "zac" %}Male, enthusiastic, dynamic
+                      {% elif voice_option == "zac" %}Male, English, enthusiastic, dynamic
-                      {% elif voice_option == "zoe" %}Female, calm, soothing
+                      {% elif voice_option == "zoe" %}Female, English, calm, soothing
+                      
+                      <!-- French voices -->
+                      {% elif voice_option == "pierre" %}Male, French, sophisticated
+                      {% elif voice_option == "amelie" %}Female, French, elegant
+                      {% elif voice_option == "marie" %}Female, French, spirited
+                      
+                      <!-- German voices -->
+                      {% elif voice_option == "jana" %}Female, German, clear
+                      {% elif voice_option == "thomas" %}Male, German, authoritative
+                      {% elif voice_option == "max" %}Male, German, energetic
+                      
+                      <!-- Korean voices -->
+                      {% elif voice_option == "유나" %}Female, Korean, melodic
+                      {% elif voice_option == "준서" %}Male, Korean, confident
+                      
+                      <!-- Hindi voice -->
+                      {% elif voice_option == "ऋतिका" %}Female, Hindi, expressive
+                      
+                      <!-- Mandarin voices -->
+                      {% elif voice_option == "长乐" %}Female, Mandarin, gentle
+                      {% elif voice_option == "白芷" %}Female, Mandarin, clear
+                      
+                      <!-- Spanish voices -->
+                      {% elif voice_option == "javi" %}Male, Spanish, warm
+                      {% elif voice_option == "sergio" %}Male, Spanish, professional
+                      {% elif voice_option == "maria" %}Female, Spanish, friendly
+                      
+                      <!-- Italian voices -->
+                      {% elif voice_option == "pietro" %}Male, Italian, passionate
+                      {% elif voice_option == "giulia" %}Female, Italian, expressive
+                      {% elif voice_option == "carlo" %}Male, Italian, refined
                       {% endif %}
                     </div>
                   </div>
@@ -404,20 +451,83 @@
       // Initialize char count
       charCount.textContent = textArea.value.length;
       
-      // Voice selection
+      // Language selection
+      const languageOptions = document.querySelectorAll('.language-option');
       const voiceCards = document.querySelectorAll('.voice-card');
       
+      languageOptions.forEach(option => {
+        option.addEventListener('click', function() {
+          // Unselect all language options
+          languageOptions.forEach(o => o.classList.remove('active'));
+          
+          // Select this language option
+          this.classList.add('active');
+          
+          // Check the radio button
+          const radio = this.querySelector('input[type="radio"]');
+          radio.checked = true;
+          
+          // Get the selected language
+          const selectedLanguage = this.getAttribute('data-language');
+          
+          // Show/hide voice cards based on language
+          let firstVisibleCard = null;
+          
+          voiceCards.forEach(card => {
+            const cardLanguage = card.getAttribute('data-language');
+            if (cardLanguage === selectedLanguage) {
+              card.style.display = '';
+              if (!firstVisibleCard) {
+                firstVisibleCard = card;
+              }
+            } else {
+              card.style.display = 'none';
+              // If this hidden card was selected, unselect it
+              if (card.classList.contains('active')) {
+                card.classList.remove('active');
+                const cardRadio = card.querySelector('input[type="radio"]');
+                if (cardRadio) {
+                  cardRadio.checked = false;
+                }
+              }
+            }
+          });
+          
+          // If no voice card is selected for this language, select the first one
+          if (firstVisibleCard && document.querySelectorAll(`.voice-card[data-language="${selectedLanguage}"].active:not([style*="display: none"])`).length === 0) {
+            firstVisibleCard.classList.add('active');
+            const firstVisibleRadio = firstVisibleCard.querySelector('input[type="radio"]');
+            if (firstVisibleRadio) {
+              firstVisibleRadio.checked = true;
+            }
+          }
+        });
+      });
+      
+      // Voice card selection
       voiceCards.forEach(card => {
         card.addEventListener('click', function() {
-          // Unselect all cards
+          // Only allow selection of cards that are visible (not display:none)
-          voiceCards.forEach(c => c.classList.remove('active'));
+          if (this.style.display !== 'none') {
+            // Unselect all cards with the same language
+            const cardLanguage = this.getAttribute('data-language');
+            document.querySelectorAll(`.voice-card[data-language="${cardLanguage}"]`).forEach(c => {
+              c.classList.remove('active');
+              const radio = c.querySelector('input[type="radio"]');
+              if (radio) {
+                radio.checked = false;
+              }
+            });
             
             // Select this card
             this.classList.add('active');
             
             // Check the radio button
             const radio = this.querySelector('input[type="radio"]');
+            if (radio) {
               radio.checked = true;
+            }
+          }
         });
       });
       

tts_engine/__init__.py

@@ -11,5 +11,7 @@ from .inference import (
     generate_speech_from_api,
     AVAILABLE_VOICES,
     DEFAULT_VOICE,
+    VOICE_TO_LANGUAGE,
+    AVAILABLE_LANGUAGES,
     list_available_voices
 )

tts_engine/inference.py

@@ -136,10 +136,43 @@ if not IS_RELOADER:
 # Parallel processing settings
 NUM_WORKERS = 4 if HIGH_END_GPU else 2
 
-# Available voices based on the Orpheus-TTS repository
+# Define voices by language
-AVAILABLE_VOICES = ["tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"]
+ENGLISH_VOICES = ["tara", "leah", "jess", "leo", "dan", "mia", "zac", "zoe"]
+FRENCH_VOICES = ["pierre", "amelie", "marie"]
+GERMAN_VOICES = ["jana", "thomas", "max"]
+KOREAN_VOICES = ["유나", "준서"]
+HINDI_VOICES = ["ऋतिका"]
+MANDARIN_VOICES = ["长乐", "白芷"]
+SPANISH_VOICES = ["javi", "sergio", "maria"]
+ITALIAN_VOICES = ["pietro", "giulia", "carlo"]
+
+# Combined list for API compatibility
+AVAILABLE_VOICES = (
+    ENGLISH_VOICES + 
+    FRENCH_VOICES + 
+    GERMAN_VOICES + 
+    KOREAN_VOICES + 
+    HINDI_VOICES + 
+    MANDARIN_VOICES + 
+    SPANISH_VOICES + 
+    ITALIAN_VOICES
+)
 DEFAULT_VOICE = "tara"  # Best voice according to documentation
 
+# Map voices to languages for the UI
+VOICE_TO_LANGUAGE = {}
+VOICE_TO_LANGUAGE.update({voice: "english" for voice in ENGLISH_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "french" for voice in FRENCH_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "german" for voice in GERMAN_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "korean" for voice in KOREAN_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "hindi" for voice in HINDI_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "mandarin" for voice in MANDARIN_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "spanish" for voice in SPANISH_VOICES})
+VOICE_TO_LANGUAGE.update({voice: "italian" for voice in ITALIAN_VOICES})
+
+# Languages list for the UI
+AVAILABLE_LANGUAGES = ["english", "french", "german", "korean", "hindi", "mandarin", "spanish", "italian"]
+
 # Import the unified token handling from speechpipe
 from .speechpipe import turn_token_into_id, CUSTOM_TOKEN_PREFIX