danvics/orpheus-fastapi
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README.md with information about ROCm support

Browse source
This commit is contained in:
wizardeur 2025-05-27 12:15:52 +02:00 committed by Kamil Andrusz
parent f271e5f5e1
commit 89a8492178
1 changed files with 12 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

14
README.md
View file

@ -101,7 +101,7 @@ Orpheus-FastAPI/
### Prerequisites ### Prerequisites
- Python 3.8-3.11 (Python 3.12 is not supported due to removal of pkgutil.ImpImporter) - Python 3.8-3.11 (Python 3.12 is not supported due to removal of pkgutil.ImpImporter)
- CUDA-compatible GPU (recommended: RTX series for best performance) - CUDA-compatible or ROCm-compatible GPU (recommended: RTX series for best performance)
- Using docker compose or separate LLM inference server running the Orpheus model (e.g., LM Studio or llama.cpp server) - Using docker compose or separate LLM inference server running the Orpheus model (e.g., LM Studio or llama.cpp server)
### 🐳 Docker compose ### 🐳 Docker compose
@ -122,11 +122,16 @@ ORPHEUS_MODEL_NAME=Orpheus-3b-French-FT-Q8_0.gguf # Example for French
Then start the services: Then start the services:
For GPU support run For CUDA GPU support run
```bash ```bash
docker compose -f docker-compose-gpu.yml up docker compose -f docker-compose-gpu.yml up
``` ```
For ROCm GPU support run
```bash
docker compose -f docker-compose-gpu-rocm.yml up
```
For CPU support run: For CPU support run:
```bash ```bash
docker compose -f docker-compose-cpu.yml up docker compose -f docker-compose-cpu.yml up
@ -157,6 +162,11 @@ conda activate orpheus-tts
```bash ```bash
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124 pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124
``` ```
or
Install PyTorch with ROCm support:
```bash
pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/rocm6.4/
```
4. Install other dependencies: 4. Install other dependencies:
```bash ```bash