danvics/orpheus-fastapi
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Update README.md

Browse source
This commit is contained in:
Alex Yong 2025-04-18 09:37:42 -04:00 committed by GitHub
parent 99bec5ccd6
commit ae68770e71
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 9 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

10
README.md
View file

@ -106,6 +106,7 @@ Orpheus-FastAPI/
### 🐳 Docker compose
The docker compose file orchestrates the Orpheus-FastAPI for audio and a llama.cpp inference server for the base model token generation. The GGUF model is downloaded with the model-init service.
There are two versions, one for machines that have access to GPU support `docker-compose-gpu.yaml` and one for CPU support only: `docker-compose-cpu.yaml`
```bash
cp .env.example .env # Create your .env file from the example
@ -119,8 +120,15 @@ ORPHEUS_MODEL_NAME=Orpheus-3b-French-FT-Q8_0.gguf # Example for French
```
Then start the services:
For GPU support run
```bash
docker compose up --build
docker compose -f docker-compose-gpu.yml up
```
For CPU support run:
```bash
docker compose -f docker-compose-cpu.yml up
```
The system will automatically download the specified model from Hugging Face before starting the service.