Reads with emotion that fits the text and its context, no tags needed.
[angry] [whispering] [laughing] [sighing] … placed where they should take effect.
From a few seconds of reference audio and its transcript, or with the model's own voice.
Blocks of 0.74 s are final once decoded and can be played immediately.
Architecture
A text-and-audio transformer (OmniVoice, initialised from Qwen3-0.6B) generates the semantic codebook of the Fish Audio codec block by block. An axial multi-token-prediction decoder fills in the 9 acoustic codebooks, and the codec renders 44.1 kHz audio.
| Component | Details |
|---|---|
| Backbone | OmniVoice / Qwen3-0.6B, 28 layers, d = 1024, block attention (Figure 2) |
| Acoustic decoder | Axial MTP, 6 layers, d = 768, 110M parameters |
| Parameters | ~0.75B (codec excluded) |
| Codec | Fish Audio Drama 3 / S2: 44.1 kHz, 21.5 frames/s, 1 semantic codebook (4096) + 9 residual (1024) |
| Block | 16 frames = 0.74 s |
| Decoding | 6 unmasking iterations with classifier-free guidance, then 9 MTP steps, per block; KV cache |
| Training | about 8,200 h of studio-quality speech in 8 languages (a private collection), 1× RTX PRO 6000, 400k steps planned |
Samples
Checkpoint 120k, cfg=1.5, no voice prompt: the voices are the model's own. Every language has its own texts. Each item was generated 3 times; the take shown first was selected with Whisper (language identification and transcript), the other takes are under each sample.
Lines
Context sets the delivery
The same line, voice and seed. Only the preceding line, given as text context and not spoken, differs.
Longer text in one pass
Evaluation
Checkpoint 120k, cfg=1.5, one sample per item. WER with Whisper large-v3-turbo, speaker similarity with ECAPA2 (cosine), pitch variation as F0 standard deviation in semitones.
Held-out recordings
136 validation items in English, Spanish and German. Recordings and texts were never trained on. Each item clones a voice from a 6–10 s prompt and speaks two sentences that also exist as a real recording (passed through the same codec as the reference).
| OGG TTS 120k | Real recording | |
|---|---|---|
| WER | 1.4 % | 2.6 % |
| Speaker similarity to the prompt | 0.773 | 0.737 |
| Duration vs. the real recording (median) | 0.99 | 1.00 |
| Pitch variation (F0 std, semitones) | 3.58 | 4.25 |
All 8 languages
New texts, a neutral and an angry line per voice (32 items).
| EN | PL | ES | DE | RU | FR | PT | IT | |
|---|---|---|---|---|---|---|---|---|
| WER | 0.0 % | 0.8 % | 0.0 % | 0.0 % | 0.0 % | 0.0 % | 3.3 % | 6.7 % |
| Similarity | 0.780 | 0.767 | 0.828 | 0.824 | 0.763 | 0.708 | 0.771 | 0.841 |
The pace matches real speech. Intonation is still about 15 % flatter than in the real recordings; this is the main open point for the rest of the run. Public benchmarks (Seed-TTS, CV3) follow with the final checkpoint.
Usage
git clone https://github.com/FashionFlora/ogg-tts && cd ogg-tts git clone https://github.com/fishaudio/fish-speech ogg_tts/fish-speech export OMNIVOICE_PATH=$(huggingface-cli download k2-fsa/OmniVoice) export FISH_CODEC_PATH=$(huggingface-cli download fishaudio/drama-3 codec.pth)
import sys; sys.path.insert(0, "ogg_tts")
from huggingface_hub import hf_hub_download
import infer_omnifish as inf, fish_codec as fc
model, tok = inf.load_model(hf_hub_download("FashionFlora/ogg-tts", "model.safetensors"), "cuda")
codec = fc.load_codec("cuda")
codes = inf.generate_cached(model, tok, None, "", "[sighing] Hello! I am OGG.", "en", cfg=1.5)
audio = fc.decode(codec, codes[None].cuda())[0].float().cpu().numpy() # 44.1 kHz
Voice cloning: pass the reference codes (fc.encode) and its exact transcript. Context: semantic_left=(…) and after=(…). Details in the model card.