higher audio bandwidth out of the box, smaller/faster, measured higher word accuracy on identical prompts (invWER 0.97 vs 0.83)
It keeps the multilingual coverage of the base model (English, German, Chinese, and many more — see the language list), and is fully API-compatible with MOSS-TTS-v1.5: same processor, same generate interface, same input schema.
The model is prompted through the processor's native build_user_message fields:
instruction — how to say it: the emotion, character, or performance direction (e.g. "Speak like a weary old sailor telling a ghost story, low and gravelly.").
text — what to say: the words to be spoken.
language — the language of text (e.g. "English", "German", "Chinese"). Set it whenever the language is known; it improves multilingual stability.
reference — optional audio (a file path, URL, or waveform tensor) to clone a voice. Omit it to let the model invent a voice consistent with the instruction.
from transformers import AutoModel, AutoProcessor
import torch, torchaudio
# Recommended SDPA backend settings (the cuDNN SDPA path is broken for this model)
torch.backends.cuda.enable_cudnn_sdp(False)
torch.backends.cuda.enable_flash_sdp(True)
torch.backends.cuda.enable_mem_efficient_sdp(True)
torch.backends.cuda.enable_math_sdp(True)
model_id = "laion/moss-tts-v1.5-8b-voice-acting"
device, dtype = "cuda", torch.bfloat16
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
processor.audio_tokenizer = processor.audio_tokenizer.to(device)
model = AutoModel.from_pretrained(
model_id,
trust_remote_code=True,
dtype=dtype,
attn_implementation="sdpa", # or "flash_attention_2" if installed
).to(device).eval()
def synth(message, out_path):
batch = processor([[message]], mode="generation")
with torch.no_grad():
outputs = model.generate(
input_ids=batch["input_ids"].to(device),
attention_mask=batch["attention_mask"].to(device),
max_new_tokens=4096,
)
audio = processor.decode(outputs)[0].audio_codes_list[0]
torchaudio.save(out_path, audio.unsqueeze(0).cpu().float(),
processor.model_config.sampling_rate)
# 1) Expressive delivery, model invents a voice (no reference)
synth(processor.build_user_message(
instruction="Excited and breathless, like sharing amazing news with a best friend!",
text="You will not believe what just happened. We actually did it!",
language="English",
), "invented_voice.wav")
# 2) Voice cloning — clone the timbre/style of a reference clip
synth(processor.build_user_message(
instruction="Calm, warm, reassuring bedtime-story narrator.",
text="Once upon a time, in a quiet little village, everyone slept soundly.",
language="English",
reference=["/path/to/reference.wav"],
), "cloned_voice.wav")
processor.decode(...) returns a list of messages; each carries the decoded waveform in message.audio_codes_list[0] at processor.model_config.sampling_rate (24 kHz).
Sampling settings. A grid search over temperature and audio repetition penalty (scored for intelligibility, quality, and genuineness on a held-out English + German set) gives good defaults of temperature 1.0, repetition penalty 1.1 with a reference clip, and temperature 0.8, repetition penalty 1.1 without a reference. See GENERATION_SETTINGS.md for the full results table and per-metric best configs.
This model was fine-tuned on a mix of the Emolia corpus and synthetic speech generated by several TTS models — primarily Gemini-3.1 TTS and DramaBox TTS. The goal of the mix was to concentrate on expressive, emotional, and character-driven delivery while preserving the base model's multilingual and voice-cloning abilities. It is a conservative full fine-tune: the base model's general capabilities are largely retained.
Put performance cues in instruction. Emotion, character, pacing, and vocal-burst / performance cues (laughter, sighs, gasps, whispering, shouting) belong in instruction, not in text.
German text: write it naturally. Use lowercase with proper umlauts (ä ö ü, not ae/oe/ue) and avoid ALL-CAPS, which the model tends to mispronounce. Reserve capitals for normal sentence casing.
Use punctuation for emphasis and prosody.!, ?, ., and ... meaningfully shape intonation and pauses — lean on them instead of capitalisation.
Set language whenever you know it; it stabilises multilingual synthesis.
Data augmentation. Voice cloning combined with text paraphrase is an effective way to generate additional, style-consistent training material.
This is a conservative full fine-tune. It broadens expressive and voice-acting behaviour, but it may not beat task-specific baselines on every axis (e.g. raw intelligibility, speaker similarity, or a single narrow language/domain).
Expressiveness is driven by the instruction field; vague or contradictory instructions can produce inconsistent delivery. Iterate on wording.
As with any generative TTS, outputs should not be used to impersonate real individuals without consent. Use responsibly.
onnx-community 
mlx-community