Timbre-Whisper is a Whisper-based model fine-tuned for vocal timbre tagging and natural-language voice description. It builds directly on BUD-E Whisper V1.1, extending its emotional speech captioning capabilities toward detailed perceptual descriptions of voice quality.
Short natural-language summaries that contextualize these tags in fluent English.
Architecturally, Timbre-Whisper remains a fine-tuned variant of OpenAI Whisper (Small), retaining robust speech-to-text performance while shifting the output space toward expressive timbre-focused captioning.
female_soprano, slightly_harsh, bright_shiny, mild_breathy, slight_nasal_touch, slightly_tense, slight_roughness, head_mixed, slightly_thin, mild_wear, mostly_natural, slight_wobble,
Remarkable female voice, likely a young adult, with a high-pitched, bright, and shiny timbre. The voice exhibits slight harshness, mild breathiness, and a touch of nasality, with a slightly tense production. There is slight roughness and a head-mixed resonance, contributing to a slightly thin vocal weight. The vocal health shows mild wear, and while mostly natural, there is a slight wobble in stability.
Example 2
male_baritone, slightly_harsh, moderately_bright_clear, slightly_breathy, slight_nasal_touch, tense_pressed, slight_roughness, chest_mixed, moderately_heavy, noticeable_wear, perfectly_natural, slight_wobble,
Harsh middle-aged adult male voice with medium-pitched delivery, exhibiting slight breathiness and a touch of nasality. The voice is slightly harsh and tense, with a hint of roughness. The resonance is chest-mixed, and the vocal weight is moderately heavy. There is noticeable wear, but the voice sounds perfectly natural with a slight wobble.
transformersSpeech recognition with the transformers pipeline.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="laion/timbre-whisper", revision="0eba2ef9f74ef8a37a31774dbf9fbbd40568e6fe")
print(asr("audio.wav")["text"])
