A fine-tuned OpenAI Whisper Small model that generates structured voice attribute tags from speech audio. Instead of transcribing words, this model describes how the voice sounds — its quality, style, loudness, articulation, intonation, and emotional delivery.
Built on top of BUD-E-Whisper (V1.0) and trained for 20 epochs on voice-annotated speech data.
The model outputs a comma-separated sequence of ~8–10 voice attribute tags describing multiple dimensions of the voice simultaneously. There is no free-form caption — the entire output consists of structured tags.
Based on analysis of 570 diverse audio samples spanning 57 voice taxonomy dimensions, the tags follow a consistent positional order covering these voice dimensions:
A clean, professional narrator voice — fluent delivery with precise articulation, balanced loudness, and monotone intonation typical of audiobook or documentary narration.
An emotionally charged voice with halting, hesitant speech — quiet and breathy with falling pitch and rough voice quality, characteristic of genuine crying or deep distress.
Suitable for Work, natural pop, fluent, dramatic style, strained voice,
pressed voice, very loud, dynamic, precise articulation, screaming
A loud, high-energy voice with pressed phonation and dramatic style — the kind of intense screaming you might hear in animated entertainment or dramatic performance.
A soft, intimate ASMR-style whisper with breathy voice quality, monotone intonation, and very quiet delivery — characteristic of ASMR content or intimate speech.
Suitable for Work, natural-Suitable for Work, fluent, ranting style, strained voice,
pressed voice, very loud, dynamic, precise articulation, screaming
Agitated, high-intensity speech with ranting style, strained phonation, and very loud delivery — characteristic of passionate arguing or emotional outbursts.
Suitable for Work, natural speaking, fluent, casual speaking style, modal voice,
neutral airflow, normal loudness, slightly dynamic, precise articulation, normal speaking
A typical conversational voice — relaxed, fluent, with balanced loudness and natural modal phonation. This is the most common pattern in everyday speech.
Detection rates for specific voice characteristics across 570 diverse samples:
Voice Characteristic
Samples Detected
Detection Rate
Screaming / shouting
48
8.4%
Crying
48
8.4%
Ranting
53
9.3%
Disfluent / halting speech
58
10.2%
Narration style
67
11.8%
Whisper
7
1.2%
ASMR
3
0.5%
Pleading
6
1.1%
Not Suitable for Work
16
2.8%
These rates reflect the distribution of input samples, not the model's intrinsic sensitivity. The model reliably distinguishes these categories when presented with appropriate audio.
This is a full encoder-decoder Whisper model. The encoder maps audio to rich voice representations (also usable standalone for embedding extraction — see Voice-Taxonomy-57), while the decoder generates the tag sequence auto-regressively.
The model uses the standard Whisper tokenizer and generation config. Audio is processed as 30-second chunks of 80-bin log-mel spectrograms at 16kHz.
The encoder of this model also produces high-quality voice embeddings for downstream tasks. It is used as one of the whisper encoders in the Voice-Taxonomy-57 pipeline for 57-dimension voice classification:
from transformers import WhisperModel, WhisperFeatureExtractor
model = WhisperModel.from_pretrained("laion/voice-tagging-whisper", torch_dtype=torch.float16)
encoder = model.encoder.to("cuda").eval()
fe = WhisperFeatureExtractor.from_pretrained("openai/whisper-small")
inputs = fe(waveform, sampling_rate=16000, return_tensors="pt")
with torch.no_grad():
hidden_states = encoder(inputs.input_features.cuda().half()).last_hidden_state
# hidden_states shape: (batch, 1500, 768)
transformersSpeech recognition with the transformers pipeline.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import pipeline
asr = pipeline("automatic-speech-recognition", model="laion/voice-tagging-whisper", revision="dc223dda0fabb29a90a9d8c16b1e8ae5ff0988fe")
print(asr("audio.wav")["text"])
onnx-community 
mlx-community