CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.text / BUD-E-Whisper_V1.1
From: laion <huggingface.co/laion>Newsgroups: alt.freeweights.textSubject:

BUD-E-Whisper_V1.1

Date: 2026-11-15Commit: 9a4b112Signed: 2DC28CC7B26A283B ✓

Automatic speech recognition model with 242M parameters. Safetensors weights, 2.89 GB in all. Published by laion under CC-BY-4.0.

242M params2.89GBBUD-E-Whisper_V1.1safetensorspermissive
FactsParams242MSize2.89 GBFormatssafetensorsTaskautomatic speech recognitionLicenseCC-BY-4.0From sourcepermissive
$ fw get laion/BUD-E-Whisper_V1.1
# manifest sha256 ebb0ce5c2606… · 2.89 GB · 11 files
$ fw verify ./BUD-E-Whisper_V1.1 laion/BUD-E-Whisper_V1.1
From sourceHosted (signed)laion/BUD-E-Whisper_V1.1Last check: 2026-11-15 04:16 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run24Fine-tune7Convert8Scan2
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
2.89 GB
Weights
967 MB (safetensors)
Parameters
242M
Formats
safetensors
Task
automatic speech recognition
License
CC-BY-4.0 (permissive)
Files
11, each with a sha256
Publisher
laion
Added
2026-11-15
Section
alt.freeweights.text
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 04:16 UTCpassed

Model card

From the publisher at commit 9a4b112. Shown as plain text with links; images open at the publisher.

BUD-E Whisper: Emotional Speech Captioning Model#

BUD-E Whisper is a suite of Whisper models fine-tuned for direct emotional speech captioning. The core models are built upon OpenAI's Whisper architecture, with the current primary variant being a fine-tune of OpenAI Whisper Small. These models are designed to generate text captions that not only transcribe speech but also inherently reflect its emotional content.

V 1.1 Update: This version was finetuned on 20k samples from the Emolia dataset, that score high in the 40 categories & had been recaptioned with Gemini 2.5 Pro to describe reverb, gender & background events better.

License#

This model is released under the CC-by-4.0 license. Please give attribution to Maurice Kraus & Christoph Schuhmann, who made this model.

Colab#

Open In Colab

Training Data#

BUD-E Whisper was trained on a combination of:

Training Procedure & Caption Generation#

A key aspect of BUD-E Whisper's development was a multi-step caption refinement process to create rich training targets:

  1. Initial Score Generation: An iterative process using Gemini Flash 2.0 generated initial 40-dimensional emotion scores (0-4 scale) and 15 additional dimensions like age, arousal, valence, dominance, harshness, vocalbursts,... for all audio snippets.
  2. Templated Captions: These scores were converted into templated string captions.
  3. Paraphrasing for Richness: Gemini Flash 2.0 was then used to paraphrase these templated captions, creating diverse and semantically rich training targets.
  4. Fine-tuning: Various Whisper model sizes (including the aforementioned fine-tune of OpenAI Whisper Small) were fine-tuned on these refined, emotionally-aware captions.

This multi-step caption refinement was crucial for performance. Direct score regression or simple templated captions were found to lead to suboptimal performance for emotional speech captioning with Whisper models.

Intended Use#

  • Generating emotionally nuanced captions for audio content.
  • Providing rich embeddings for downstream emotion recognition tasks (e.g., with Empathic Insight - Voice).

Files

11 files · 2.89 GBWeights 967 MB (safetensors). Sizes and sha256 from the signed manifest.
Hide all 11 files with their sha256
FileSizesha256
.gitattributes1.52 KB11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361
README.md2.63 KBec46f9fda68dbaf6fc4e8b20973fc5f6c31bde9eb5855d0f554f837029bb06f3
config.json1.37 KB87b48f14cae61948f72f61f4229303a9b372cf2d26a3a2e2593ffc835188d5e8
generation_config.json3.74 KB0fe721f62a067007d6a9f7fbd8798a80047753a675aaef0785405ea6b5dad9ee
model.safetensors967 MBead909372a2a766b617993f043dad8ff1a11db8a899965f7dfbac94be38e36d2
optimizer.pt1.93 GB44634153b29535e1e880df19ce8bc7f5844756840b937b705b0ecf808e514f62
preprocessor_config.json339 Ba6a76d28c93edb273669eb9e0b0636a2bddbb1272c3261e47b7ca6dfdbac1b8d
rng_state.pth14.6 KB4dbde3d16d238f431ad72913d705cad907865e263e6e7e2857de211ea4d33697
scheduler.pt1.51 KB02e5f7f9540ce3531a31ffc42733cd24272cb816c4d68435ce7e4716362c26bc
trainer_state.json23.6 KBf4587a446a1f0ddcdc3cc0c00c66fd164859b23f8559ee66f59bd85654d3b67c
training_args.bin5.58 KBa9d5aa7e7e3f9001a7603fc3c3bbe51c76cadea792556380fa7785b8747c7cd0
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
9a4b11214f pinned now2026-11-15 04:16 UTC11 files · 2.89 GB

Use it

$ fw get laion/BUD-E-Whisper_V1.1
# manifest sha256 ebb0ce5c2606… · 2.89 GB · 11 files
$ fw verify ./BUD-E-Whisper_V1.1 laion/BUD-E-Whisper_V1.1
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/laion/BUD-E-Whisper_V1.1.json --dir ./BUD-E-Whisper_V1.1 $ python3 fw.py verify ./BUD-E-Whisper_V1.1 ./laion__BUD-E-Whisper_V1.1.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download laion/BUD-E-Whisper_V1.1 --revision 9a4b11214f2f37e87a05dc75a51edd472e7bba59 --local-dir ./BUD-E-Whisper_V1.1
transformersSpeech recognition with the transformers pipeline.
import os os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com" from transformers import pipeline asr = pipeline("automatic-speech-recognition", model="laion/BUD-E-Whisper_V1.1", revision="9a4b11214f2f37e87a05dc75a51edd472e7bba59") print(asr("audio.wav")["text"])
Run at home

License

Name
Creative Commons Attribution 4.0
Id
CC-BY-4.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
ec46f9fda68dbaf6fc4e8b20973fc5f6c31bde9eb5855d0f554f837029bb06f3
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: laion <huggingface.co/laion>Newsgroups: alt.freeweights.textSubject: BUD-E-Whisper_V1.1Date: 2026-11-15Commit: 9a4b112Signed: 2DC28CC7B26A283B ✓
Upstream
laion/BUD-E-Whisper_V1.1
Pinned commit
9a4b11214f2f37e87a05dc75a51edd472e7bba59
Manifest
BUD-E-Whisper_V1.1.json · .minisig
Manifest sha256
ebb0ce5c2606fcabff25439f91ef5b7bf322659fcbb1636e0a33fb44212a7d7b
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 04:16 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/laion/BUD-E-Whisper_V1.1.json $ curl -O https://freeweights.allternit.com/manifests/laion/BUD-E-Whisper_V1.1.json.minisig $ minisign -Vm BUD-E-Whisper_V1.1.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

1 model in the catalog name this one as their parent.

Fine-tunes
1
Fine-tunes 1
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download laion/BUD-E-Whisper_V1.1
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. BUD-E-Whisper_V1.1.json. Provenance. Takedown history: none.
Source
huggingface.co/laion/BUD-E-Whisper_V1.1

Related models

All of alt.freeweights.text →
From: laion <huggingface.co/laion>Subject: a2-rl-e2egit_large-50-32B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_e2egit-large-15-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_e2egit-v2-10-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_unitsyn-python-large-10-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_mix_h2_language_proportional-65-8B