CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.text / BUD-E-Whisper
From: laion <huggingface.co/laion>Newsgroups: alt.freeweights.textSubject:

BUD-E-Whisper

Date: 2026-11-15Commit: 9400e67Signed: 2DC28CC7B26A283B ✓

Automatic speech recognition model with 242M parameters. Safetensors weights, 969 MB in all. Published by laion under CC-BY-4.0.

242M params969MBBUD-E-Whispersafetensorspermissive
FactsParams242MSize969 MBFormatssafetensorsTaskautomatic speech recognitionLicenseCC-BY-4.0From sourcepermissive
$ fw get laion/BUD-E-Whisper
# manifest sha256 282525de6199… · 969 MB · 12 files
$ fw verify ./BUD-E-Whisper laion/BUD-E-Whisper
From sourceHosted (signed)laion/BUD-E-WhisperLast check: 2026-11-15 03:28 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run24Fine-tune7Convert8Scan2
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
969 MB
Weights
967 MB (safetensors)
Parameters
242M
Formats
safetensors
Task
automatic speech recognition
License
CC-BY-4.0 (permissive)
Files
12, each with a sha256
Publisher
laion
Added
2026-11-15
Section
alt.freeweights.text
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 03:28 UTCpassed

Model card

From the publisher at commit 9400e67. Shown as plain text with links; images open at the publisher.

BUD-E Whisper: Emotional Speech Captioning Model#

BUD-E Whisper is a suite of Whisper models fine-tuned for direct emotional speech captioning. The core models are built upon OpenAI's Whisper architecture, with the current primary variant being a fine-tune of OpenAI Whisper Small. These models are designed to generate text captions that not only transcribe speech but also inherently reflect its emotional content.

The embeddings generated by BUD-E Whisper can also serve as input for Empathic Insight - Voice, a downstream ensemble of Multi-Layer Perceptrons (MLPs) designed to predict dimensional emotion scores.

License#

This model is released under the CC-by-4.0 license. Please give attribution to Maurice Kraus & Christoph Schuhmann, who made this model.

Colab#

Open In Colab

Training Data#

BUD-E Whisper was trained on a combination of:

Training Procedure & Caption Generation#

A key aspect of BUD-E Whisper's development was a multi-step caption refinement process to create rich training targets:

  1. Initial Score Generation: An iterative process using Gemini Flash 2.0 generated initial 40-dimensional emotion scores (0-4 scale) and 15 additional dimensions like age, arousal, valence, dominance, harshness, vocalbursts,... for all audio snippets.
  2. Templated Captions: These scores were converted into templated string captions.
  3. Paraphrasing for Richness: Gemini Flash 2.0 was then used to paraphrase these templated captions, creating diverse and semantically rich training targets.
  4. Fine-tuning: Various Whisper model sizes (including the aforementioned fine-tune of OpenAI Whisper Small) were fine-tuned on these refined, emotionally-aware captions.

This multi-step caption refinement was crucial for performance. Direct score regression or simple templated captions were found to lead to suboptimal performance for emotional speech captioning with Whisper models.

Intended Use#

  • Generating emotionally nuanced captions for audio content.
  • Providing rich embeddings for downstream emotion recognition tasks (e.g., with Empathic Insight - Voice).

Files

12 files · 969 MBWeights 967 MB (safetensors). Sizes and sha256 from the signed manifest.
Hide all 12 files with their sha256
FileSizesha256
.gitattributes1.52 KB11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361
README.md2.62 KBddc3d16cd54153607314d9283b08aad8cc315c5184fd5af387c89ac1767144d8
added_tokens.json34.6 KB9715fd2243b6f06a5858b5e32950d2853f73dd5bc201aafcf76f5082a2d8acd1
config.json1.43 KB199b4ed36c036908f3b54b5bef47a04fd721587e2ed2c8e33a80369ace168a8f
generation_config.json3.84 KB445fa0dc13ccd9da93f91285f80eb92b1ab1ef7376a034a15a4c43f36ff62ec3
merges.txt494 KB2df2990a395e35e8dfbc7511e08c12d56018d8d04691e0133e5d63b21e154dc6
model.safetensors967 MB7bbd466bf003d717fcc17119bc2619165dba237aaf2b47bc727e494539c1351d
normalizer.json52.7 KBbf1c507dc8724ca9cf9903640dacfb69dae2f00edee4f21ceba106a7392f26dd
preprocessor_config.json339 Ba6a76d28c93edb273669eb9e0b0636a2bddbb1272c3261e47b7ca6dfdbac1b8d
special_tokens_map.json2.19 KBe67ae3a0aaa99abcd9f187138e12db1f65c16a14761c50ef10eef2c174a7a691
tokenizer_config.json283 KB21a4fc0483c14b87f4e0bbc177a9a357479bfa7c95aaf21ad53d71e9c5afafb9
vocab.json1.04 MB50d6a919f0a0601d56a04eb583c780d18553aa388254ba3158eb6a00f13e2c1a
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
9400e6740e pinned now2026-11-15 03:28 UTC12 files · 969 MB

Use it

$ fw get laion/BUD-E-Whisper
# manifest sha256 282525de6199… · 969 MB · 12 files
$ fw verify ./BUD-E-Whisper laion/BUD-E-Whisper
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/laion/BUD-E-Whisper.json --dir ./BUD-E-Whisper $ python3 fw.py verify ./BUD-E-Whisper ./laion__BUD-E-Whisper.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download laion/BUD-E-Whisper --revision 9400e6740ea103bf619a4ce66374576fab8a4e23 --local-dir ./BUD-E-Whisper
transformersSpeech recognition with the transformers pipeline.
import os os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com" from transformers import pipeline asr = pipeline("automatic-speech-recognition", model="laion/BUD-E-Whisper", revision="9400e6740ea103bf619a4ce66374576fab8a4e23") print(asr("audio.wav")["text"])
Run at home

License

Name
Creative Commons Attribution 4.0
Id
CC-BY-4.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
ddc3d16cd54153607314d9283b08aad8cc315c5184fd5af387c89ac1767144d8
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: laion <huggingface.co/laion>Newsgroups: alt.freeweights.textSubject: BUD-E-WhisperDate: 2026-11-15Commit: 9400e67Signed: 2DC28CC7B26A283B ✓
Upstream
laion/BUD-E-Whisper
Pinned commit
9400e6740ea103bf619a4ce66374576fab8a4e23
Manifest
BUD-E-Whisper.json · .minisig
Manifest sha256
282525de61993b9762516df614a96e6cfdabcea74062fb42ed7938ce667d8c18
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 03:28 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/laion/BUD-E-Whisper.json $ curl -O https://freeweights.allternit.com/manifests/laion/BUD-E-Whisper.json.minisig $ minisign -Vm BUD-E-Whisper.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

1 model in the catalog name this one as their parent.

Fine-tunes
1
Fine-tunes 1
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download laion/BUD-E-Whisper
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. BUD-E-Whisper.json. Provenance. Takedown history: none.
Source
huggingface.co/laion/BUD-E-Whisper

Related models

All of alt.freeweights.text →
From: laion <huggingface.co/laion>Subject: a2-rl-e2egit_large-50-32B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_e2egit-large-15-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_e2egit-v2-10-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_unitsyn-python-large-10-8B
From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_mix_h2_language_proportional-65-8B