CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.embed / distilhubert
From: ntu-spml <huggingface.co/ntu-spml>Newsgroups: alt.freeweights.embedSubject:

distilhubert

Date: 2026-11-15Commit: fa87d96Signed: 2DC28CC7B26A283B ✓

Feature extraction model with 23M parameters. Safetensors weights, 188 MB in all. Published by ntu-spml under Apache-2.0.

23M params188MBdistilhubertsafetensorspermissive
FactsParams23MSize188 MBFormatssafetensorsTaskfeature extractionLicenseApache-2.0From sourcepermissive
$ fw get ntu-spml/distilhubert
# manifest sha256 4e472596a926… · 188 MB · 6 files
$ fw verify ./distilhubert ntu-spml/distilhubert
From sourceHosted (signed)ntu-spml/distilhubertLast check: 2026-11-15 02:36 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run9Fine-tune8Convert8Evaluate1Scan2Apps16

Evaluate with

MTEB
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
188 MB
Weights
94.0 MB (safetensors)
Parameters
23M
Formats
safetensors
Task
feature extraction
License
Apache-2.0 (permissive)
Files
6, each with a sha256
Publisher
ntu-spml
Added
2026-11-15
Section
alt.freeweights.embed
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 02:36 UTCpassed

Model card

From the publisher at commit fa87d96. Shown as plain text with links; images open at the publisher.

DistilHuBERT#

DistilHuBERT by NTU Speech Processing & Machine Learning Lab

The base model pretrained on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz.

Note: This model does not have a tokenizer as it was pretrained on audio alone. In order to use this model speech recognition, a tokenizer should be created and the model should be fine-tuned on labeled text data. Check out this blog for more in-detail explanation of how to fine-tune the model.

Paper: DistilHuBERT: Speech Representation Learning by Layer-wise Distillation of Hidden-unit BERT

Authors: Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee

Abstract Self-supervised speech representation learning methods like wav2vec 2.0 and Hidden-unit BERT (HuBERT) leverage unlabeled speech data for pre-training and offer good representations for numerous speech processing tasks. Despite the success of these methods, they require large memory and high pre-training costs, making them inaccessible for researchers in academia and small companies. Therefore, this paper introduces DistilHuBERT, a novel multi-task learning framework to distill hidden representations from a HuBERT model directly. This method reduces HuBERT's size by 75% and 73% faster while retaining most performance in ten different tasks. Moreover, DistilHuBERT required little training time and data, opening the possibilities of pre-training personal and on-device SSL models for speech.

The original model can be found under https://github.com/s3prl/s3prl/tree/master/s3prl/upstream/distiller .

Usage#

See this blog for more information on how to fine-tune the model. Note that the class Wav2Vec2ForCTC has to be replaced by HubertForCTC.

Files

6 files · 188 MBWeights 94.0 MB (safetensors). Sizes and sha256 from the signed manifest.
Hide all 6 files with their sha256
FileSizesha256
.gitattributes1.23 KB98ccb431c012ebfe976280fbd45aea4cec7409935868ccecf3954370f96732a1
README.md2.05 KB66d58120d6c4456a3bc9aa82f96187174d5a49e9ad79b4702477059a44c94901
config.json1.30 KB0427043d1c4c8d58298d2050420a0c2ce85c8b92ec3865eaecc62c3fc5a97006
model.safetensors94.0 MB77ad8f985c97750d8d27d6950c917800d80732a141370bf316058bded71a6cbe
preprocessor_config.json214 B7a8f0fc8ee1272ed28a8b468e183c56a19329a92babd3ca8d4d21ae02ba511cd
pytorch_model.bin94.0 MB6d7fb982c1365c60aae4e1a7af329ec0bbe50aec47ffafa9647e17e375009de6
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
fa87d96265 pinned now2026-11-15 02:36 UTC6 files · 188 MB

Use it

$ fw get ntu-spml/distilhubert
# manifest sha256 4e472596a926… · 188 MB · 6 files
$ fw verify ./distilhubert ntu-spml/distilhubert
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/ntu-spml/distilhubert.json --dir ./distilhubert $ python3 fw.py verify ./distilhubert ./ntu-spml__distilhubert.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download ntu-spml/distilhubert --revision fa87d96265d6b7af66e112faff6ff44df419cec9 --local-dir ./distilhubert
transformersLoad the weights in Python.
import os os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com" from transformers import AutoModel, AutoTokenizer tok = AutoTokenizer.from_pretrained("ntu-spml/distilhubert", revision="fa87d96265d6b7af66e112faff6ff44df419cec9") model = AutoModel.from_pretrained("ntu-spml/distilhubert", revision="fa87d96265d6b7af66e112faff6ff44df419cec9", device_map="auto")
Run at home

License

Name
Apache License 2.0
Id
Apache-2.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
66d58120d6c4456a3bc9aa82f96187174d5a49e9ad79b4702477059a44c94901
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: ntu-spml <huggingface.co/ntu-spml>Newsgroups: alt.freeweights.embedSubject: distilhubertDate: 2026-11-15Commit: fa87d96Signed: 2DC28CC7B26A283B ✓
Upstream
ntu-spml/distilhubert
Pinned commit
fa87d96265d6b7af66e112faff6ff44df419cec9
Manifest
distilhubert.json · .minisig
Manifest sha256
4e472596a9265446a3001d60e27f2b271c9a3c5259e6d3fddd0f42b67f733c98
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 02:36 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/ntu-spml/distilhubert.json $ curl -O https://freeweights.allternit.com/manifests/ntu-spml/distilhubert.json.minisig $ minisign -Vm distilhubert.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download ntu-spml/distilhubert
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. distilhubert.json. Provenance. Takedown history: none.
Source
huggingface.co/ntu-spml/distilhubert

Related models

All of alt.freeweights.embed →
From: sentence-transformers <huggingface.co/sentence-transformers>Subject: all-MiniLM-L6-v1
From: unsloth <huggingface.co/unsloth>Subject: all-MiniLM-L6-v2
From: mlx-community <huggingface.co/mlx-community>Subject: all-MiniLM-L6-v2-4bit
From: sentence-transformers <huggingface.co/sentence-transformers>Subject: all-mpnet-base-v1
From: sentence-transformers <huggingface.co/sentence-transformers>Subject: allenai-specter
From: sentence-transformers <huggingface.co/sentence-transformers>Subject: bert-base-nli-max-tokens