CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.olmo / StdMoE_1b14b_1T_EmoAnnealed
From: allenai <huggingface.co/allenai>Newsgroups: alt.freeweights.olmoSubject:

StdMoE_1b14b_1T_EmoAnnealed

Date: 2026-11-15Commit: d18c640Signed: 2DC28CC7B26A283B ✓

Text generation model with 13.6B parameters. Safetensors weights, 54.3 GB in all. Published by allenai under Apache-2.0.

13.6B params54.3GBStdMoE_1b14b_1T_EmoAnnealedsafetensorspermissive
FactsParams13.6BSize54.3 GBFormatssafetensorsTasktext generationLicenseApache-2.0From sourcepermissive
$ fw get allenai/StdMoE_1b14b_1T_EmoAnnealed
# manifest sha256 b386941aa676… · 54.3 GB · 24 files
$ fw verify ./StdMoE_1b14b_1T_EmoAnnealed allenai/StdMoE_1b14b_1T_EmoAnnealed
From sourceHosted (signed)allenai/StdMoE_1b14b_1T_EmoAnnealedLast check: 2026-11-15 03:50 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run24Fine-tune29Convert15Evaluate13Scan13Apps58
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
54.3 GB
Weights
54.3 GB (safetensors)
Parameters
13.6B
Formats
safetensors
Task
text generation
License
Apache-2.0 (permissive)
Files
24, each with a sha256
Publisher
allenai
Added
2026-11-15
Section
alt.freeweights.olmo
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 03:50 UTCpassed

Model card

From the publisher at commit d18c640. Shown as plain text with links; images open at the publisher.

StdMoE_1b14b_1T_EmoAnnealed#

The ablation checkpoint released alongside EMO: Pretraining Mixture of Experts for Emergent Modularity — referred to as EMO-anneal in Appendix B.4 / Table 3 of the paper.

1B-active / 14B-total parameter Mixture-of-Experts model (128 experts: 127 routed + 1 shared, k=8 active per token) pretrained on 1T tokens of the OLMoE pretraining mix with the standard MoE objective, then annealed for an additional 50B tokens under the EMO document-level expert pool constraint. Used to test whether modularity can be induced after pretraining instead of during it.

Usage#

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "allenai/StdMoE_1b14b_1T_EmoAnnealed"
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

inputs = tokenizer(["Language modeling is "], return_tensors="pt", return_token_type_ids=False)
out = model.generate(**inputs, max_new_tokens=100, do_sample=True, temperature=1.0, top_p=0.7)
print(tokenizer.batch_decode(out, skip_special_tokens=True)[0])

Citation#

@article{wang2026emo,
  title  = {EMO: Pretraining Mixture of Experts for Emergent Modularity},
  author = {Wang, Ryan and Bhagia, Akshita and Min, Sewon},
  year   = {2026},
  url    = {https://arxiv.org/abs/2605.06663}
}

License#

This model is licensed under Apache 2.0. It is intended for research and educational use in accordance with Ai2's Responsible Use Guidelines

Files

24 files · 54.3 GBWeights 54.3 GB (safetensors). Sizes and sha256 from the signed manifest.
Show all 24 files with their sha256
FileSizesha256
.gitattributes1.52 KB11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361
README.md1.93 KBb4fcc48ed9500c8272b21e914976e40184a6c0ec39ef8c26a94ed225f29ab447
chat_template.jinja196 B153280e3ff55d19da1398bdb3914ee2a51b80429bfaedde11d7d216c39db80f3
config.json1.12 KB508fb6d7ae0419a27af8e9d776ba546152e89d259a80c35a9c5e3f8cfe8ee66e
configuration_emo.py11.7 KBf1bd419d8dd926cf7d15131b8192938a0feda1b003586320fa56de20360e4bf2
generation_config.json69 B49b7b376c9a77281e37b1544e00f6b83ffe1d46bd47ff1c5ee04a7d642ecbe2b
merges.txt917 KBb6fe424e334903f7fb84d3a106d9730455f4744b9fe3c21ee136d97a00e72502
model-00001-of-00011.safetensors4.99 GBe540f8b2153f65a4b4e537c4a5527d679d61ddb8d16ecd2423a9559b513ebddf
model-00002-of-00011.safetensors4.99 GBeebf81fb544c865d782959823469dd8061a972667486b7f577866f94f8fe17d3
model-00003-of-00011.safetensors4.99 GB01c1fd06789f7613d0d738ac04b006988ad6005036fca02800f02db555b910b3
model-00004-of-00011.safetensors4.99 GBc823ef8946c1020f4e080bbcc9288a4f6444974d6cd2af6d857981da50ca51af
model-00005-of-00011.safetensors4.99 GBc8f8f5fd9f3d52366e2bfc5d4bea53ca33e04ba6574c9fdb2be2d30fa44295f3
model-00006-of-00011.safetensors4.99 GBd91e2bd2fe51c87ea3fcd50053f68964c053a6c8b22355f7998d590776a8d8cf
model-00007-of-00011.safetensors4.99 GBeed70e87a4b8636519b2ab1942664648a09e3894750613f1aaea07e2c413f80e
model-00008-of-00011.safetensors4.99 GBfbb0c6bc433747e746aff235fcd834d81d09babab218297213e77ccb25f3e074
model-00009-of-00011.safetensors4.99 GB8df1724d074aeb54eb6f7ceb4bb917c93f8a040051a28dac04b61c83a8a7dde4
model-00010-of-00011.safetensors4.99 GB3806c9a0cb5f7e010f1486579b9d0d996b6f5b7e11f153287410db127e940670
model-00011-of-00011.safetensors4.35 GBdf53e986441e8a4b8d41befe3ca5230e423199d305287922201c2f1fc74031b0
model.safetensors.index.json562 KB934e6eb0b66c52b4f4a5d7ef7fde489f41d8fa697ea8ff52862c959377412339
modeling_emo.py55.5 KB26f57354940655db673b35d47e9c6b1a85900068f41d91cb08afed4ec53219d6
special_tokens_map.json207 B4cb2b52960bababa8ec27153831e405ef7811e3fad1226106e09c28e715cfb21
tokenizer.json7.14 MB73fd5254624f39a88e3faac6a8e11300fc3c735ed37880d4f4f08db898eaecca
tokenizer_config.json4.31 KB733b2ec0f743cf206dd20335eacbd4427c85487fd6f5216bb361f92a85314315
vocab.json1.61 MB9e14712c91b37c7aab74b1306baa46ac342d620637a4b44523cdc3aec7d24195
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
d18c640b39 pinned now2026-11-15 03:50 UTC24 files · 54.3 GB

Use it

$ fw get allenai/StdMoE_1b14b_1T_EmoAnnealed
# manifest sha256 b386941aa676… · 54.3 GB · 24 files
$ fw verify ./StdMoE_1b14b_1T_EmoAnnealed allenai/StdMoE_1b14b_1T_EmoAnnealed
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/allenai/StdMoE_1b14b_1T_EmoAnnealed.json --dir ./StdMoE_1b14b_1T_EmoAnnealed $ python3 fw.py verify ./StdMoE_1b14b_1T_EmoAnnealed ./allenai__StdMoE_1b14b_1T_EmoAnnealed.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download allenai/StdMoE_1b14b_1T_EmoAnnealed --revision d18c640b39e67f89669f38c5db8b5ed22a6927ca --local-dir ./StdMoE_1b14b_1T_EmoAnnealed
transformersLoad the weights in Python.
import os os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com" from transformers import AutoModelForCausalLM, AutoTokenizer tok = AutoTokenizer.from_pretrained("allenai/StdMoE_1b14b_1T_EmoAnnealed", revision="d18c640b39e67f89669f38c5db8b5ed22a6927ca") model = AutoModelForCausalLM.from_pretrained("allenai/StdMoE_1b14b_1T_EmoAnnealed", revision="d18c640b39e67f89669f38c5db8b5ed22a6927ca", device_map="auto")
vLLMServe an OpenAI-compatible API.
$ HF_ENDPOINT=https://freeweights.allternit.com vllm serve allenai/StdMoE_1b14b_1T_EmoAnnealed --revision d18c640b39e67f89669f38c5db8b5ed22a6927ca
Run at home

License

Name
Apache License 2.0
Id
Apache-2.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
b4fcc48ed9500c8272b21e914976e40184a6c0ec39ef8c26a94ed225f29ab447
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: allenai <huggingface.co/allenai>Newsgroups: alt.freeweights.olmoSubject: StdMoE_1b14b_1T_EmoAnnealedDate: 2026-11-15Commit: d18c640Signed: 2DC28CC7B26A283B ✓
Upstream
allenai/StdMoE_1b14b_1T_EmoAnnealed
Pinned commit
d18c640b39e67f89669f38c5db8b5ed22a6927ca
Manifest
StdMoE_1b14b_1T_EmoAnnealed.json · .minisig
Manifest sha256
b386941aa6767d37bd24da73c74c224fd3837b9a995f8ba2fdd5ef19ca26fad6
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 03:50 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/allenai/StdMoE_1b14b_1T_EmoAnnealed.json $ curl -O https://freeweights.allternit.com/manifests/allenai/StdMoE_1b14b_1T_EmoAnnealed.json.minisig $ minisign -Vm StdMoE_1b14b_1T_EmoAnnealed.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download allenai/StdMoE_1b14b_1T_EmoAnnealed
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. StdMoE_1b14b_1T_EmoAnnealed.json. Provenance. Takedown history: none.
Source
huggingface.co/allenai/StdMoE_1b14b_1T_EmoAnnealed

Related models

All of alt.freeweights.olmo →
From: bartowski <huggingface.co/bartowski>Subject: allenai_Olmo-3-7B-Instruct-GGUF
From: bartowski <huggingface.co/bartowski>Subject: allenai_Olmo-3-7B-Think-GGUF
From: bartowski <huggingface.co/bartowski>Subject: allenai_Olmo-3.1-32B-Instruct-GGUF
From: bartowski <huggingface.co/bartowski>Subject: allenai_Olmo-3.1-32B-Think-GGUF
From: bartowski <huggingface.co/bartowski>Subject: allenai_olmOCR-2-7B-1025-GGUF
From: bartowski <huggingface.co/bartowski>Subject: allenai_olmOCR-7B-0225-preview-GGUF