Monad is a 56 million parameters generalist Small Reasoning Model, trained on 200 billions tokens from SYNTH, a fully open generalist dataset.
As of 2025, Monad is the best contender for the smallest viable language models. Despite being less than half of gpt-2, Monad not only answers in consistent English but performs significanly beyond chance on MMLU and other major industry benchmarks.
Monad has been natively trained for instructions with thinking traces. We implemented a series of dedicated pipelines for:
Memorization of encyclopedic knowledge (50,000 vital articles from Wikipedia), though in this size range hallucinations have to be expected.
Retrieval-Augmented Generation with grounding (following on our initial experiments with Pleias-RAG series)
Arithmetic and simple math resolution problem
Editing tasks
Information extraction
Creative writing, including unusual synthetic exercises like lipograms or layout poems.
Monad is strictly monolingual in English. We trained a new custom tokenizer (likely one of the smallest tokenizer to date, less than 8,000 individual tokens), exclusively trained on SYNTH so that we maintain a relatively good compression ratio.
Monad is a 56M parameters decoders with a standard Qwen/Llama-like design, except for its extremely compact size and overall opiniated architecture for depth (with 64 layers)
Monad attains performance on MMLU significantly beyond chance with close to 30% of positive rate. We also find non-random results on gsm8k (8%) and HotPotQA (8%)
To our knowledge, there is no model remotely close in this size range for evaluation comparison. Spiritually and practically, Monad remains unique.
Monad has been trained on the standard instruction style from Qwen.
<|im_start|>user
Who are you?<|im_end|>
<|im_start|>assistant
<think>
Monad has no support yet for multi-turn.
A major envisioned use case for Monad is explainability, as the model does provide a unique trade-off between observability and actual reasoning performance.
Files
12 files · 114 MBWeights 113 MB (safetensors). Sizes and sha256 from the signed manifest.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("PleIAs/Monad", revision="d8fce6b01ee5c5575bf00a74f6e6463ab5796112")
model = AutoModelForCausalLM.from_pretrained("PleIAs/Monad", revision="d8fce6b01ee5c5575bf00a74f6e6463ab5796112", device_map="auto")
