Mellum2.1-12B-A2.5B-Thinking-MTP

Image: Mellum

Mellum2.1 Thinking with the MTP head#

Note. This repository is Mellum2.1 Thinking with its multi-token prediction (MTP) head included. The model weights are byte-identical to that release; the head adds one shard and one config key. Use this repository for speculative decoding with vLLM 0.33 or newer and transformers 5.20 or newer, and the Thinking repository with older versions, which cannot load the extra tensors.

Note. Mellum2.1 is a thinking model. Use it for complex agentic tasks, such as working in a repository, running commands, and calling tools, and for hard non-agentic problems in coding, math, and reasoning.

Image: Mellum2.1 benchmarks

Mellum2.1 Highlights#

Mellum2.1 is the next version of Mellum2 Thinking. The architecture is unchanged (a 12B mixture-of-experts model with 2.5B active parameters), and almost all of the work for this version went into post-training, primarily reinforcement learning (RL):

• Reinforcement learning at a new scale: RL went from a short final stage to the main part of training, after many experiments on both the methods and the data.
• More data, filtered harder: new RL tasks in math, competitive programming, science, tool use, and software engineering, combining open RL datasets with tasks we built ourselves and filtering every source before training.
• Real environments for agentic skills: for software engineering, the model trains inside real repositories with a shell and file-editing tools and is rewarded when the tests pass.

As a result, Mellum2.1 handles agentic tasks much better than Mellum2. After millions of sandboxed runs in real environments during training, it explores a codebase, edits files, and checks its own changes.

Model Overview#

Mellum2.1 Thinking has the following features:

• Number of Parameters: 12B total, 2.5B active
• Number of Layers: 28
• Hidden Size: 2304
• Intermediate Size: 7168
• MoE Intermediate Size: 896
• Number of Experts: 64
• Number of Activated Experts: 8
• Number of Attention Heads (GQA): 32 for Q and 4 for KV
• Context Length: 131,072
• Sliding Window: 1,024 (3 of every 4 layers)
• Vocabulary Size: 98,304
• Precision: bfloat16
• Multi-token prediction head: 1 layer, used as a speculative decoding draft
• License: Apache 2.0

Serving with vLLM#

Requires vLLM 0.33 or newer with transformers 5.20 or newer. The --speculative-config flag turns on speculative decoding with the MTP head; without it the server runs the plain model and ignores the head.

# Without tool calling
vllm serve JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

# With tool calling
vllm serve JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP \
  --max-model-len 131072 \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice \
  --tool-call-parser hermes \
  --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}'

GGUF builds for llama.cpp, Ollama, and LM Studio, including the MTP head as a draft file, are in JetBrains/Mellum2.1-12B-A2.5B-Thinking-GGUF.

Quickstart#

from openai import OpenAI
# Configured by environment variables
client = OpenAI()

messages = [
    {"role": "user", "content": "Find the bug in this function and explain the fix: def mean(xs): return sum(xs) / len(xs) - 1"},
]

chat_response = client.chat.completions.create(
    model="JetBrains/Mellum2.1-12B-A2.5B-Thinking-MTP",
    messages=messages,
    max_tokens=81920,
    temperature=0.6,
    top_p=0.95,
    extra_body={"top_k": 20},
)
print("Chat response:", chat_response)

Multi-token prediction head#

The multi-token prediction (MTP) head is one extra transformer block, trained together with the model, that predicts the token after the one just sampled. Inference engines use it as a speculative draft: the model verifies every drafted token, so the output is unchanged while decoding runs 1.4–1.8x faster, with about 90% of the drafted tokens accepted. The head lives in model-mtp.safetensors under the mtp. prefix, announced by num_nextn_predict_layers: 1 in config.json; everything else is identical to Mellum2.1 Thinking.

• vLLM 0.33 or newer: --speculative-config '{"method": "mtp", "num_speculative_tokens": 1}', as in the serving commands above.
• transformers 5.20 or newer: pass use_mtp=True to generate.
• llama.cpp: use the GGUF repository with --spec-type draft-mtp --spec-draft-n-max 1.

Older versions cannot load the extra tensors: transformers before 5.20 skips them with a warning, vLLM releases up to 0.32 refuse the checkpoint. With those, serve Mellum2.1 Thinking instead.

Evaluation#

The model weights are those of Mellum2.1 Thinking, so its results apply unchanged. All values are percentages; higher is better except HarmBench, where lower is better. All models were evaluated by JetBrains with the same pipeline in thinking mode. All values are self-reported by JetBrains.

Notes:

• Non-agentic benchmarks use greedy decoding.
• Agentic benchmarks use the same open-source agent harness (Pi v0.73.1, shell and file tools) for every model, with each model's default sampling (temperature 1.0 for Mellum2.1), a 114K-token context, and up to 16K tokens per turn.
• AIME is the mean of AIME 2025 and AIME 2026 (30 questions each).
• BFCL v4 is the macro-average of five subtasks: v1, v2, v3, web search, memory.
• Mellum2 Thinking was re-evaluated with this pipeline, so its numbers differ slightly from the Mellum2 Technical Report.

License#

Released under the Apache 2.0 license.