Note. Use this model when you want direct, low-latency answers without an explicit chain of thought — interactive chat, code assistance, tool use, and instruction following. If you need explicit reasoning before the answer (complex debugging, planning, multi-step agentic flows), use Thinking instead.
Mellum2 Instruct is a post-trained assistant model trained by JetBrains.
The model uses a Mixture-of-Experts architecture with 64 experts and activates 8 experts per token. It uses a combination of sliding-window and full attention layers, with a context length of 131,072 tokens.
It is produced from Mellum2-12B-A2.5B-Base by supervised fine-tuning followed by reinforcement learning with verifiable rewards (RLVR) on math, executable coding, tool use, instruction following, reasoning, and knowledge tasks. Mellum2 Instruct answers directly, without an externalized chain of thought.
Post-training evaluation for the instruct (no-thinking) variants. All values are percentages; higher is better except HarmBench, where lower is better. All values self-reported by JetBrains.
Benchmark
Mellum2 Instruct SFT
Mellum2 Instruct
Qwen3.5 (4B)
Qwen3.5 (9B)
OLMo-3 (7B)
Ministral 3 (14B)
Seed-Coder (8B)
Coding
LiveCodeBench v6
30.9
37.2
51.0
63.7
28.2
42.4
28.1
EvalPlus
76.2
78.4
69.4
71.8
67.3
74.1
73.8
MultiPL-E
64.6
67.1
51.0
67.1
36.1
71.5
77.0
Tool Use
BFCL v4
31.8
44.2
52.0
60.6
19.8
38.8
—
BFCL v3
43.1
66.3
64.1
70.5
41.9
52.7
—
Math
AIME
29.9
41.7
38.3
58.3
40.0
33.3
0.0
GSM-Plus
73.0
80.5
85.2
87.9
85.8
86.6
50.4
Knowledge
MMLU-Redux
77.4
78.1
87.5
91.1
71.8
85.9
38.1
GPQA Diamond
38.9
40.9
76.8
79.8
40.9
58.6
20.2
Conversational
IFEval
69.3
75.8
82.1
83.9
83.2
67.3
56.2
JetBrains pairwise
66.7
68.1
60.6
77.8
44.4
72.4
43.0
MixEval
62.9
62.2
65.9
71.1
59.4
71.2
37.2
BS-Bench
24.0
18.0
56.9
61.0
22.0
9.0
5.0
Safety
HarmBench (↓)
8.4
23.1
20.3
20.9
14.7
56.5
40.0
XSTest
78.3
81.2
93.2
91.2
91.2
96.8
86.3
Notes:
EvalPlus is the mean of HumanEval+ and MBPP+.
AIME is the mean of AIME 2025 and AIME 2026 (30 questions each).
BFCL v4 is the macro-average of five subtasks: v1, v2, v3, web search, memory.
JetBrains pairwise is win rate against Qwen2.5-7B-Instruct on an internal benchmark.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("JetBrains/Mellum2-12B-A2.5B-Instruct", revision="273d9c1c6882ac8e1562e5b7e043857d35a787b6")
model = AutoModelForCausalLM.from_pretrained("JetBrains/Mellum2-12B-A2.5B-Instruct", revision="273d9c1c6882ac8e1562e5b7e043857d35a787b6", device_map="auto")
