Natively Supports 128K (Long-context extension to 512K)
Precision
bfloat16
Tested Languages
English, German, Spanish, French, Japanese, Portuguese, Arabic, Czech, Italian, Korean, Dutch, Chinese (other languages may work but have not been fully tested)
Granite is a family of open-source large language models developed by IBM, designed for enterprise and research use. Granite models are built to be versatile, safe, and efficient — covering a range of sizes and capabilities from compact edge-deployable models to large-scale reasoning systems. All Granite models are released under the Apache 2.0 license, enabling unrestricted commercial and academic use.
The Granite 4.2 generation introduces native reasoning (thinking) capabilities, allowing models to perform step-by-step chain-of-thought reasoning before producing final answers. This significantly improves performance on complex math, coding, multi-step logic, and agentic tool-calling tasks.
Granite-4.2-3B is the compact reasoning model in the Granite 4.2 family. Despite its small parameter count, it delivers strong performance on reasoning-intensive tasks by leveraging built-in <think>...</think> chain-of-thought. It supports flexible thinking modes — full thinking (default), non-thinking, and low-effort — allowing users to balance depth vs. latency on a per-query basis.
Key capabilities:
Built-in Reasoning: Native chain-of-thought that significantly improves performance on math, coding, and complex multi-step problems.
Flexible Thinking Modes: Seamlessly switch between full thinking, non-thinking, and low-effort modes within a single model.
Reasoning-Augmented Tool Calling: The model reasons about which tools to invoke and why, producing more accurate function calls.
512K Context Window: Supports long documents, multi-turn conversations, and complex agentic workflows.
Apache 2.0 Licensed: Fully open for commercial and research use.
Granite-4.2-3B is post-trained from Granite-4.1-3B-Base through a rigorous multi-stage pipeline that progressively unlocks reasoning, tool use, and instruction-following capabilities. A full listing of training datasets is available in the Granite 4.2 GitHub repository. The training pipeline consists of three stages:
Granite-4.2-3B builds on Granite-4.1-3B-Base, which was pre-trained on a large-scale English as well as multilingual corpus. For full pre-training details (data composition, training recipe, and infrastructure), refer to our Granite 4.1 Technical Blog.
The SFT stage draws on instruction-following, chain-of-thought, and reasoning data to cultivate the model's reasoning and thinking abilities. For all the three, 3B, 8B and 30B models, the training corpus comprises four sources: (1) publicly available datasets under permissive licenses, (2) internally generated synthetic data targeting reasoning, tool calling, and chain-of-thought capabilities, (3) agentic traces collected across a diverse range of tasks, and (4) a curated selection of human-authored data. Hyperparameters were tuned before training was scaled to all three model sizes. For the 30B model, we conducted a second SFT phase, in which the agentic data was up-sampled while a smaller share of general replay data was retained. This phase trained for a single epoch, starting from a lower learning rate than Phase 1.
The final stage of training applies multi-phase, multi-environment reinforcement learning using Group Relative Policy Optimization (GRPO). Training spans a broad mix of environments including math, code, science, instruction following, tool use, general chat and structured output. Most environments provide verifiable rewards, while open-ended prompts are scored by a generative reward model. Training runs asynchronously: generation and policy updates occupy separate GPU pools rather than proceeding in lockstep, and weights are refreshed in flight.
After the reward-driven phases, a preference-alignment (RLHF) phase tunes helpfulness, conversational quality, and safety. Reinforcement learning is carried out with NeMo RL, and the RL environments run on NeMo Gym.
Infrastructure: We trained the Granite 4.2 Language Models utilizing an NVIDIA GB200 NVL72 cluster hosted in CoreWeave. Intra-rack communication occurs via the 72-GPU NVLink domain, and a non-blocking, full Fat-Tree NDR 400 Gb/s InfiniBand network provides inter-rack communication.
Thinking enabled — The generation prompt ends with <|im_start|>assistant\n<think>\n, causing the model to reason until it emits </think>, then produce the final answer.
Thinking disabled — The prompt ends with <|im_start|>assistant\n<think></think>, bypassing reasoning entirely.
Low-effort — Appends {reasoning effort: low} to the user message, signaling shorter reasoning chains.
In multi-turn conversations, thinking content from previous assistant turns is automatically stripped (truncate_history_thinking=True by default) to conserve context window space. Only the current generation produces full reasoning. Set truncate_history_thinking=False to preserve full reasoning history.
Granite-4.2-3B is optimized for deployment with vLLM.
Reasoning parser: Use the custom granite_thinking_parser included in this repository (requires vLLM v0.20+). The model also works with the built-in nemotron_v3 parser, but granite_thinking_parser provides better formatting of reasoning output. Native support for granite_thinking_parser will be added to vLLM and SGLang very soon. Tool calling parser: Use qwen3_coder.
Granite-4.2-3B can also be served with SGLang (v0.5.18+) for high-throughput inference.
Reasoning parser: Use --reasoning-parser auto, which resolves to the built-in nemotron_3 parser for this checkpoint. It separates the thinking trace into reasoning_content and the final answer into content, and it handles all three thinking modes (enable_thinking=True/False, low_effort=True) described in Thinking Modes. Tool calling parser: Use --tool-call-parser auto, which resolves to qwen3_coder for this checkpoint.
Granite-4.2-3B can be used as the backbone model for agentic coding tools. Since it supports reasoning and tool calling via the OpenAI-compatible API, it integrates with popular agentic harnesses out of the box. Start the vLLM server as shown in the Serving with vLLM section above, then follow the harness-specific instructions below.
Note: The openai/ prefix is required when connecting to OpenAI-compatible endpoints like vLLM. Refer to the OpenHands local LLM documentation for detailed setup instructions, troubleshooting, and alternative installation methods.
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_path = "ibm-granite/granite-4.2-3b"
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForCausalLM.from_pretrained(model_path, device_map="cuda", torch_dtype=torch.bfloat16)
model.eval()
messages = [
{"role": "user", "content": "How many r's are in the word 'strawberry'?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
with torch.no_grad():
output = model.generate(**inputs, max_new_tokens=8192, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
Example Output
<think>
Okay, let's see. The problem is to find how many 'r's are in the word 'strawberry'.
First, I need to write out the word: s t r a w b e r r y.
Now, I need to count the number of 'r' letters. Let's list each letter and check for 'r'.
1. s – not r
2. t – not r
3. r – yes, that's one
4. a – no
5. w – no
6. b – no
7. e – no
8. r – yes, that's two
9. r – yes, that's three
10. y – no
Total r's = 3.
</think>
There are **3** r's in the word "strawberry".<|im_end|>
Granite-4.2-3B supports tool calling with integrated reasoning — the model thinks about which tool to call and why before making the call. Tools are defined using the OpenAI function definition schema.
tools = [
{
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather for a specified city.",
"parameters": {
"type": "object",
"properties": {
"city": {"type": "string", "description": "Name of the city"}
},
"required": ["city"]
}
}
}
]
messages = [
{"role": "user", "content": "What's the weather like in Boston right now?"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, tools=tools,
add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=4096, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
Example Output
<think>
Okay, the user is asking for the weather in Boston right now. Let me check the tools
available. There's a function called get_current_weather that takes a city parameter.
I need to call that with the city set to Boston.
</think>
<tool_call>
<function=get_current_weather>
<parameter=city>
Boston
</parameter>
</function>
</tool_call>
<|im_end|>
messages = [
{"role": "user", "content": "What's the weather like in Boston right now?"},
{"role": "assistant", "content": "<think>\nThe user wants to know the current weather in Boston. I should call get_current_weather.\n</think>",
"tool_calls": [{"function": {"name": "get_current_weather", "arguments": {"city": "Boston"}}}]},
{"role": "tool", "content": '{"temperature": "72°F", "condition": "Partly cloudy", "humidity": "65%"}'},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, tools=tools,
add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
output = model.generate(**inputs, max_new_tokens=4096, temperature=1.0, top_p=0.95, do_sample=True)
print(tokenizer.decode(output[0][inputs.input_ids.shape[-1]:], skip_special_tokens=False))
Example Output
<think>
The tool returned the weather data for Boston: temperature 72°F, partly cloudy, humidity 65%.
I need to present this information clearly to the user.
</think>
The current weather in Boston is 72°F, partly cloudy, with 65% humidity.<|im_end|>
Granite 4.2 models are primarily finetuned using instruction-response pairs mostly in English, but also multilingual data covering the supported languages listed above. Although this model handles multilingual dialog, its performance may vary compared to English. Few-shot examples can help in such cases.
While aligned for safety, the model may occasionally produce inaccurate, biased, or unsafe responses. The content within <think>...</think> tags represents internal reasoning and may contain unpolished or intermediate thoughts that do not represent final conclusions.
To enhance safety in deployments, we recommend using Granite 4.2 alongside Granite Guardian to detect and flag risks across key dimensions outlined in the IBM AI Risk Atlas.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("ibm-granite/granite-4.2-3b", revision="4e9c275e232f10211ec88644d106a1f6a1b60e06")
model = AutoModelForCausalLM.from_pretrained("ibm-granite/granite-4.2-3b", revision="4e9c275e232f10211ec88644d106a1f6a1b60e06", device_map="auto")
