The config.json and tokenizer_config.json files in this repository have been updated since the initial release. If you downloaded MiMo-V2.5 before this commit (4da2748), please re-pull or manually update these two files to ensure correct model behavior. Using the outdated config may lead to degraded model performance. We apologize for any inconvenience.
MiMo-V2.5 is a native omnimodal model with strong agentic capabilities, supporting text, image, video, and audio understanding within a unified architecture. Built upon the MiMo-V2-Flash backbone and extended with dedicated vision and audio encoders, it delivers robust performance across multimodal perception, long-context reasoning, and agentic workflows. Key features include:
Hybrid Attention Architecture: Inherits the hybrid design from MiMo-V2-Flash, interleaving Sliding Window Attention (SWA) and Global Attention (GA) with a 5:1 ratio and 128 sliding window. This reduces KV-cache storage by nearly 6× while maintaining long-context performance via learnable attention sink bias.
Native Omnimodal Encoders: Equipped with a 729M-param Vision Transformer (ViT) featuring hybrid window attention and a dedicated audio encoder initialized from the weights of MiMo-Audio, enabling high-quality image, video, and audio understanding.
Multi-Token Prediction (MTP): Three lightweight MTP modules with dense FFNs accelerate inference via speculative decoding and improve RL training efficiency.
Efficient Pre-Training: Trained on a total of ~48T tokens using FP8 mixed precision. The context window supports up to 1M tokens.
Agentic Capabilities: Post-training incorporates SFT, large-scale agentic RL, and Multi-Teacher On-Policy Distillation (MOPD), achieving strong performance on agentic tasks and multimodal understanding benchmarks.
Text Pre-training: We collect diverse text data for pre-training the LLM backbone.
Projector Warmup: Short-duration warmup of multimodal projectors (audio and visual MLP projectors).
Multimodal Pre-training: High-quality multimodal data collected for large-scale pretraining.
SFT & Agentic Post Training: Supervised fine-tuning with diverse agentic data. During this stage, the context window is progressively extended from 32K → 256K → 1M.
RL & MOPD Training: Reinforcement learning for improving perception, reasoning, and agentic capabilities.
Since inference engines are continuously being updated and optimized, this guide only provides deployment examples for reference. For the best performance, we strongly recommend following our referenced approach to get the latest best practices and optimal performance.
For the best performance, we strongly recommend deploying using this approach, which is officially supported by the SGLang community. Please refer to SGLang MiMo-V2.5 Cookbook for the latest deployment guide.
The following is an example of running the model with SGLang, referenced from sgl-project/sglang#23811:
For the best performance, we strongly recommend deploying using this approach, which is officially supported by the vLLM community. Please refer to vLLM MiMo-V2-Flash Cookbook for the latest deployment guide.
For local deployment, we recommend setting the sampling parameters to temperature=1.0, top_p=0.95.
You are solely responsible for your use of XiaomiMiMo/MiMo-V2.5 and any outputs, actions, or consequences arising therefrom, and you agree not to use the XiaomiMiMo/MiMo-V2.5 or any derivatives thereof:
For any unlawful, fraudulent, or malicious purpose, or in any manner that violates any applicable laws or regulations；
To infringe upon the intellectual property rights, privacy rights, publicity rights, or other lawful rights or interests of any third party;
To exploit, harm, harass, defame, unlawfully discriminate against, or otherwise adversely affect any individual or group, including minors or vulnerable persons;
For any military purpose or application;
To engage in or facilitate malicious cyber activities, including but not limited to malware, ransomware, phishing, credential theft, or unauthorized access, or any activity intended to compromise or unlawfully control computer systems, networks, or encryption systems;
To access, collect, process, disclose, or disseminate personal data or other protected information without appropriate authorization or lawful basis; or to access, interact with, scrape, or automate any third-party platform or service in violation of any applicable laws, policies, or authorization requirements;
To use the XiaomiMiMo/MiMo-V2.5 in a manner that autonomously executes high-risk actions without appropriate human oversight or authorization.
This notice is an informational reminder only and does not amend the applicable model licence or impose additional licence conditions. The permissions granted under that licence remain unaffected. Nothing in this notice restricts uses permitted by applicable law, including applicable exceptions and limitations.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("XiaomiMiMo/MiMo-V2.5", revision="d6bf9db68673d1f903a53d8da417b293fdc97d1f")
model = AutoModelForCausalLM.from_pretrained("XiaomiMiMo/MiMo-V2.5", revision="d6bf9db68673d1f903a53d8da417b293fdc97d1f", device_map="auto")
XiaomiMiMo <huggingface.co/XiaomiMiMo>Newsgroups: alt.freeweights.mimoSubject: 
mlx-community