MiniCPM-V-4.7-1B-gguf

This repository hosts the GGUF (llama.cpp) quantized version of MiniCPM-V 4.7 1B. For the original BF16 weights and the full model card, please refer to openbmb/MiniCPM-V-4.7-1B.

GGUF files#

Two sets of files are provided. They differ only in the embedded chat template — the tensor weights are byte-identical.

Which one to use#

• Root files — thinking is disabled inside the embedded template and cannot be turned on. This is the recommended default for this model, which is not trained as a reasoning model.
• thinking/ files — the official upstream template is restored, so the model emits a [Start thinking] … [End thinking] trace before answering. Useful when you want to see the reasoning steps; note that it multiplies the token count, so set a larger -n.

One-click (macOS / Windows) — install Llama, the desktop app from the llama.cpp team and Hugging Face (brew install --cask llama-app, or the Windows installer), then choose Use this model → Llama on this page. Llama downloads a build that fits your machine and opens the chat window.

Or from the terminal — -hf pulls the model and the matching mmproj automatically:

llama cli   -hf openbmb/MiniCPM-V-4.7-1B-gguf
llama serve -hf openbmb/MiniCPM-V-4.7-1B-gguf     # WebUI + OpenAI-compatible API on :9931

llama-mtmd-cli and llama-server remain available for development; llama cli / llama serve are the normal entrypoints. To use a local file instead, pick a quant above and pass -m with the shared mmproj:

# default (thinking disabled, no flag needed)
llama cli -m MiniCPM-V-4_7-1B-Q4_K_M.gguf --mmproj mmproj-model-f16.gguf --image img.jpg -p "Describe."

# thinking enabled
llama cli -m thinking/MiniCPM-V-4_7-1B-Q4_K_M-thinking.gguf --mmproj mmproj-model-f16.gguf \
          --image img.jpg -p "Describe." -n 1024

llama.cpp always injects enable_thinking into the chat template, so --reasoning off has no effect on the root files; on the thinking/ files it disables the trace. mmproj files are shared by both sets — they carry no chat template.

MiniCPM-V 4.7#

GitHub · Evaluation · Examples · Usage · Model weights · DSpark-gguf

MiniCPM-V 4.7 is the latest model in the MiniCPM-V series, available in a 1B variant. It builds on the family's single-image, multi-image, and video understanding capabilities, combining stronger foundation capabilities with efficient visual encoding and on-device inference. Notable features of MiniCPM-V 4.7 include:

• 🔥 A New Pareto Frontier in Performance and Efficiency. As model capabilities continue to advance, inference efficiency has become an important measure of a model's value. On resource-constrained devices, delivering the strongest possible capabilities within a limited resource budget calls for higher capability density. The 1B variant of MiniCPM-V 4.7 advances the performance–efficiency Pareto frontier for MLLMs. In Instruct mode, the 1B variant continues to improve foundation capabilities over MiniCPM-V 4.6 and substantially outperforms Qwen3.5 0.8B in both overall score and inference efficiency.
• 🚀 Efficient Architecture. Based on the intra-ViT early compression technique from LLaVA-UHD v4, MiniCPM-V 4.7 compresses visual tokens early within the vision encoder, reducing computation in subsequent encoder layers. It also supports mixed 4x/16x visual token compression, allowing developers to balance visual detail and inference cost across tasks.
• 💻 Broad Platform and Chip Support. MiniCPM-V 4.7 supports iOS, Android, and HarmonyOS, together with open-source on-device deployment support. It also provides day-0 support for chip platforms from Rockchip, Houmo, Intel, AMD, and Arm, extending deployment options across mobile and local computing devices.
• 🛠️ Developer Friendly. MiniCPM-V 4.7 supports inference frameworks including SGLang, vLLM, llama.cpp, and Ollama, along with fine-tuning frameworks including SWIFT and LLaMA-Factory. It provides GGUF, BNB, AWQ, and GPTQ quantized variants and adds FP8 quantization support. We also release Self-Replay Fine-Tuning (SRFT), a fine-tuning approach that helps developers extend model capabilities at low cost while reducing forgetting of existing capabilities, making further customization easier.

Get started#

Model weights  ·  Inference  ·  Fine-tuning  ·  Live demo  ·  Local deployment  ·  API  ·  Cookbook

Evaluation#

Significantly Advancing the Performance–Efficiency Pareto Frontier

The 1B variant of MiniCPM-V 4.7 advances the performance–efficiency Pareto frontier. In Instruct mode, 1B substantially outperforms Qwen3.5 0.8B in both overall score and inference efficiency.

Performance versus inference time for MiniCPM-V 4.7 1B

Overall Performance (1B, Instruct)

Full performance comparison for MiniCPM-V 4.7 1B in Instruct mode

Examples#

MiniCPM-V 4.7 can run on phones and tablets. The clips below are unedited screen recordings captured directly on the devices.

| 1B iPhone 17 Pro Max

| Image: image

Usages#

Inference with Transformers#

Click to expand the Transformers inference examples.

Installation#

pip install "transformers[torch]>=5.18.0" torchvision torchcodec

Note on CUDA compatibility: torchcodec (used for video decoding) may have compatibility issues with certain CUDA versions. For example, torch>=2.11 bundles CUDA 13.1 by default, while environments with CUDA 12.x may encounter errors such as RuntimeError: Could not load libtorchcodec. Two workarounds:

1. Replace torchcodec with PyAV — supports both image and video inference without CUDA version constraints:

   pip install "transformers[torch]>=5.18.0" torchvision av

Copy

1. Pin the CUDA version when installing torch to match your environment (e.g. CUDA 12.8):

   pip install "transformers>=5.18.0" torchvision torchcodec --index-url https://download.pytorch.org/whl/cu128

Copy

Load the Model#

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "openbmb/MiniCPM-V-4.7-1B"

processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForImageTextToText.from_pretrained(
    model_id, torch_dtype="auto", device_map="auto"
)

# Flash Attention 2 is recommended for faster inference and lower GPU memory usage,
# especially in multi-image and video scenarios.
# model = AutoModelForImageTextToText.from_pretrained(
#     model_id,
#     torch_dtype=torch.bfloat16,
#     attn_implementation="flash_attention_2",
#     device_map="auto",
# )

Image Inference#

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/openbmb/DemoCase/resolve/main/refract.png"},
            {"type": "text", "text": "What causes this phenomenon?"},
        ],
    }
]

downsample_mode = "16x"  # Using `downsample_mode="4x"` for Finer Detail

inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
    downsample_mode=downsample_mode,
    max_slice_nums=36,
    enable_thinking=False,
).to(model.device)

generated_ids = model.generate(
    **inputs,
    downsample_mode=downsample_mode,
    max_new_tokens=512,
    # recommended instruct-mode sampling parameters
    do_sample=True,
    temperature=0.7,
    top_p=0.95,
    top_k=100,
    min_p=0.0,
    presence_penalty=0,
    repetition_penalty=1.03,
)
generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])

Video Inference#

messages = [
    {
        "role": "user",
        "content": [
            {"type": "video", "url": "https://huggingface.co/datasets/openbmb/DemoCase/resolve/main/football.mp4"},
            {"type": "text", "text": "Describe this video in detail. Follow the timeline and focus on on-screen text, interface changes, main actions, and scene changes."},
        ],
    }
]

downsample_mode = "16x"  # Using `downsample_mode="4x"` for Finer Detail

inputs = processor.apply_chat_template(
    messages, tokenize=True, add_generation_prompt=True,
    return_dict=True, return_tensors="pt",
    downsample_mode=downsample_mode,
    max_num_frames=128,
    stack_frames=1,
    max_slice_nums=1,
    use_image_id=False,
    enable_thinking=False,
).to(model.device)

generated_ids = model.generate(
    **inputs,
    downsample_mode=downsample_mode,
    max_new_tokens=2048,
    # recommended instruct-mode sampling parameters
    do_sample=True,
    temperature=0.7,
    top_p=0.95,
    top_k=100,
    min_p=0.0,
    presence_penalty=0,
    repetition_penalty=1.03,
)
generated_ids_trimmed = [
    out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
]
output_text = processor.batch_decode(
    generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
)
print(output_text[0])

Advanced Parameters#

You can customize image/video processing by passing additional parameters to apply_chat_template:

Note: downsample_mode must be passed to both apply_chat_template (for correct placeholder count) and generate (for the vision encoder). All other parameters only need to be passed to apply_chat_template.

Serving with transformers serve#

Hugging Face Transformers includes a lightweight OpenAI-compatible server for quick testing and moderate-load deployment.

pip install "transformers[serving]>=5.18.0"

Start the server:

transformers serve openbmb/MiniCPM-V-4.7-1B --port 8000 --host 0.0.0.0

Send a request:

curl -s http://localhost:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "openbmb/MiniCPM-V-4.7-1B",
    "messages": [{
      "role": "user",
      "content": [
        {"type": "image_url", "image_url": {"url": "https://huggingface.co/datasets/openbmb/DemoCase/resolve/main/refract.png"}},
        {"type": "text", "text": "What causes this phenomenon?"}
      ]
    }]
  }'

Tool calling example:

curl -s http://localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "openbmb/MiniCPM-V-4.7-1B",
  "messages": [{"role": "user", "content": [
    {"type": "text", "text": "the weather of Beijing"}
  ]}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "get_weather",
      "description": "Get the current weather for a given location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {"type": "string", "description": "City name"}
        },
        "required": ["location"]
      }
    }
  }]
}'

Recommended Sampling Parameters#

We recommend using the following set of sampling parameters for generation in instruct mode: temperature=0.7, top_p=0.95, top_k=100, min_p=0.0, presence_penalty=0, repetition_penalty=1.03.

Speculative decoding (DSpark)#

Click to show speculative decoding.

Speculative decoding (DSpark): we also release MiniCPM-V-4.7-1B-DSpark, a DSpark draft model trained for MiniCPM-V-4.7-1B.

Inference with OpenVINO™#

OpenVINO™ is an open-source toolkit for optimizing and deploying AI inference on Intel CPU/GPU/NPU platforms. OpenVINO provides day-0 support for MiniCPM-V-4.7 on Intel platforms with high-performance inference. See the OpenVINO™ Toolkit to get started. You can install and run the example with this notebook.

On-Device Deployment#

MiniCPM-V 4.7 supports deployment on iOS, Android, and HarmonyOS, with day-0 support for Rockchip, Houmo, Intel, AMD, and Arm chip platforms. See the edge deployment repository for the apps and source code.

Inference and Fine-Tuning#

For supported inference and training frameworks, see Supported Inference and Training Frameworks and API. The MiniCPM-V 4.7 Cookbook covers deployment, fine-tuning, and quantization workflows.

Task guides

• Single-image chat
• Multi-image chat
• Video understanding
• OCR
• PDF parsing
• Visual grounding

Quantization

Acknowledgements#

Click to view acknowledgements.

We would like to thank the following projects:

• Qwen3.5 and Qwen3.6 for providing the language backbone
• SigLIP2 for providing the vision understanding module
• Transformers

Model Weights#

MiniCPM-V 4.7 is available in 35B-A3B and 1B variants. Each model is paired with its DSpark speculative decoding heads, available in BF16 and GGUF.

| Model | Description | Model weights | DSpark heads

| MiniCPM‑V 4.7 35B-A3B | The first open-source MoE model to run smoothly on-device without compromising performance, raising the bar for on-device performance. | 🤗  ModelScope BF16

🤗  ModelScope FP8

🤗  ModelScope int4

🤗  ModelScope AWQ

🤗  ModelScope GPTQ

🤗  ModelScope GGUF

| 🤗  ModelScope BF16

🤗  ModelScope GGUF

| MiniCPM‑V 4.7 1B | Combines stronger capabilities with extremely high efficiency, advancing the performance–efficiency Pareto frontier for multimodal large language models. | 🤗  ModelScope BF16

🤗  ModelScope int4

🤗  ModelScope AWQ

🤗  ModelScope GPTQ

🤗  ModelScope GGUF

| 🤗  ModelScope BF16

🤗  ModelScope GGUF

The 1B int4 variant uses bitsandbytes; AWQ and GPTQ use W4A16 quantization. Quantized variants reduce memory usage; GGUF weights support CPU inference.

Also available on Ollama.