MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B

mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B#

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. All OptiQ quants · Docs

19B parameters at 13.5 GB, down from 35B at 24.4 GB, trading Capability Score 87.61 for 81.91.

Half the routed experts are gone. Each of the 40 language layers kept 128 of its 256 experts; top-8 routing is unchanged, so every token still activates 8 experts plus the shared one. The retained experts were copied bit-for-bit from the parent quant, nothing re-quantized or retrained. The cost is uneven: MMLU falls from 86.2 to 63.8, a 22.4 point drop, while GSM8K holds at 90.0, IFEval at 79.9, BFCL at 92.0, and HumanEval at 87.8. Knowledge questions suffer; skills largely survive. Pick it over the parent on a 24 GB Mac, where the parent doesn't fit in memory; pick the parent if knowledge questions matter.

It works on image and text#

Both ran through this quantized model with optiq serve, MLX only:

Image (the words "OPTIQ 42" on a white background, "What text is in the image?"):

OPTIQ 42

Text ("What is the capital of France? One word."):

Paris

What pruning costs#

Details#

Run it#

MiniCPM-V 4.7 needs OptiQ 0.5.25 or later. Stock mlx-lm does not know the minicpmv4_7 architecture, so import optiq registers it, and OptiQ loads the vision sidecar.

pip install "mlx-optiq>=0.5.25"

For image input, serve it with an OpenAI- and Anthropic-compatible endpoint:

optiq serve --model mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B

For the measured mixed-precision KV cache, fetch kv_config.json from this repo and pass it in:

hf download mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B kv_config.json --local-dir .
optiq serve --model mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B --kv-config kv_config.json

Then send an image as image_url content. Text-only generation also loads directly:

import optiq  # registers minicpmv4_7 and the vision sidecar
from mlx_lm import load, generate

model, tok = load("mlx-community/MiniCPM-V-4.7-35B-A3B-OptiQ-4bit-REAP-19B")
msgs = [{"role": "user", "content": "Explain why the sky is blue."}]
prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
print(generate(model, tok, prompt=prompt, max_tokens=512))

Method#

Expert pruning follows REAP (Cerebras Research, ICLR 2026, REAP the Experts: Why Pruning Prevails for One-Shot MoE Compression). OptiQ applies it in the quantized domain, directly on the quantized checkpoint with no bf16 parent, via optiq prune-experts. See the pruning docs.

Links#

• Project website: mlx-optiq.com
• Family guide: mlx-optiq.com/docs/minicpm
• All OptiQ quants: mlx-optiq.com/models
• Base model: openbmb/MiniCPM-V-4.7-35B-A3B