Spark-X2.5-4B-OptiQ-4bit

mlx-community/Spark-X2.5-4B-OptiQ-4bit#

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

A 4-bit mixed-precision MLX quant of XHToken/Spark-X2.5-4B, a dense 4B trained for conversation, coding, tool use and agent work, with a native 1M-token context and 200+ languages. Sensitive layers are kept at 8-bit and robust ones at 4-bit, so the on-disk size stays within a few percent of a stock uniform 4-bit quant while the parts that matter keep their precision.

Spark-X2.5 is a new architecture for MLX, and mlx-optiq adds support for it, so import optiq before mlx_lm.load: stock mlx-lm does not know the spark2_5 model type.

Quantization details#

We follow the same naming convention llama.cpp uses for Q4_K_M and similar mixed-precision quants: the "4-bit" label is for the predominant precision, not the weighted average.

Capability Score#

Measured on this quant against a uniform 4-bit quant of the same model, same harness, same prompts (MMLU 5-shot 1000, GSM8K, IFEval strict, BFCL V3 simple, HumanEval pass@1, HashHop; the Capability Score is their mean).

Usage#

pip install mlx-optiq   # brings mlx-lm and the spark2_5 architecture

import optiq  # registers spark2_5 with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Spark-X2.5-4B-OptiQ-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain quantum computing in simple terms."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=300))

Serving, with the mixed-precision KV cache, tool calling and the OpenAI, Responses and Anthropic Messages APIs:

optiq serve --model mlx-community/Spark-X2.5-4B-OptiQ-4bit --kv-config kv_config.json

Only the nine full-attention layers' cache grows with the prompt; the sliding-window layers hold a fixed 512-token window. At 4-bit that is about 9 KB per token, so a 36 GB Mac serves roughly 200k tokens of prompt under the memory-safe default cap. optiq code uses the same server as a local coding agent.

Thinking and sampling#

Thinking is on by default in the chat template: the model reasons in a <think> block first, and a small max_tokens can be spent entirely on it. Give it a few hundred tokens for a direct question, or pass enable_thinking: false (as a chat_template_kwargs entry at the endpoint, or a keyword to apply_chat_template). Upstream evaluates in thinking mode with temperature 1.0, top_p 0.95; those are bundled in generation_config.json and optiq serve applies them unless you pass your own.

See the Spark-X2.5 family guide for serving flags and family caveats.

Links#

• Project website: mlx-optiq.com
• Spark-X2.5 family guide: mlx-optiq.com/docs/spark
• PyPI: pypi.org/project/mlx-optiq
• Calibration mix: mlx-optiq.com/blog/calibration-mix
• Base model: XHToken/Spark-X2.5-4B

Quantize your own#

This quant was produced by mlx-optiq. Point it at any Hugging Face model to get the same sensitivity-aware mixed precision:

pip install mlx-optiq
optiq convert <hf-model-id> --target-bpw 5.0 --candidate-bits 4,8
optiq lab   # full local workbench: chat, compare, quantize, fine-tune

License#

Apache 2.0, inherited from the base model.