Spark-X2.6-4B-OptiQ-4bit

mlx-community/Spark-X2.6-4B-OptiQ-4bit#

Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs

A mixed-precision MLX quant of XHToken/Spark-X2.6-4B, the on-device agent model that follows Spark-X2.5 with improved post-training for coding, tool use, long context and reasoning. 2.8 GB on disk, down from 7.7 GB bf16. See the original card for what the model is trained to do and how XHToken evaluates it.

Spark is a new architecture for MLX, and mlx-optiq adds support for it, so import optiq before mlx_lm.load: stock mlx-lm does not know the spark2_5 model type.

What it is#

Following llama.cpp's naming for mixed quants, the "4bit" label denotes the predominant precision, not the weighted average.

How the bit-widths were chosen#

The per-layer allocation was not measured on this model. It was transferred from mlx-community/Spark-X2.5-4B-OptiQ-4bit, whose allocation came from a KL-divergence sensitivity sweep over a six-domain calibration mix. Spark-X2.6 keeps exactly the same architecture and modeling code as X2.5, and all 217 tensors matched with none unmatched.

Checked after conversion#

Run through this quant and its bf16 parent on Apple Silicon with greedy decoding:

The one miss is bare mental arithmetic with thinking off, where the quant answers in a single token with no working; the Spark-X2.5 quant makes the same slip. With thinking on, the default, it gets it right.

Usage#

pip install mlx-optiq   # brings mlx-lm and the spark2_5 architecture

import optiq  # registers spark2_5 with mlx-lm
from mlx_lm import load, generate

model, tokenizer = load("mlx-community/Spark-X2.6-4B-OptiQ-4bit")
prompt = tokenizer.apply_chat_template(
    [{"role": "user", "content": "Explain quantum computing in simple terms."}],
    tokenize=False, add_generation_prompt=True, enable_thinking=False)
print(generate(model, tokenizer, prompt=prompt, max_tokens=300))

Serving, with the mixed-precision KV cache, tool calling and the OpenAI, Responses and Anthropic Messages APIs:

optiq serve --model mlx-community/Spark-X2.6-4B-OptiQ-4bit --kv-config kv_config.json

Thinking and sampling#

Thinking is on by default. The chat template takes reasoning_effort of high (the default) or low to set how much it thinks, or enable_thinking: false to turn it off; pass them as chat_template_kwargs at the endpoint or as keywords to apply_chat_template. Through optiq serve, the standard reasoning_effort: "none" also turns it off. XHToken's sampling defaults, temperature 1.0 and top_p 0.95, are bundled in generation_config.json.

See the Spark family guide for serving flags and family caveats.

Links#

• Project website: mlx-optiq.com
• Spark family guide: mlx-optiq.com/docs/spark
• PyPI: pypi.org/project/mlx-optiq
• Base model: XHToken/Spark-X2.6-4B