Built with mlx-optiq, the MLX-native toolkit to quantize, fine-tune, and serve LLMs locally on Apple Silicon, no PyTorch and no cloud. Try the Lab · All OptiQ quants · Docs
A mixed-precision MLX quant of RecursiveSI/flint, Recursive's 40B-A2.5B mixture-of-experts model for coding agents. 25.6 GB on disk, down from about 80 GB bf16. Per-layer bit-widths come from a KL-divergence sensitivity pass on a six-domain calibration mix: sensitive layers get 8 bits, robust ones 4. See the original card for what the model is trained to do and how Recursive evaluates it.
Stock mlx-lm has no flint class yet. OptiQ ships an MLX port of the architecture that registers with mlx-lm on import optiq. It was checked against Recursive's SGLang serving kit: learned attention sinks, a 513-token sliding window on 35 of the 49 layers, no positional encoding on the full-attention layers, and sigmoid top-6 routing over 256 experts plus a shared expert.
pip install "mlx-optiq>=0.7.2"
import optiq # registers the flint arch with mlx-lm
from mlx_lm import load, generate
model, tok = load("mlx-community/flint-OptiQ-4bit")
prompt = tok.apply_chat_template(
[{"role": "user", "content": "Write a Python function that merges overlapping intervals."}],
tokenize=False, add_generation_prompt=True,
)
print(generate(model, tok, prompt=prompt, max_tokens=4096))
Flint reasons before it answers, so give it a generous max_tokens. Recursive recommends temperature 1.0, top-p 0.95 and top-k 20. For an OpenAI- and Anthropic-compatible server with tool calls and prompt caching:
The weights take 26 GB of unified memory. It was built and scored on a 36 GB Mac, where long prompts leave little headroom; 48 GB or more is comfortable.
Six-metric Capability Score: the unweighted mean of MMLU, GSM8K, IFEval, BFCL, HumanEval and HashHop. Scored in reasoning mode (generative MMLU and a large thinking budget), the setting that measures an always-on thinking model fairly.
Metric
OptiQ-4bit
MMLU (reasoning, 969 samples)
86.4%
GSM8K (1000 samples)
92.6%
IFEval (full set, strict)
90.2%
BFCL-V3 simple (200 calls)
87.0%
HumanEval (164 problems, pass@1)
89.0%
HashHop (long-context retrieval)
72.0%
Capability Score (mean of 6)
86.20
Every metric gets one equal vote. See the eval-framework writeup for the full methodology.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("mlx-community/flint-OptiQ-4bit", revision="d5efe414b30ee0cf2042ec712321a6fa060c2a53")
model = AutoModelForCausalLM.from_pretrained("mlx-community/flint-OptiQ-4bit", revision="d5efe414b30ee0cf2042ec712321a6fa060c2a53", device_map="auto")
mlx-community <huggingface.co/mlx-community>Newsgroups: alt.freeweights.textSubject: 
