Recursive Flint (40B total, ~2.5B active per token) quantized for Apple Silicon with oMLX's oQ2e: data-driven mixed precision with imatrix-calibrated clipping.
13.9 GB on disk (~2.80 bpw), down from 79.6 GB bf16. Peak memory while generating: ~14.0 GB at short context, ~15.5 GB at 16k.
Quantized and tested on a MacBook Pro M5 Max 128GB 40 GPU.
The smallest and lossiest variant. Perplexity is about double bf16's (29 vs 14.3) and it agrees with bf16's top token only half the time. It still answered my short probes coherently, but expect noticeably more mistakes on real work. Use it only if nothing larger fits (16 GB Macs); otherwise prefer oQ2.5e or oQ2.7e. It is ~4% faster than oQ4e.
oQ level 2, enhanced (oQ2e), group size 64, affine. The allocator measured each layer's sensitivity on the bf16 weights and puts every routed expert at 2 bits and keeps everything read on every token at 8 bits: attention, the shared expert, the dense layers, embed_tokens and lm_head. Those 8-bit floors are why the real average lands at ~2.8 bpw, not 2. The "e" adds an imatrix pass (128 calibration samples) that picks the clipping range of each group by activation importance; it doesn't change which tensor gets how many bits, only the scales inside them.
On oQ4e I also tried group size 128 for every tensor. It saved 1.2 GB but lost everything the imatrix bought (mean KL 0.263 vs 0.222) and wasn't faster, so every variant uses 64.
Built with oMLX e2396c7 (2026-10-27). Output is standard MLX affine quantization: no custom kernels needed to load it.
How closely each variant tracks the bf16 model's next-token distribution, measured over 64 × 2048-token chunks (65,472 scored positions, second half of each chunk), KL truncated to the bf16 top-256.
The corpus is English prose from oMLX's bench set, which isn't guaranteed disjoint from oQ's calibration data. That makes the numbers good for ranking these variants against each other, not as a held-out quality score. Truncating to top-256 drops ~0.9% of the probability mass and biases KL slightly low for every row alike. I haven't run task benchmarks yet.
Generation tok/s, greedy decode, 256 tokens, median of 3 runs, oMLX's fused Flint decode path enabled, MacBook Pro M5 Max 128GB 40 GPU.
Variant
~30-token context
4k
16k
Peak memory
oQ8e
110
103
95
42.4–43.9 GB
oQ6e
108
96
87
32.9–34.4 GB
oQ5e
103
97
91
28.2–29.7 GB
oQ4e
120
110
105
23.5–25.0 GB
oQ3e
109
106
98
18.5–20.0 GB
oQ2.7e
123
112
105
16.3–17.8 GB
oQ2.5e
117
106
99
15.6–17.1 GB
oQ2e (this repo)
130
124
113
14.0–15.5 GB
Each variant was timed against oQ4e in the same session; across sessions the absolute numbers drift by up to ~10%, so read differences under that as noise. Below oQ4e, fewer bits buy memory but not speed: at ~2.5B active parameters, decode is mostly bound by kernel-launch overhead. Above it, oQ5e, oQ6e and oQ8e are 5–15% slower.
oQ quantizes Flint through mlx-vlm's model class, so the weights are stored under a language_model. prefix. oMLX and mlx-vlm load them as-is; mlx-lm's own flint model expects unprefixed names.
oMLX ships a fused MoE routing/combine kernel for one-token Flint decode, worth ~16% generation speed in my runs on oQ4 (~108 → ~125 tok/s at short context). It's on by default in oMLX builds from 2026-10-27 onward.
oMLX (recommended): download the repo and it shows up as mlx-community--flint-oQ2e. The server exposes an OpenAI-compatible API with tool calls and returns reasoning in reasoning_content.
hf download mlx-community/flint-oQ2e
omlx serve
mlx-vlm:
pip install -U "mlx-vlm>=0.7.10"
mlx_vlm.generate --model mlx-community/flint-oQ2e \
--prompt "Write a Python function that merges overlapping intervals." \
--max-tokens 4096 --temperature 1.0
Recursive recommends temperature 1.0, top-p 0.95, top-k 20, and an output budget of at least 32k tokens for hard problems, since reasoning and answer share it. Keep the returned reasoning in the conversation history across turns. The chat template is the upstream one, unchanged.
Apache 2.0, same as the original weights. Flint is developed by Recursive; this repo only changes the weight precision. Recursive's hosted API terms are separate from the weight license.
Files
13 files · 13.9 GBWeights 13.9 GB (safetensors). Sizes and sha256 from the signed manifest.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("mlx-community/flint-oQ2e", revision="547e7872f338f5e25df14b55a1e9ce6a048fe08a")
model = AutoModelForCausalLM.from_pretrained("mlx-community/flint-oQ2e", revision="547e7872f338f5e25df14b55a1e9ce6a048fe08a", device_map="auto")
mlx-community <huggingface.co/mlx-community>Newsgroups: alt.freeweights.textSubject: 
