Qwen-AgentWorld-35B-A3B-oQ4

Qwen-AgentWorld-35B-A3B-oQ4#

oQ4 (data-driven mixed-precision, ≈4.6 bpw) MLX quantization of Qwen/Qwen-AgentWorld-35B-A3B, produced with oMLX's quantize_oq_streaming.

For Apple Silicon. Runs in mlx-lm, oMLX, or any MLX app. Siblings: oQ3.5 (smaller/faster, ≈3.5 bpw) and bf16 (full precision).

Notes#

• Text-only. The base checkpoint declares a vision_config and MTP heads in config.json, but ships no vision or mtp.* weights (693 tensors, 0 vision, 0 MTP) — both are vestigial skeleton inherited from the Qwen3.5 base. This quant is the faithful language model; nothing multimodal was dropped.
• ≈19 GB on disk (from ≈69 GB bf16). Peak memory ≈20 GB generating.
• Mixed-precision: per-layer bit allocation from oQ's sensitivity measurement; most weights are 4-bit-class with sensitive layers boosted.

Performance#

Measured with oMLX (Auto engine) on an M5 Max (40-core GPU, 128 GB RAM). Single request:

Continuous batching (pp1024/tg128): 1×→136 · 2×→181 · 4×→252 · 8×→325 tok/s aggregate decode (2.40× at 8 concurrent requests).

Reference: BF16 source#

Same setup, full-precision Qwen/Qwen-AgentWorld-35B-A3B:

Takeaway: oQ4 gives ≈1.8× the single-request decode throughput at ≈⅓ the memory (20 GB vs 66 GB), and scales to 2.40× under 8-way batching.

Accuracy (quick reference)#

A quick, non-representative sanity check — 100-question samples per benchmark with thinking enabled, run via oMLX's accuracy bench. Not enough to draw firm conclusions, but it gives a rough idea of quality retention across precisions.

At this sample size oQ4 tracks the BF16 source within ≈1 pp (it even edges it on MMLU-Pro — noise at 100 questions).

Usage#

mlx_lm.generate --model mlx-community/Qwen-AgentWorld-35B-A3B-oQ4 \
  --system-prompt "You are a language world model simulating a Linux terminal. Given the user's command, predict the terminal output." \
  --prompt $'Action: execute_bash\nCommand: ls -la /home/user/project/' \
  --max-tokens 512 --temp 0.6

The model uses thinking mode (<think>...</think>) by default. Recommended sampling: temperature=0.6, top_p=0.95, top_k=20. See the base model card for the seven agent domains and domain-specific system prompts.