oQ6e (imatrix-enhanced mixed-precision 6-bit) conversion of IFM/K2-Horizon-7B, a dense reasoning model from the K2 Horizon family (7B parameters, 512K context).
Upstream model:IFM/K2-Horizon-7B by the IFM Team, released under Apache-2.0.
Conversion: Quantized to MLX format using Hermes Agent with mlx-lm and oMLX.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("mlx-community/K2-Horizon-7B-oQ6e", revision="2cca82ec88e24bd5fc264e4bcd29922182c8f62f")
model = AutoModelForCausalLM.from_pretrained("mlx-community/K2-Horizon-7B-oQ6e", revision="2cca82ec88e24bd5fc264e4bcd29922182c8f62f", device_map="auto")