MLX-format INT4 weights for Recursive Flint, primarily intended for local inference on Apple Silicon. Projections and experts use group-64 affine INT4; embeddings use INT8. The weights occupy approximately 22.6 GB.
See Flint’s model card for model details, benchmarks, chat conventions, and limitations.
Flint’s training data included publicly available and crawled web documents, text and code, commercially licensed data, and AI-generated synthetic data. Post-training included instruction tuning, reinforcement learning and human-preference data. See the base Flint model card for shared model and training information.
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("RecursiveSI/flint-int4", revision="3b93f9053b530488405ae9fca51d4900d6ce21f0")
model = AutoModelForCausalLM.from_pretrained("RecursiveSI/flint-int4", revision="3b93f9053b530488405ae9fca51d4900d6ce21f0", device_map="auto")