alt.freeweights.text / GOT-OCR2_0-4bit

From:

mlx-community <huggingface.co/mlx-community>Newsgroups: alt.freeweights.textSubject:

GOT-OCR2_0-4bit

Date: 2026-11-15Commit: 0e28c87Signed: 2DC28CC7B26A283B ✓

Image text to text model with 561M parameters. Safetensors weights, 457 MB in all. Published by mlx-community under Apache-2.0.

561M params

457MBGOT-OCR2_0-4bitsafetensorspermissive

FactsParams561MSize457 MBFormatssafetensorsTaskimage text to textLicenseApache-2.0From sourcepermissive

$ fw get mlx-community/GOT-OCR2_0-4bit
# manifest sha256 2f2e783a4600… · 457 MB · 13 files
$ fw verify ./GOT-OCR2_0-4bit mlx-community/GOT-OCR2_0-4bit

From sourceHosted (signed)mlx-community/GOT-OCR2_0-4bitLast check: 2026-11-15 05:18 UTC

Fine-tune in Studio Get help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Quantized of GOT-OCR2_0.

Runs with

AIBrix BentoML CTranslate2 ExLlamaV2 ExLlamaV3 exo fastllm FlashInfer KTransformers LitServe LMCache LMDeploy LocalAI mistral.rs MLC LLM Mooncake

SGLang Text Generation WebUI tinygrad

Transformers Triton Inference Server

vLLM XinferenceFine-tune with

Accelerate

Alignment Handbook

Axolotl Colossal-AI DeepSpeed EasyR1 FlashAttention H2O LLM Studio JAX Keras Liger Kernel LitGPT LLaMA-Factory Megatron-LM ms-swift

nanotron

OLMo-core OpenRLHF Oumi

PEFT PyTorch Lightning sd-scripts SimpleTuner TorchTitan torchtune Transformer Engine

TRL

Unsloth verlConvert withAQLM AutoRound Core ML Tools ExecuTorch GPTQModel

LLM Compressor mergekit Neural Compressor

Olive ONNX

Optimum Quanto

Optimum

safetensors TensorRT Model Optimizer torchaoEvaluate withAutoevals DeepEval

Evaluate HELM

LightEval lm-evaluation-harness OpenCompass Opik Ragas

simple-evals SWE-bench TruLens WeaveScan withAgentDojo AI Fairness 360 DeepTeam Detoxify Fairlearn garak Giskard Guardrails MCP Scanner ModelScan NeMo Guardrails picklescan promptfooWorks with

Agent Development Kit AG2 Google AI Edge Gallery Aider any-llm AnythingLLM BAML browser-use CAMEL

Hugging Face Chat UI Chatbox Cherry Studio Cline

Codex CLI Cognee CrewAI DeerFlow docTR DSPy EasyOCR Enchanted Gemini CLI GitHub MCP Server goose GPT Researcher Gradio Graphiti Guidance Hollama Instructor Kilo Code Langflow LangGraph LiteLLM LlamaIndex LLM LocalAGI LoLLMs WebUI MCP Python SDK Mem0 MetaGPT NextChat

Ollama JavaScript

Ollama Python

olmOCR

OpenAI Agents SDK opencode OpenHands oterm Outlines Page Assist

Playwright MCP Portkey AI Gateway PrivateGPT Qwen Code RWKV Runner

Semantic Kernel

smolagents Strands Agents Surya Vane XGrammar

Overview Model card Files Use it License Provenance Model tree

Overview

Size: 457 MB
Weights: 454 MB (safetensors)
Parameters: 561M
Formats: safetensors
Task: image text to text
License: Apache-2.0 (permissive)
Files: 13, each with a sha256
Publisher: mlx-community
Added: 2026-11-15
Section: alt.freeweights.text

Lineage

Original release from the publisher.

Base model: stepfun-ai/GOT-OCR2_0

Where these bytes come from

Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources

Badge	Host	Last check	Result
From source	huggingface.co	2026-11-15 05:18 UTC	passed

Model card

From the publisher at commit 0e28c87. Shown as plain text with links; images open at the publisher.

mlx-community/GOT-OCR2_0-4bit#

4-bit MLX quantization of stepfun-ai/GOT-OCR2_0, the 560M-parameter OCR-2.0 model. Converted with mlx-vlm 0.6.14 (mlx 0.32.0) for inference on Apple Silicon.

Fidelity was measured against the unquantized bf16 source and against the reference PyTorch implementation, not assumed. Every number below comes from a run recorded in this card.

Requires mlx-vlm with GOT-OCR 2.0 support, which is currently open as Blaizzy/mlx-vlm#1908. The PR is not merged yet, so a released mlx-vlm cannot load this repo.

python -m mlx_vlm generate \
    --model mlx-community/GOT-OCR2_0-4bit \
    --image document.png \
    --prompt "OCR: " \
    --max-tokens 1024

GOT is not a chat model. It takes two instructions: OCR: for plain text and OCR with format: for structured output (tables, formulas, sheet music). Any other prompt is out of distribution.

Quantization#


Bits	4
Group size	64
Mode	affine
Effective bits/weight	6.522
Size on disk	457 MB
Quantized tensors	169 language-model tensors
Vision tower	bf16, unquantized (0 `.scales` under `vision_tower`)
Projector	bf16, unquantized

The effective figure sits well above the nominal 4 because mlx-vlm quantizes only the language model. The vision tower and the projector are 96.7M of the 560.5M parameters, 17%, and stay bf16 by design. Packing checks out exactly: U32 x (32/4) = 463,785,984, the quantized parameter count.

Tied embeddings are 155.5M of those 463.8M, 34%, much higher than in a typical conversion where the transformer body dominates. They are the worst-hit tensor in both quantizations, which is what the per-tensor numbers below show.

Fidelity#

Weight level, against the bf16 source#


Weight SNR	20.53 dB
Tensors compared	169
Parameters compared	463,785,984
Worst tensor	`embed_tokens` (19.10 dB)

Exact, prompt-free and hardware-independent: the quantized weights are dequantized and compared element-wise against the source.

Against the reference PyTorch implementation#

One forward pass, same image and same prompt (the MPT conversation the model was trained with), comparing the final-position logits over all 151,860 classes.

Comparison	max abs error	cosine	argmax
MLX bf16 vs torch fp32	0.47469	0.99979109	same, top-5 identical
MLX fp32 vs torch fp32	0.000130	1.0000000000	same

The second row is the one that matters: the source weights are bf16, so running the MLX port in fp32 loses nothing, and the residual is porting error alone. The first row's larger figure is bf16 rounding, not a defect.

Task level, against ground truth#

Six documents rendered locally with exactly known text (invoice, lab report, shipping label, receipt, spec table, rotated receipt), transcribed with OCR: and scored directly. This sidesteps using bf16 as the reference at all.

Variant	field	content	numeric	CER vs bf16	tok/s	peak GB
bf16	0.8684	0.9605	0.9720	0 (ref)	138.3	2.50
8-bit	0.8684	0.9605	0.9720	0.0000	210.8	2.06
4-bit	0.8947	0.9474	0.9623	0.0116	272.9	1.83

field counts required strings present exactly, content ignores markup and whitespace, numeric counts ground-truth numbers recovered.

The 8-bit output is byte-identical to bf16 on all six documents, down to the same fields missed.

Two caveats a reader should have:

The 4-bit field score is higher than bf16 and that is not an improvement. The denominator is 38 strings, so one string is 0.026. 4-bit happens to get the invoice fully right and then loses 12.00 on the receipt. content and numeric, which are less format-sensitive, both move the other way. Treat the three variants as separated by one or two strings, not by field.
shipping_label loses the same three fields in every variant, bf16 included (1Z-994-AX-77310582, Halberd Components, 3011AB). That is a limit of the source model on that document, not a quantization effect.

Throughput measured on an M-series Mac during the same run, single image, generation only.

What was not measured#

Only the plain 1024x1024 single-crop path. GOT's fine-grained mode (region by box or by colour), its multi-crop path for dense pages, and its multi-page mode are not exercised by anything above.
OCR with format: was not scored. The accuracy table uses OCR: only. Format mode was smoke-tested and runs, but no ground-truth scoring was done for tables, formulas or sheet music.
No perplexity, top-1 agreement or KL. GOT only emits transcriptions, so generic passages give meaningless perplexity. The ground-truth OCR scoring above replaces that layer rather than supplementing it.
Six synthetic documents, one language. They are rendered, not photographed, and English only. Real scans, handwriting and the multilingual claims of the source are untested here.
No accuracy measurement at long context. Everything ran well under the 32k window.

Source model: stepfun-ai/GOT-OCR2_0 (paper).

Files

13 files · 457 MBWeights 454 MB (safetensors). Sizes and sha256 from the signed manifest.

Hide all 13 files with their sha256

File	Size	sha256
.gitattributes	1.52 KB	11ad7efa24975ee4b0c3c3a38ed18737f0658a5f75a0a96787b576a78a023361
README.md	5.89 KB	0a54462b39d14f47041afdd43d6ff6ca003e2a7646fa33de3eff2619bc4ada59
config.json	1.39 KB	c3e0932e0d555b829b7f1e74f224293157e699db1a78b836f1eb195802d8ef82
generation_config.json	117 B	64e4f9d67c678c6dd6b1ac2f1b0d2562ae12cec463493ce2951daf1dba4e494f
got_vision_b.py	16.1 KB	8931c2e1145afa6e99aa707fa73ab24afabe3ad69fe5341a5b7160e2688cf750
model.safetensors	454 MB	ba4d1151e3b4ba4bd1c6cf1c1d26b6b6d015ef005e362ed5526d813a47e9a5cf
model.safetensors.index.json	65.9 KB	62dff65301bea48401b95627efe80f4c2bcf8e09d7c2184699c14fabad15ab54
modeling_GOT.py	33.8 KB	68375b6bf7683cca41bb987953f0e31fde802652ad01e1accc4b53127b841609
qwen.tiktoken	2.56 MB	b2b1b8dfb5cc5f024bafc373121c6aba3f66f9a5a0269e243470a1de16a33186
render_tools.py	1.99 KB	11393028ac7095431cfaa9c8ee14b9c6e8ca1d618158dfe729b89e83e889fb99
special_tokens_map.json	149 B	337f1a03344485ec7ea2acd6f7021567feab077fd81f96b72423fcd3f8f5fc09
tokenization_qwen.py	9.47 KB	eec08335cdcdcb538120e204f19b41bbd9e5ab989e14b1b1cf8f506ac584c601
tokenizer_config.json	372 B	5058c180033d6cec3aa913c517235d5be2f99363496ced0c7a88a407dcfd7778

VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.

Upstream commit	Date	Contents
0e28c87720 pinned now	2026-11-15 05:18 UTC	13 files · 457 MB

Use it

$ fw get mlx-community/GOT-OCR2_0-4bit
# manifest sha256 2f2e783a4600… · 457 MB · 13 files
$ fw verify ./GOT-OCR2_0-4bit mlx-community/GOT-OCR2_0-4bit

fw.pyChecks the signature and every sha256, tries each source in order, resumes.

$ python3 fw.py get https://freeweights.allternit.com/manifests/mlx-community/GOT-OCR2_0-4bit.json --dir ./GOT-OCR2_0-4bit
$ python3 fw.py verify ./GOT-OCR2_0-4bit ./mlx-community__GOT-OCR2_0-4bit.json

hfThe Hugging Face CLI with one setting. Same repo id.

$ HF_ENDPOINT=https://freeweights.allternit.com hf download mlx-community/GOT-OCR2_0-4bit --revision 0e28c87720de5d8af67389a0444cf8397524cb61 --local-dir ./GOT-OCR2_0-4bit

transformersLoad the weights in Python.

import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForImageTextToText, AutoProcessor
processor = AutoProcessor.from_pretrained("mlx-community/GOT-OCR2_0-4bit", revision="0e28c87720de5d8af67389a0444cf8397524cb61")
model = AutoModelForImageTextToText.from_pretrained("mlx-community/GOT-OCR2_0-4bit", revision="0e28c87720de5d8af67389a0444cf8397524cb61", device_map="auto")

vLLMServe an OpenAI-compatible API.

$ HF_ENDPOINT=https://freeweights.allternit.com vllm serve mlx-community/GOT-OCR2_0-4bit --revision 0e28c87720de5d8af67389a0444cf8397524cb61

Fine-tune in Studio

License

Name: Apache License 2.0
Id: Apache-2.0
Class: Permissive
Redistribution: yes
Listing: Hosted (signed)
Decided: License as declared by the publisher. No person read it. 2026-11-15
Text in: README.md
Text sha256: 0a54462b39d14f47041afdd43d6ff6ca003e2a7646fa33de3eff2619bc4ada59
Terms gate: No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From:

mlx-community <huggingface.co/mlx-community>Newsgroups: alt.freeweights.textSubject: GOT-OCR2_0-4bitDate: 2026-11-15Commit: 0e28c87Signed: 2DC28CC7B26A283B ✓

Upstream: mlx-community/GOT-OCR2_0-4bit
Pinned commit: 0e28c87720de5d8af67389a0444cf8397524cb61
Manifest: GOT-OCR2_0-4bit.json · .minisig
Manifest sha256: 2f2e783a46001244fd603deb5b5b08837a6af2a9fde8845525f144f52597c252
Signature: minisign, Ed25519, key 2DC28CC7B26A283B
Created: 2026-11-15 05:18 UTC
Generator: freeweights-manifest/0.1.0
Takedowns: None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/mlx-community/GOT-OCR2_0-4bit.json
$ curl -O https://freeweights.allternit.com/manifests/mlx-community/GOT-OCR2_0-4bit.json.minisig
$ minisign -Vm GOT-OCR2_0-4bit.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq
$ shasum -a 256 <file>   # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

Parent

stepfun-ai/GOT-OCR2_0 · this model is a quantization of it

No model in the catalog names this one as its parent.

Get it

Download

$ HF_ENDPOINT=https://freeweights.allternit.com hf download mlx-community/GOT-OCR2_0-4bit

Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest

minisign, key 2DC28CC7B26A283B. GOT-OCR2_0-4bit.json. Provenance. Takedown history: none.

Source

huggingface.co/mlx-community/GOT-OCR2_0-4bit

Related models

All of alt.freeweights.text →

From: laion <huggingface.co/laion>Subject: a2-rl-e2egit_large-50-32B

From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_e2egit-large-15-8B

From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_e2egit-v2-10-8B

From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B

From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_exp_rpt_unitsyn-python-large-10-8B

From: laion <huggingface.co/laion>Subject: a3-rl-DCAgent_mix_h2_language_proportional-65-8B