alt.freeweights.glm / arcee-ai_GLM-4-32B-Base-32K-GGUF
From: bartowski <huggingface.co/bartowski>Newsgroups: alt.freeweights.glmSubject:

arcee-ai_GLM-4-32B-Base-32K-GGUF

Date: 2026-11-15Commit: 162faaaSigned: 2DC28CC7B26A283B ✓

Text generation model. Gguf weights, 530 GB in all. Published by bartowski under MIT.

glm530GBarcee-ai_GLM-4-32B-Base-32K-GGUFggufpermissive
FactsParamsnot statedSize530 GBFormatsggufTasktext generationLicenseMITFrom sourcepermissive
$ fw get bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF
# manifest sha256 107adb30fd09… · 530 GB · 31 files
$ fw verify ./arcee-ai_GLM-4-32B-Base-32K-GGUF bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF
From sourceHosted (signed)bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUFLast check: 2026-11-15 04:59 UTC

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Quantized of GLM-4-32B-Base-32K.

Works with

Convert with

Overview

Size
530 GB
Weights
8.97 GB (smallest of 27 GGUF variants)
Parameters
not stated
Formats
gguf
Task
text generation
License
MIT (permissive)
Files
31, each with a sha256
Publisher
bartowski
Added
2026-11-15
Section
alt.freeweights.glm
Lineage

Original release from the publisher.

Base model: arcee-ai/GLM-4-32B-Base-32K

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 04:59 UTCpassed

Model card

From the publisher at commit 162faaa. Shown as plain text with links; images open at the publisher.

Llamacpp imatrix Quantizations of GLM-4-32B-Base-32K by arcee-ai#

Using llama.cpp release b5697 for quantization.

Original model: https://huggingface.co/arcee-ai/GLM-4-32B-Base-32K

All quants made using imatrix option with dataset from here

Run them in LM Studio

Run them directly with llama.cpp, or any other llama.cpp based project

Prompt format#

No prompt format found, check original model page

Download a file (not the whole branch) from below:#

FilenameQuant typeFile SizeSplitDescription
GLM-4-32B-Base-32K-bf16.ggufbf1665.14GBtrueFull BF16 weights.
GLM-4-32B-Base-32K-Q8_0.ggufQ8_034.61GBfalseExtremely high quality, generally unneeded but max available quant.
GLM-4-32B-Base-32K-Q6_K_L.ggufQ6_K_L27.18GBfalseUses Q8_0 for embed and output weights. Very high quality, near perfect, recommended.
GLM-4-32B-Base-32K-Q6_K.ggufQ6_K26.73GBfalseVery high quality, near perfect, recommended.
GLM-4-32B-Base-32K-Q5_K_L.ggufQ5_K_L23.67GBfalseUses Q8_0 for embed and output weights. High quality, recommended.
GLM-4-32B-Base-32K-Q5_K_M.ggufQ5_K_M23.10GBfalseHigh quality, recommended.
GLM-4-32B-Base-32K-Q5_K_S.ggufQ5_K_S22.53GBfalseHigh quality, recommended.
GLM-4-32B-Base-32K-Q4_1.ggufQ4_120.55GBfalseLegacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon.
GLM-4-32B-Base-32K-Q4_K_L.ggufQ4_K_L20.37GBfalseUses Q8_0 for embed and output weights. Good quality, recommended.
GLM-4-32B-Base-32K-Q4_K_M.ggufQ4_K_M19.68GBfalseGood quality, default size for most use cases, recommended.
GLM-4-32B-Base-32K-Q4_K_S.ggufQ4_K_S18.70GBfalseSlightly lower quality with more space savings, recommended.
GLM-4-32B-Base-32K-Q4_0.ggufQ4_018.63GBfalseLegacy format, offers online repacking for ARM and AVX CPU inference.
GLM-4-32B-Base-32K-IQ4_NL.ggufIQ4_NL18.58GBfalseSimilar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference.
GLM-4-32B-Base-32K-Q3_K_XL.ggufQ3_K_XL18.03GBfalseUses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability.
GLM-4-32B-Base-32K-IQ4_XS.ggufIQ4_XS17.60GBfalseDecent quality, smaller than Q4_K_S with similar performance, recommended.
GLM-4-32B-Base-32K-Q3_K_L.ggufQ3_K_L17.21GBfalseLower quality but usable, good for low RAM availability.
GLM-4-32B-Base-32K-Q3_K_M.ggufQ3_K_M15.89GBfalseLow quality.
GLM-4-32B-Base-32K-IQ3_M.ggufIQ3_M14.82GBfalseMedium-low quality, new method with decent performance comparable to Q3_K_M.
GLM-4-32B-Base-32K-Q3_K_S.ggufQ3_K_S14.37GBfalseLow quality, not recommended.
GLM-4-32B-Base-32K-IQ3_XS.ggufIQ3_XS13.66GBfalseLower quality, new method with decent performance, slightly better than Q3_K_S.
GLM-4-32B-Base-32K-Q2_K_L.ggufQ2_K_L13.20GBfalseUses Q8_0 for embed and output weights. Very low quality but surprisingly usable.
GLM-4-32B-Base-32K-IQ3_XXS.ggufIQ3_XXS12.78GBfalseLower quality, new method with decent performance, comparable to Q3 quants.
GLM-4-32B-Base-32K-Q2_K.ggufQ2_K12.29GBfalseVery low quality but surprisingly usable.
GLM-4-32B-Base-32K-IQ2_M.ggufIQ2_M11.27GBfalseRelatively low quality, uses SOTA techniques to be surprisingly usable.
GLM-4-32B-Base-32K-IQ2_S.ggufIQ2_S10.42GBfalseLow quality, uses SOTA techniques to be usable.
GLM-4-32B-Base-32K-IQ2_XS.ggufIQ2_XS9.90GBfalseLow quality, uses SOTA techniques to be usable.
GLM-4-32B-Base-32K-IQ2_XXS.ggufIQ2_XXS8.97GBfalseVery low quality, uses SOTA techniques to be usable.

Embed/output weights#

Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.

Downloading using huggingface-cli#

Click to view download instructions

First, make sure you have hugginface-cli installed:

pip install -U "huggingface_hub[cli]"

Then, you can target the specific file you want:

huggingface-cli download bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF --include "arcee-ai_GLM-4-32B-Base-32K-Q4_K_M.gguf" --local-dir ./

If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:

huggingface-cli download bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF --include "arcee-ai_GLM-4-32B-Base-32K-Q8_0/*" --local-dir ./

You can either specify a new local-dir (arcee-ai_GLM-4-32B-Base-32K-Q8_0) or download them all in place (./)

ARM/AVX information#

Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.

Now, however, there is something called "online repacking" for weights. details in this PR. If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.

As of llama.cpp build b4282 you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.

Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to this PR which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.

Click to view Q4_0_X_X information (deprecated

I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.

Click to view benchmarks on an AVX2 system (EPYC7702)

modelsizeparamsbackendthreadstestt/s% (vs Q4_0)
qwen2 3B Q4_01.70 GiB3.09 BCPU64pp512204.03 ± 1.03100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64pp1024282.92 ± 0.19100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64pp2048259.49 ± 0.44100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64tg12839.12 ± 0.27100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64tg25639.31 ± 0.69100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64tg51240.52 ± 0.03100%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64pp512301.02 ± 1.74147%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64pp1024287.23 ± 0.20101%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64pp2048262.77 ± 1.81101%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64tg12818.80 ± 0.9948%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64tg25624.46 ± 3.0483%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64tg51236.32 ± 3.5990%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64pp512271.71 ± 3.53133%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64pp1024279.86 ± 45.63100%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64pp2048320.77 ± 5.00124%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64tg12843.51 ± 0.05111%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64tg25643.35 ± 0.09110%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64tg51242.60 ± 0.31105%

Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation

Which file should I choose?#

Click here for details

A great write up with charts showing various performances is provided by Artefact2 here

The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.

If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.

If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.

Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.

If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.

If you want to get more into the weeds, you can check out this extremely useful feature chart:

llama.cpp feature matrix

But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.

These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.

Credits#

Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.

Thank you ZeroWw for the inspiration to experiment with embed/output.

Thank you to LM Studio for sponsoring my work.

Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

Files

31 files · 530 GBWeights 8.97 GB (smallest of 27 GGUF variants). Sizes and sha256 from the signed manifest.
Show all 31 files with their sha256
FileSizesha256
.gitattributes3.80 KB748a9240cb33712cbc8e18ee0e1c3795324e2e739decd6fa35d038952bbcf3f0
README.md14.5 KBa25c922049567dc5bbfed3a7c4ad2fc422f84f0e4ee0886fa2bbbeb2ce3fab5a
arcee-ai_GLM-4-32B-Base-32K-IQ2_M.gguf11.3 GB3d9f399c579e5a575590f0a9e617911551d155df4e415b54ed0cd305e3756a07
arcee-ai_GLM-4-32B-Base-32K-IQ2_S.gguf10.4 GB8651c43a0d1e3cbf878d0daf5ee558c32241297fe4e7bf3a4ed4a4f42b8d4fec
arcee-ai_GLM-4-32B-Base-32K-IQ2_XS.gguf9.90 GB7c90867e2f3664410d3afc9e52e54f4b0e423a39d40e5a6065906d45c35df1cf
arcee-ai_GLM-4-32B-Base-32K-IQ2_XXS.gguf8.97 GB30fba03b56338f43a951ed9c2260e6069d9a3796dc9cb26afea3518c6b148115
arcee-ai_GLM-4-32B-Base-32K-IQ3_M.gguf14.8 GB5d6bbf34041e27c10d332b8fcb28205aa6d76e31f6785303a31c8cddfc1b360a
arcee-ai_GLM-4-32B-Base-32K-IQ3_XS.gguf13.7 GB41d0423ce173811c337a2faa11df84e166ce82798c72aa0ce82520535a31e333
arcee-ai_GLM-4-32B-Base-32K-IQ3_XXS.gguf12.8 GB25d76a84d4b90a364d2ef2b7d9f2b54f911170b65f99dd3a7ef1412233595d06
arcee-ai_GLM-4-32B-Base-32K-IQ4_NL.gguf18.6 GB3fc1dde9de8e840d42e3c5aad0df62342c2def859e00ca70a9ec4a8e02f84011
arcee-ai_GLM-4-32B-Base-32K-IQ4_XS.gguf17.6 GB7a1fc50c49ce0444c9acda943ae808c710a860ab628fd9f1d4d54cda5676b856
arcee-ai_GLM-4-32B-Base-32K-Q2_K.gguf12.3 GBaa8096159e31680e5cc3b6df0fe6250c93bf2b7f5d611884518153cbe0edd58b
arcee-ai_GLM-4-32B-Base-32K-Q2_K_L.gguf13.2 GB97529a17e23f9164e34bf429b0d363c3609e2f62cbc2df29113ccef6ab51e30f
arcee-ai_GLM-4-32B-Base-32K-Q3_K_L.gguf17.2 GB3d4c8cf2517baeffd8ca71bb17279ff87c2d530862333ff49b5337174be56585
arcee-ai_GLM-4-32B-Base-32K-Q3_K_M.gguf15.9 GB66bec6c2578ec76d8f135b674b112119fe7f8b06dea21ce6bccc914f49796503
arcee-ai_GLM-4-32B-Base-32K-Q3_K_S.gguf14.4 GBaac67b597780be89067424a6bfa039426279ddeef4e22ba718e634b4b86bff48
arcee-ai_GLM-4-32B-Base-32K-Q3_K_XL.gguf18.0 GBbad4f69e3b9587177aaa106e14bf619b99680466017f101c74f7fc5b1e14e6c4
arcee-ai_GLM-4-32B-Base-32K-Q4_0.gguf18.6 GB19ec253a5abe0b5003faa14f555f15f132c6113978a03758284c90b2329fc31b
arcee-ai_GLM-4-32B-Base-32K-Q4_1.gguf20.5 GB4fade7e9cfdb140fde24794a63fef859f35e144f3bcaf2c8e378d73f94ddfd24
arcee-ai_GLM-4-32B-Base-32K-Q4_K_L.gguf20.4 GBdf772c3d5e985ed1f07bfe547f49b6837bce4b38727cca3643e71fe70d031f31
arcee-ai_GLM-4-32B-Base-32K-Q4_K_M.gguf19.7 GBf6fdc0854ed4ed33bc18190bca8a4b2a808a84884c697ca07bbedbff99dbc7f0
arcee-ai_GLM-4-32B-Base-32K-Q4_K_S.gguf18.7 GBe3ced95942f710f9a366097f06f559aee7f2a5ee24e279826b3eba4530cc52cb
arcee-ai_GLM-4-32B-Base-32K-Q5_K_L.gguf23.7 GB22cb1adad435cf3ea764d3cbd0081ed0b31aa5c8ea2b16534cc9df5c8b4f5d6c
arcee-ai_GLM-4-32B-Base-32K-Q5_K_M.gguf23.1 GB69b43d1597fb54ed48759a253340ba2a3ffbeac241a320083587b93ed59fb14a
arcee-ai_GLM-4-32B-Base-32K-Q5_K_S.gguf22.5 GB54ee0b83c5eea510968cd57b47abb4c0fddad40456cac8162cd35dd2a84d4f1e
arcee-ai_GLM-4-32B-Base-32K-Q6_K.gguf26.7 GBaee8180a40e4b85e08e405b3cb8785638c1bbb8f5c09c25b5889322b0d832d4f
arcee-ai_GLM-4-32B-Base-32K-Q6_K_L.gguf27.2 GB90da920bf39e39962a34c7b5b0b3ef28f5c2475c83920d15a996519973733cf4
arcee-ai_GLM-4-32B-Base-32K-Q8_0.gguf34.6 GB39d8396bac41296378e33d3e8f6d71523f31151d1d86731bc2756e4697b8b36c
arcee-ai_GLM-4-32B-Base-32K-bf16/arcee-ai_GLM-4-32B-Base-32K-bf16-00001-of-00002.gguf40.0 GB2ac72b5c6b4c233c51f948fb048d0cea236dd1d937ace8d7d1dbad8eb7178f82
arcee-ai_GLM-4-32B-Base-32K-bf16/arcee-ai_GLM-4-32B-Base-32K-bf16-00002-of-00002.gguf25.2 GB8d7f0e62e27a77ae26439e05caffd6e1f8942b4c43b36fa43cd25777feae88bc
arcee-ai_GLM-4-32B-Base-32K.imatrix13.1 MB80c0cfc912a66acf9a1e8c2ed62e140d5010210dd211b8d178644e84d3ae691b
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
162faaa510 pinned now2026-11-15 04:59 UTC31 files · 530 GB

Use it

$ fw get bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF
# manifest sha256 107adb30fd09… · 530 GB · 31 files
$ fw verify ./arcee-ai_GLM-4-32B-Base-32K-GGUF bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF.json --dir ./arcee-ai_GLM-4-32B-Base-32K-GGUF $ python3 fw.py verify ./arcee-ai_GLM-4-32B-Base-32K-GGUF ./bartowski__arcee-ai_GLM-4-32B-Base-32K-GGUF.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF --revision 162faaa510cf521bd9766d5536249ee7a1e0ba86 --local-dir ./arcee-ai_GLM-4-32B-Base-32K-GGUF
llama.cppDownload one GGUF variant (arcee-ai_GLM-4-32B-Base-32K-IQ2_XXS, 8.97 GB), then serve it.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF --include "arcee-ai_GLM-4-32B-Base-32K-IQ2_XXS*" --local-dir ./arcee-ai_GLM-4-32B-Base-32K-GGUF $ llama-server -m ./arcee-ai_GLM-4-32B-Base-32K-GGUF/arcee-ai_GLM-4-32B-Base-32K-IQ2_XXS.gguf
OllamaMake an Ollama model from the downloaded file.
$ printf 'FROM ./arcee-ai_GLM-4-32B-Base-32K-GGUF/arcee-ai_GLM-4-32B-Base-32K-IQ2_XXS.gguf\n' > Modelfile $ ollama create arcee-ai-glm-4-32b-base-32k-gguf -f Modelfile

License

Name
MIT License
Id
MIT
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
a25c922049567dc5bbfed3a7c4ad2fc422f84f0e4ee0886fa2bbbeb2ce3fab5a
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: bartowski <huggingface.co/bartowski>Newsgroups: alt.freeweights.glmSubject: arcee-ai_GLM-4-32B-Base-32K-GGUFDate: 2026-11-15Commit: 162faaaSigned: 2DC28CC7B26A283B ✓
Upstream
bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF
Pinned commit
162faaa510cf521bd9766d5536249ee7a1e0ba86
Manifest
arcee-ai_GLM-4-32B-Base-32K-GGUF.json · .minisig
Manifest sha256
107adb30fd094cf4495580014f34bf5218a25e72e90c420bc441bee2dba208d5
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 04:59 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF.json $ curl -O https://freeweights.allternit.com/manifests/bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF.json.minisig $ minisign -Vm arcee-ai_GLM-4-32B-Base-32K-GGUF.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

Parent

arcee-ai/GLM-4-32B-Base-32K · this model is a quantization of it

No model in the catalog names this one as its parent.

Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/arcee-ai_GLM-4-32B-Base-32K-GGUF
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. arcee-ai_GLM-4-32B-Base-32K-GGUF.json. Provenance. Takedown history: none.
From: ggml-org <huggingface.co/ggml-org>Subject: AutoGLM-Phone-9B-GGUF
From: zai-org <huggingface.co/zai-org>Subject: cogagent-chat-hf
From: zai-org <huggingface.co/zai-org>Subject: CogView3-Plus-3B
From: zai-org <huggingface.co/zai-org>Subject: cogvlm-base-490-hf
From: laion <huggingface.co/laion>Subject: dev_set_part1_10k_glm_4_7_traces_jupiter
From: laion <huggingface.co/laion>Subject: dev_set_part1_10k_glm_4_7_traces_jupiter_cleaned