CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.deepseek / deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
From: bartowski <huggingface.co/bartowski>Newsgroups: alt.freeweights.deepseekSubject:

deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF

Date: 2026-11-15Commit: 75b8e3dSigned: 2DC28CC7B26A283B ✓

Text generation model. Gguf weights, 130 GB in all. Published by bartowski under MIT.

deepseek130GBdeepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUFggufpermissive
FactsParamsnot statedSize130 GBFormatsggufTasktext generationLicenseMITFrom sourcepermissive
$ fw get bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
# manifest sha256 0bd0b0bf3947… · 130 GB · 27 files
$ fw verify ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
From sourceHosted (signed)bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUFLast check: 2026-11-15 04:45 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Quantized of DeepSeek-R1-0528-Qwen3-8B.

Works with

Run14Convert1Evaluate13Scan12Apps58

Convert with

ggml
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
130 GB
Weights
3.05 GB (smallest of 24 GGUF variants)
Parameters
not stated
Formats
gguf
Task
text generation
License
MIT (permissive)
Files
27, each with a sha256
Publisher
bartowski
Added
2026-11-15
Section
alt.freeweights.deepseek
Lineage

Original release from the publisher.

Base model: deepseek-ai/DeepSeek-R1-0528-Qwen3-8B

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 04:45 UTCpassed

Model card

From the publisher at commit 75b8e3d. Shown as plain text with links; images open at the publisher.

Llamacpp imatrix Quantizations of DeepSeek-R1-0528-Qwen3-8B by deepseek-ai#

Using llama.cpp release b5524 for quantization.

Original model: https://huggingface.co/deepseek-ai/DeepSeek-R1-0528-Qwen3-8B

All quants made using imatrix option with dataset from here

Run them in LM Studio

Run them directly with llama.cpp, or any other llama.cpp based project

Prompt format#

<｜begin▁of▁sentence｜>{system_prompt}<｜User｜>{prompt}<｜Assistant｜>

Download a file (not the whole branch) from below:#

FilenameQuant typeFile SizeSplitDescription
DeepSeek-R1-0528-Qwen3-8B-bf16.ggufbf1616.39GBfalseFull BF16 weights.
DeepSeek-R1-0528-Qwen3-8B-Q8_0.ggufQ8_08.71GBfalseExtremely high quality, generally unneeded but max available quant.
DeepSeek-R1-0528-Qwen3-8B-Q6_K_L.ggufQ6_K_L7.03GBfalseUses Q8_0 for embed and output weights. Very high quality, near perfect, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q6_K.ggufQ6_K6.73GBfalseVery high quality, near perfect, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q5_K_L.ggufQ5_K_L6.24GBfalseUses Q8_0 for embed and output weights. High quality, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q5_K_M.ggufQ5_K_M5.85GBfalseHigh quality, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q5_K_S.ggufQ5_K_S5.72GBfalseHigh quality, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q4_K_L.ggufQ4_K_L5.49GBfalseUses Q8_0 for embed and output weights. Good quality, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q4_1.ggufQ4_15.25GBfalseLegacy format, similar performance to Q4_K_S but with improved tokens/watt on Apple silicon.
DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.ggufQ4_K_M5.03GBfalseGood quality, default size for most use cases, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q3_K_XL.ggufQ3_K_XL4.98GBfalseUses Q8_0 for embed and output weights. Lower quality but usable, good for low RAM availability.
DeepSeek-R1-0528-Qwen3-8B-Q4_K_S.ggufQ4_K_S4.80GBfalseSlightly lower quality with more space savings, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q4_0.ggufQ4_04.79GBfalseLegacy format, offers online repacking for ARM and AVX CPU inference.
DeepSeek-R1-0528-Qwen3-8B-IQ4_NL.ggufIQ4_NL4.79GBfalseSimilar to IQ4_XS, but slightly larger. Offers online repacking for ARM CPU inference.
DeepSeek-R1-0528-Qwen3-8B-IQ4_XS.ggufIQ4_XS4.56GBfalseDecent quality, smaller than Q4_K_S with similar performance, recommended.
DeepSeek-R1-0528-Qwen3-8B-Q3_K_L.ggufQ3_K_L4.43GBfalseLower quality but usable, good for low RAM availability.
DeepSeek-R1-0528-Qwen3-8B-Q3_K_M.ggufQ3_K_M4.12GBfalseLow quality.
DeepSeek-R1-0528-Qwen3-8B-IQ3_M.ggufIQ3_M3.90GBfalseMedium-low quality, new method with decent performance comparable to Q3_K_M.
DeepSeek-R1-0528-Qwen3-8B-Q2_K_L.ggufQ2_K_L3.89GBfalseUses Q8_0 for embed and output weights. Very low quality but surprisingly usable.
DeepSeek-R1-0528-Qwen3-8B-Q3_K_S.ggufQ3_K_S3.77GBfalseLow quality, not recommended.
DeepSeek-R1-0528-Qwen3-8B-IQ3_XS.ggufIQ3_XS3.63GBfalseLower quality, new method with decent performance, slightly better than Q3_K_S.
DeepSeek-R1-0528-Qwen3-8B-IQ3_XXS.ggufIQ3_XXS3.37GBfalseLower quality, new method with decent performance, comparable to Q3 quants.
DeepSeek-R1-0528-Qwen3-8B-Q2_K.ggufQ2_K3.28GBfalseVery low quality but surprisingly usable.
DeepSeek-R1-0528-Qwen3-8B-IQ2_M.ggufIQ2_M3.05GBfalseRelatively low quality, uses SOTA techniques to be surprisingly usable.

Embed/output weights#

Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.

Downloading using huggingface-cli#

Click to view download instructions

First, make sure you have hugginface-cli installed:

pip install -U "huggingface_hub[cli]"

Then, you can target the specific file you want:

huggingface-cli download bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF --include "deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.gguf" --local-dir ./

If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:

huggingface-cli download bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF --include "deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q8_0/*" --local-dir ./

You can either specify a new local-dir (deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q8_0) or download them all in place (./)

ARM/AVX information#

Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.

Now, however, there is something called "online repacking" for weights. details in this PR. If you use Q4_0 and your hardware would benefit from repacking weights, it will do it automatically on the fly.

As of llama.cpp build b4282 you will not be able to run the Q4_0_X_X files and will instead need to use Q4_0.

Additionally, if you want to get slightly better quality for , you can use IQ4_NL thanks to this PR which will also repack the weights for ARM, though only the 4_4 for now. The loading time may be slower but it will result in an overall speed incrase.

Click to view Q4_0_X_X information (deprecated

I'm keeping this section to show the potential theoretical uplift in performance from using the Q4_0 with online repacking.

Click to view benchmarks on an AVX2 system (EPYC7702)

modelsizeparamsbackendthreadstestt/s% (vs Q4_0)
qwen2 3B Q4_01.70 GiB3.09 BCPU64pp512204.03 ± 1.03100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64pp1024282.92 ± 0.19100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64pp2048259.49 ± 0.44100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64tg12839.12 ± 0.27100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64tg25639.31 ± 0.69100%
qwen2 3B Q4_01.70 GiB3.09 BCPU64tg51240.52 ± 0.03100%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64pp512301.02 ± 1.74147%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64pp1024287.23 ± 0.20101%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64pp2048262.77 ± 1.81101%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64tg12818.80 ± 0.9948%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64tg25624.46 ± 3.0483%
qwen2 3B Q4_K_M1.79 GiB3.09 BCPU64tg51236.32 ± 3.5990%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64pp512271.71 ± 3.53133%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64pp1024279.86 ± 45.63100%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64pp2048320.77 ± 5.00124%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64tg12843.51 ± 0.05111%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64tg25643.35 ± 0.09110%
qwen2 3B Q4_0_8_81.69 GiB3.09 BCPU64tg51242.60 ± 0.31105%

Q4_0_8_8 offers a nice bump to prompt processing and a small bump to text generation

Which file should I choose?#

Click here for details

A great write up with charts showing various performances is provided by Artefact2 here

The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.

If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.

If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.

Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.

If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.

If you want to get more into the weeds, you can check out this extremely useful feature chart:

llama.cpp feature matrix

But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.

These I-quants can also be used on CPU, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.

Credits#

Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset.

Thank you ZeroWw for the inspiration to experiment with embed/output.

Thank you to LM Studio for sponsoring my work.

Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

Files

27 files · 130 GBWeights 3.05 GB (smallest of 24 GGUF variants). Sizes and sha256 from the signed manifest.
Show all 27 files with their sha256
FileSizesha256
.gitattributes3.65 KB049cec6c880e90780a188bf50ee2fb597b6f443ceee628543056a0eedeab472c
README.md14.6 KBd8d2941ac09e6232d33bfe456d346a4753c1cc0aa0b0709fbfef923a1e177b58
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ2_M.gguf3.05 GB8aaa12ea8ffa4a8ed3702af0a07e6f0422ba9252625e56d64d41c9415495d717
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ3_M.gguf3.90 GBa44259398b1ea4dd36f6e2147452d4f8d0feb6fc887a6c827d6f88c9e0f0a52c
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ3_XS.gguf3.63 GB6bab013ec073cba291d32353e46d7e1a4c9e9e5a01567886fd8ad13d78ce5025
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ3_XXS.gguf3.37 GB6d963159f643e881e6c80cb8c77cba70cc580d5eddb06c58bcb7828f1cd3bae9
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ4_NL.gguf4.79 GB884de7279fd8e9d80f01c0ec9ec8a48ad8b988d91a79e871f78e0b7ec133425d
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ4_XS.gguf4.56 GBce3ed47e622d5d331d251069814fc45745637242715306641766946fc8de4940
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q2_K.gguf3.28 GBebe2029c5978581b980cdcbad06febe663447586783b8d884ad8c238b5f047bd
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q2_K_L.gguf3.89 GBaf04458dbfbac80e31d43a832d401c30a7f13b5de5ffd685aa7a4aefd2e4c696
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q3_K_L.gguf4.43 GBafd9a3010a2918f45001d760d2790cd01145db5af00f8e5fffcc18c2b8e4fcaf
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q3_K_M.gguf4.12 GB55f30d347c42b347d39cb46a17acf4945fa1c9f795b0b07f876bef5298a139f1
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q3_K_S.gguf3.77 GBc04a1a72e4bd5dd6add80423de05a4b07d6f1bd08af033f3b901816ea892d110
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q3_K_XL.gguf4.98 GB50cc544b43e2d7bb5d203f9e2763c3b6c73b7862283e73eec25d3b3595a93269
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q4_0.gguf4.79 GBa71a983c64eb72a2b4a885993cd0675474afe7e92d72b051ab8716b23157daa0
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q4_1.gguf5.25 GB17a1546a120de793ae79465331e1dbab335c0c3f096f5587d0c381f8398e8a25
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q4_K_L.gguf5.49 GB3ddf1b2707b269ec15dfb90c0aa71f478a381c19b5fb1715e219e0529415b2d4
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q4_K_M.gguf5.03 GBe0c2f118fd59f3a16f20d18b0e7f79e960c84bc8c66d94fd71a691e05151d54f
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q4_K_S.gguf4.80 GB203945c0afc1e526febf712d122671fa44ee95b57201d4cf0b37edd6261758fe
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q5_K_L.gguf6.24 GBfae68eac5bb977f061558bc7c1bdc17cfb2a8b53494c14eaf753007a3b1d7421
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q5_K_M.gguf5.85 GB5842a7d03f76cf11a418447ef63f12750dc340430ac8a330a6faa8fe8ddc8040
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q5_K_S.gguf5.72 GBb6df7ee66a970ed3fca16e90c18f0b8da43cb09ccf44bb9da51ed38654f79fb5
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q6_K.gguf6.73 GB2bbe05d4279f4f84de9ca6d1756ffc080a17cec6e0b8f99853480b199a559785
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q6_K_L.gguf7.03 GB179354d7da8054c50197dadb736e6b800c47064c10b6bdeb0a3a791cee33c6ae
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-Q8_0.gguf8.71 GB9144694fd6a4347d9842b20797946754fb673871b92ba966e32e51b9057fa428
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-bf16.gguf16.4 GBcba24c93e99fbf5e752ac72ffc3777abc3dacd5de4c8044bce047d460c54c11b
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B.imatrix5.32 MB6f07a101ef6afeee4ae633f3b936163f6ca9b1c8881d4ce02b18ef38655982ef
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
75b8e3d4ec pinned now2026-11-15 04:45 UTC27 files · 130 GB

Use it

$ fw get bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
# manifest sha256 0bd0b0bf3947… · 130 GB · 27 files
$ fw verify ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json --dir ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF $ python3 fw.py verify ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF ./bartowski__deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF --revision 75b8e3d4ecafa21b1f9a06209712f726d0ed77bc --local-dir ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
llama.cppDownload one GGUF variant (deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ2_M, 3.05 GB), then serve it.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF --include "deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ2_M*" --local-dir ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF $ llama-server -m ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ2_M.gguf
OllamaMake an Ollama model from the downloaded file.
$ printf 'FROM ./deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-IQ2_M.gguf\n' > Modelfile $ ollama create deepseek-ai-deepseek-r1-0528-qwen3-8b-gguf -f Modelfile
Run at home

License

Name
MIT License
Id
MIT
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
d8d2941ac09e6232d33bfe456d346a4753c1cc0aa0b0709fbfef923a1e177b58
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: bartowski <huggingface.co/bartowski>Newsgroups: alt.freeweights.deepseekSubject: deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUFDate: 2026-11-15Commit: 75b8e3dSigned: 2DC28CC7B26A283B ✓
Upstream
bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
Pinned commit
75b8e3d4ecafa21b1f9a06209712f726d0ed77bc
Manifest
deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json · .minisig
Manifest sha256
0bd0b0bf39470613d54ee695a08aad2d28af1bc9a0300e30762f361e82ced500
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 04:45 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json $ curl -O https://freeweights.allternit.com/manifests/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json.minisig $ minisign -Vm deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

Parent

deepseek-ai/DeepSeek-R1-0528-Qwen3-8B · this model is a quantization of it

No model in the catalog names this one as its parent.

Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF.json. Provenance. Takedown history: none.
Source
huggingface.co/bartowski/deepseek-ai_DeepSeek-R1-0528-Qwen3-8B-GGUF

Related models

All of alt.freeweights.deepseek →
From: timm <huggingface.co/timm>Subject: deepseek_vit_412m.deepseek_v4_1_flash
From: bartowski <huggingface.co/bartowski>Subject: deepseek-ai_DeepSeek-R1-0528-GGUF
From: bartowski <huggingface.co/bartowski>Subject: deepseek-ai_DeepSeek-V3-0324-GGUF
From: bartowski <huggingface.co/bartowski>Subject: deepseek-ai_DeepSeek-V3.1-GGUF
From: mlx-community <huggingface.co/mlx-community>Subject: deepseek-ai-DeepSeek-V4-Flash-8bit
From: mlx-community <huggingface.co/mlx-community>Subject: DeepSeek-OCR-8bit