CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.mistral / Mistral-7B-Instruct-v0.3-GGUF
From: bartowski <huggingface.co/bartowski>Newsgroups: alt.freeweights.mistralSubject:

Mistral-7B-Instruct-v0.3-GGUF

Date: 2026-11-15Commit: 61fd416Signed: 2DC28CC7B26A283B ✓

Text generation model. Gguf weights, 107 GB in all. Published by bartowski under Apache-2.0.

mistral107GBMistral-7B-Instruct-v0.3-GGUFggufpermissive
FactsParamsnot statedSize107 GBFormatsggufTasktext generationLicenseApache-2.0From sourcepermissive
$ fw get bartowski/Mistral-7B-Instruct-v0.3-GGUF
# manifest sha256 5eba36fa9081… · 107 GB · 26 files
$ fw verify ./Mistral-7B-Instruct-v0.3-GGUF bartowski/Mistral-7B-Instruct-v0.3-GGUF
From sourceHosted (signed)bartowski/Mistral-7B-Instruct-v0.3-GGUFLast check: 2026-11-15 02:33 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run14Convert1Evaluate13Scan12Apps58

Convert with

ggml
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
107 GB
Weights
1.62 GB (smallest of 23 GGUF variants)
Parameters
not stated
Formats
gguf
Task
text generation
License
Apache-2.0 (permissive)
Files
26, each with a sha256
Publisher
bartowski
Added
2026-11-15
Section
alt.freeweights.mistral
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 02:33 UTCpassed

Model card

From the publisher at commit 61fd416. Shown as plain text with links; images open at the publisher.

Llamacpp imatrix Quantizations of Mistral-7B-Instruct-v0.3#

Using llama.cpp release b2965 for quantization.

Original model: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3

All quants made using imatrix option with dataset from here

Prompt format#

<s>[INST]  {prompt} [/INST]</s>

Note that this model does not support a System prompt.

Download a file (not the whole branch) from below:#

FilenameQuant typeFile SizeDescription
Mistral-7B-Instruct-v0.3-Q8_0.ggufQ8_07.70GBExtremely high quality, generally unneeded but max available quant.
Mistral-7B-Instruct-v0.3-Q6_K.ggufQ6_K5.94GBVery high quality, near perfect, recommended.
Mistral-7B-Instruct-v0.3-Q5_K_M.ggufQ5_K_M5.13GBHigh quality, recommended.
Mistral-7B-Instruct-v0.3-Q5_K_S.ggufQ5_K_S5.00GBHigh quality, recommended.
Mistral-7B-Instruct-v0.3-Q4_K_M.ggufQ4_K_M4.37GBGood quality, uses about 4.83 bits per weight, recommended.
Mistral-7B-Instruct-v0.3-Q4_K_S.ggufQ4_K_S4.14GBSlightly lower quality with more space savings, recommended.
Mistral-7B-Instruct-v0.3-IQ4_NL.ggufIQ4_NL4.13GBDecent quality, slightly smaller than Q4_K_S with similar performance recommended.
Mistral-7B-Instruct-v0.3-IQ4_XS.ggufIQ4_XS3.91GBDecent quality, smaller than Q4_K_S with similar performance, recommended.
Mistral-7B-Instruct-v0.3-Q3_K_L.ggufQ3_K_L3.82GBLower quality but usable, good for low RAM availability.
Mistral-7B-Instruct-v0.3-Q3_K_M.ggufQ3_K_M3.52GBEven lower quality.
Mistral-7B-Instruct-v0.3-IQ3_M.ggufIQ3_M3.28GBMedium-low quality, new method with decent performance comparable to Q3_K_M.
Mistral-7B-Instruct-v0.3-IQ3_S.ggufIQ3_S3.18GBLower quality, new method with decent performance, recommended over Q3_K_S quant, same size with better performance.
Mistral-7B-Instruct-v0.3-Q3_K_S.ggufQ3_K_S3.16GBLow quality, not recommended.
Mistral-7B-Instruct-v0.3-IQ3_XS.ggufIQ3_XS3.02GBLower quality, new method with decent performance, slightly better than Q3_K_S.
Mistral-7B-Instruct-v0.3-IQ3_XXS.ggufIQ3_XXS2.83GBLower quality, new method with decent performance, comparable to Q3 quants.
Mistral-7B-Instruct-v0.3-Q2_K.ggufQ2_K2.72GBVery low quality but surprisingly usable.
Mistral-7B-Instruct-v0.3-IQ2_M.ggufIQ2_M2.50GBVery low quality, uses SOTA techniques to also be surprisingly usable.
Mistral-7B-Instruct-v0.3-IQ2_S.ggufIQ2_S2.31GBVery low quality, uses SOTA techniques to be usable.
Mistral-7B-Instruct-v0.3-IQ2_XS.ggufIQ2_XS2.20GBVery low quality, uses SOTA techniques to be usable.
Mistral-7B-Instruct-v0.3-IQ2_XXS.ggufIQ2_XXS1.99GBLower quality, uses SOTA techniques to be usable.
Mistral-7B-Instruct-v0.3-IQ1_M.ggufIQ1_M1.75GBExtremely low quality, not recommended.
Mistral-7B-Instruct-v0.3-IQ1_S.ggufIQ1_S1.61GBExtremely low quality, not recommended.

Downloading using huggingface-cli#

First, make sure you have hugginface-cli installed:

pip install -U "huggingface_hub[cli]"

Then, you can target the specific file you want:

huggingface-cli download bartowski/Mistral-7B-Instruct-v0.3-GGUF --include "Mistral-7B-Instruct-v0.3-Q4_K_M.gguf" --local-dir ./

If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:

huggingface-cli download bartowski/Mistral-7B-Instruct-v0.3-GGUF --include "Mistral-7B-Instruct-v0.3-Q8_0.gguf/*" --local-dir Mistral-7B-Instruct-v0.3-Q8_0

You can either specify a new local-dir (Mistral-7B-Instruct-v0.3-Q8_0) or download them all in place (./)

Which file should I choose?#

A great write up with charts showing various performances is provided by Artefact2 here

The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.

If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.

If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.

Next, you'll need to decide if you want to use an 'I-quant' or a 'K-quant'.

If you don't want to think too much, grab one of the K-quants. These are in format 'QX_K_X', like Q5_K_M.

If you want to get more into the weeds, you can check out this extremely useful feature chart:

llama.cpp feature matrix

But basically, if you're aiming for below Q4, and you're running cuBLAS (Nvidia) or rocBLAS (AMD), you should look towards the I-quants. These are in format IQX_X, like IQ3_M. These are newer and offer better performance for their size.

These I-quants can also be used on CPU and Apple Metal, but will be slower than their K-quant equivalent, so speed vs performance is a tradeoff you'll have to decide.

The I-quants are not compatible with Vulcan, which is also AMD, so if you have an AMD card double check if you're using the rocBLAS build or the Vulcan build. At the time of writing this, LM Studio has a preview with ROCm support, and other inference engines have specific builds for ROCm.

Want to support my work? Visit my ko-fi page here: https://ko-fi.com/bartowski

Files

26 files · 107 GBWeights 1.62 GB (smallest of 23 GGUF variants). Sizes and sha256 from the signed manifest.
Show all 26 files with their sha256
FileSizesha256
.gitattributes3.25 KBd3d784d79a794b5d1178bfa6dfb32d6e01458782c563d3c0d264d647a9e8ee53
Mistral-7B-Instruct-v0.3-IQ1_M.gguf1.76 GB554d3089591e9bb89f6d8861d70021c7ee54fd68b82c6b65cd416a477dde6af6
Mistral-7B-Instruct-v0.3-IQ1_S.gguf1.62 GB638ca17a98a00935d1ebb570910d4729e52925902c662416b949c8757c4fa198
Mistral-7B-Instruct-v0.3-IQ2_M.gguf2.50 GBb54e1139f0940dd0d59369389064488bc9663bfcd985e7c6e6d74da2ad2b4654
Mistral-7B-Instruct-v0.3-IQ2_S.gguf2.31 GB622b6bafc2fa05811a8b15186927d7758fc869ad0305ff26ae3f15e0263d95c0
Mistral-7B-Instruct-v0.3-IQ2_XS.gguf2.20 GBda7cb13e0d184c15e32dd1a7ceb3e496463a95eadb75e8497d006f8786e4e414
Mistral-7B-Instruct-v0.3-IQ2_XXS.gguf1.99 GB5d993ed907dfdfe24e24d4f081a7b999c45498ecce39902983115d0d09d013db
Mistral-7B-Instruct-v0.3-IQ3_M.gguf3.29 GB4ea14c5a6c787ac2703505f04a4ee746f746d1ace3ffd907af28f6f179e6b224
Mistral-7B-Instruct-v0.3-IQ3_S.gguf3.19 GB8d137b444f93f15be78b749c72d6a5a25e3fa719310d166260abbce8175a15fc
Mistral-7B-Instruct-v0.3-IQ3_XS.gguf3.02 GBadb9a2dce4675f8e730c6519582637ef76df1b8b623981789de6940abc75ad71
Mistral-7B-Instruct-v0.3-IQ3_XXS.gguf2.83 GB131c8b3e5ab9877dd921c81cc8674e797d7ca701a828c54552ae31d24111c123
Mistral-7B-Instruct-v0.3-IQ4_NL.gguf4.13 GBc2048801d2895ac26048eae97ccf4d502f07fbde782564f12692063d54415643
Mistral-7B-Instruct-v0.3-IQ4_XS.gguf3.91 GB0f027bb80c65d1a81c5c4b88b9bc79337d30058b7a78b036a033b9f0b44b7a0d
Mistral-7B-Instruct-v0.3-Q2_K.gguf2.72 GB6696020e424ccf3bc4ab6ac2bc639220f3d09ad6a225d5c3e16033a1ff9a4885
Mistral-7B-Instruct-v0.3-Q3_K_L.gguf3.83 GBee484d7f5a871f8ddefec262c818808e7a34019c02168f52b2849fa1a7911696
Mistral-7B-Instruct-v0.3-Q3_K_M.gguf3.52 GB145b11df092f04bbecb9fd99e83d1f3d8a6d1e53805682bfba1fe314f306e1d5
Mistral-7B-Instruct-v0.3-Q3_K_S.gguf3.17 GB3716a9190b8e2061ba51b813270b5d218c2c53daf8cefe8b0986bac5bcf53d2d
Mistral-7B-Instruct-v0.3-Q4_K_M.gguf4.37 GB1270d22c0fbb3d092fb725d4d96c457b7b687a5f5a715abe1e818da303e562b6
Mistral-7B-Instruct-v0.3-Q4_K_S.gguf4.14 GB0415e35c7e6e156b949410f9d89ffb2cd058e4696a7f82dfcfcdf6a2e04d6320
Mistral-7B-Instruct-v0.3-Q5_K_M.gguf5.14 GB0374072546aaf1257d1e34610edb11b691730d8ef7d44b5629693a2dd8e8d472
Mistral-7B-Instruct-v0.3-Q5_K_S.gguf5.00 GB27df061e4e10b08c0c2c3c170038f7b4028ebf3d2dc2a8f0705a6235632bff30
Mistral-7B-Instruct-v0.3-Q6_K.gguf5.95 GB757d75df89821a94841f341f3270b0709d36891dd8fbc41dfd059d213b54c3f5
Mistral-7B-Instruct-v0.3-Q8_0.gguf7.70 GB404857e776114baada71a08ebd3bba79d721ec7fca99705e7e7b892ae8bc583f
Mistral-7B-Instruct-v0.3-f32.gguf29.0 GBcf49fe1559d48ab2f68a0add81c3742a2cad7cdfc764fcd215ac942a6bb56ca9
Mistral-7B-Instruct-v0.3.imatrix4.99 MB2e79af97172cc6ed110cbdaf46f500f7df2306afb22a78a43f16318cfabb189f
README.md8.44 KB0644c4030c8388f3e5e1db036c8a94814376a4e182fd1e0bbb6d9d8bbd05c98b
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
61fd4167ff pinned now2026-11-15 02:33 UTC26 files · 107 GB

Use it

$ fw get bartowski/Mistral-7B-Instruct-v0.3-GGUF
# manifest sha256 5eba36fa9081… · 107 GB · 26 files
$ fw verify ./Mistral-7B-Instruct-v0.3-GGUF bartowski/Mistral-7B-Instruct-v0.3-GGUF
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/bartowski/Mistral-7B-Instruct-v0.3-GGUF.json --dir ./Mistral-7B-Instruct-v0.3-GGUF $ python3 fw.py verify ./Mistral-7B-Instruct-v0.3-GGUF ./bartowski__Mistral-7B-Instruct-v0.3-GGUF.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/Mistral-7B-Instruct-v0.3-GGUF --revision 61fd4167fff3ab01ee1cfe0da183fa27a944db48 --local-dir ./Mistral-7B-Instruct-v0.3-GGUF
llama.cppDownload one GGUF variant (Mistral-7B-Instruct-v0.3-IQ1_S, 1.62 GB), then serve it.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/Mistral-7B-Instruct-v0.3-GGUF --include "Mistral-7B-Instruct-v0.3-IQ1_S*" --local-dir ./Mistral-7B-Instruct-v0.3-GGUF $ llama-server -m ./Mistral-7B-Instruct-v0.3-GGUF/Mistral-7B-Instruct-v0.3-IQ1_S.gguf
OllamaMake an Ollama model from the downloaded file.
$ printf 'FROM ./Mistral-7B-Instruct-v0.3-GGUF/Mistral-7B-Instruct-v0.3-IQ1_S.gguf\n' > Modelfile $ ollama create mistral-7b-instruct-v0.3-gguf -f Modelfile
Run at home

License

Name
Apache License 2.0
Id
Apache-2.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
README.md
Text sha256
0644c4030c8388f3e5e1db036c8a94814376a4e182fd1e0bbb6d9d8bbd05c98b
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: bartowski <huggingface.co/bartowski>Newsgroups: alt.freeweights.mistralSubject: Mistral-7B-Instruct-v0.3-GGUFDate: 2026-11-15Commit: 61fd416Signed: 2DC28CC7B26A283B ✓
Upstream
bartowski/Mistral-7B-Instruct-v0.3-GGUF
Pinned commit
61fd4167fff3ab01ee1cfe0da183fa27a944db48
Manifest
Mistral-7B-Instruct-v0.3-GGUF.json · .minisig
Manifest sha256
5eba36fa9081d60f43a01a67d773ecce4bb7567bc3a42d2cb6c00a6dc1807a0b
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 02:33 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/bartowski/Mistral-7B-Instruct-v0.3-GGUF.json $ curl -O https://freeweights.allternit.com/manifests/bartowski/Mistral-7B-Instruct-v0.3-GGUF.json.minisig $ minisign -Vm Mistral-7B-Instruct-v0.3-GGUF.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download bartowski/Mistral-7B-Instruct-v0.3-GGUF
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. Mistral-7B-Instruct-v0.3-GGUF.json. Provenance. Takedown history: none.
Source
huggingface.co/bartowski/Mistral-7B-Instruct-v0.3-GGUF

Related models

All of alt.freeweights.mistral →
From: bartowski <huggingface.co/bartowski>Subject: alamios_Mistral-Small-3.1-DRAFT-0.5B-GGUF
From: microsoft <huggingface.co/microsoft>Subject: chatbench-mistral-7b
From: NousResearch <huggingface.co/NousResearch>Subject: DeepHermes-3-Mistral-24B-Preview
From: mistralai <huggingface.co/mistralai>Subject: Devstral-Small-2-24B-Instruct-2512
From: mlx-community <huggingface.co/mlx-community>Subject: Devstral-Small-2-24B-Instruct-2512-4bit
From: mlx-community <huggingface.co/mlx-community>Subject: Devstral-Small-2-24B-Instruct-2512-6bit