CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.mistral / mistral-7b-instruct-v0.2-ONNX
From: microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.mistralSubject:

mistral-7b-instruct-v0.2-ONNX

Date: 2026-11-15Commit: 6d52410Signed: 2DC28CC7B26A283B ✓

Text generation model. Onnx weights, 32.7 GB in all. Published by microsoft under Apache-2.0.

mistral32.7GBmistral-7b-instruct-v0.2-ONNXonnxpermissive
FactsParamsnot statedSize32.7 GBFormatsonnxTasktext generationLicenseApache-2.0From sourcepermissive
$ fw get microsoft/mistral-7b-instruct-v0.2-ONNX
# manifest sha256 eb1f6857aa1d… · 32.7 GB · 45 files
$ fw verify ./mistral-7b-instruct-v0.2-ONNX microsoft/mistral-7b-instruct-v0.2-ONNX
From sourceHosted (signed)microsoft/mistral-7b-instruct-v0.2-ONNXLast check: 2026-11-15 03:58 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run5Convert3Evaluate13Scan12Apps58

Runs with

Ranked by fit to this model (onnx), then by GitHub stars.

Transformers.jsONNX Runtime GenAIUltralyticsgpu-unlockOpenVINO

Convert with

ONNXOptimumOlive
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
32.7 GB
Weights
202 KB (smallest of 5 ONNX variants)
Parameters
not stated
Formats
onnx
Task
text generation
License
Apache-2.0 (permissive)
Files
45, each with a sha256
Publisher
microsoft
Added
2026-11-15
Section
alt.freeweights.mistral
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 03:58 UTCpassed

Model card

From the publisher at commit 6d52410. Shown as plain text with links; images open at the publisher.

Mistral-7B-Instruct-v0.2 ONNX models#

This repository hosts the optimized versions of Mistral-7B-Instruct-v0.2 to accelerate inference with ONNX Runtime.

The Mistral-7B-Instruct-v0.2 Large Language Model (LLM) is an instruct fine-tuned version of the Mistral-7B-v0.2.

Optimized Mistral models are published here in ONNX format to run with ONNX Runtime on CPU and GPU across devices, including server platforms and Windows, Linux, and Mac desktops, with the precision best suited to each of these targets.

DirectML support lets developers bring hardware acceleration to Windows devices at scale across AMD, Intel, and NVIDIA GPUs. Along with DirectML, ONNX Runtime provides cross platform support for Mistral across a range of devices for CPU and GPU.

To easily get started with Mistral, you can use Olive, our easy-to-use, hardware-aware model optimization tool. See here for instructions on how to run it with Mistral.

ONNX Models#

Here are some of the optimized configurations we have added:

  1. ONNX model for int4 DML: ONNX model for AMD, Intel, and NVIDIA GPUs on Windows, quantized to int4 using AWQ.
  2. ONNX model for fp16 CUDA: ONNX model you can use to run for your NVIDIA GPUs.
  3. ONNX model for int4 CUDA: ONNX model for NVIDIA GPUs using int4 quantization via RTN.
  4. ONNX model for int4 CPU: ONNX model for your CPU, using int4 quantization via RTN.

Hardware Supported#

The models are tested on:

  • GPU SKU: RTX 4090 (DirectML)
  • GPU SKU: 1 A100 80GB GPU, SKU: Standard_ND96amsr_A100_v4 (CUDA)
  • CPU SKU: Standard F64s v2 (64 vcpus, 128 GiB memory)

Minimum Configuration Required:

  • Windows: DirectX 12-capable GPU and a minimum of 4GB of combined RAM
  • CUDA: Streaming Multiprocessors (SMs) >= 70 (i.e. V100 or newer)
Model Description#
  • Developed by: Microsoft
  • Model type: ONNX
  • Language(s) (NLP): Python, C, C++
  • License: Apache License Version 2.0
  • Model Description: This is a conversion of the Mistral-7B-Instruct-v0.2 model for ONNX Runtime inference.

Additional Details#

Appendix#

Activation Aware Quantization#

AWQ works by identifying the top 1% most salient weights that are most important for maintaining accuracy and quantizing the remaining 99% of weights. This leads to less accuracy loss from quantization compared to many other quantization techniques. For more on AWQ, see here.

Model Card Contact#

sschoenmeyer, sunghcho, kvaishnavi

Files

45 files · 32.7 GBWeights 202 KB (smallest of 5 ONNX variants). Sizes and sha256 from the signed manifest.
Show all 45 files with their sha256
FileSizesha256
.gitattributes2.24 KBa910964698aa4ead26e69eb90a05c57eb249e9fa5b801b33638c1edbe5868a15
License.txt12.7 KB9d03887a7c24d4ea5dfdb7eec5386133b786837077e7ecce79a37aa708c5b7c1
README.md3.19 KB9c8f9e1a6f8d6b243a470c5f8ed81e08d7f74d4eb45300a0c14053a58b3b3d3e
config.json68 B97c08de48c2d55c43dd77f4277af464e72d5c4641503fd7d73ab4ba5724e4dfa
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/genai_config.json1.51 KB1b3b9ae30513d009b8ba126cd697c6bffeada2b37b9e6270cc4840b10c96e684
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4.onnx233 KBa60342073f94d4ce822e4b0fd37e644fdd6e5c98ff87db4960630acf9bba6812
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4.onnx.data4.99 GB207f9a1ee9cb0df1e142e67b9647f317b57508816bf951c93367a7cabceac57d
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/special_tokens_map.json414 B6fa06efa2785e450051989a6f8fb4416b10149ded485ddd3f127a40734f5cfd0
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/tokenizer.json1.80 MBbdc45dc7f18a5c55e0dcb36f4f187a7c429d6aaa1693a17d20422b85081642d9
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/tokenizer.model493 KBdadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32-acc-level-4/tokenizer_config.json1.46 KB0b9444de796ce8352bc4c235aa6fc6ca43c63baf98015cdba16bc8b5ad296c88
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/genai_config.json1.50 KBc0f4e95a5fa7b7b6694a910eceec1c1068e25186f3a0eda3d10c86a13184293e
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32.onnx224 KB88fedcf87182a77107e58a0089c85b9531161c1ebee1e341e2a5ab8b1908035f
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32.onnx.data4.99 GB207f9a1ee9cb0df1e142e67b9647f317b57508816bf951c93367a7cabceac57d
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/special_tokens_map.json414 B6fa06efa2785e450051989a6f8fb4416b10149ded485ddd3f127a40734f5cfd0
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/tokenizer.json1.80 MBbdc45dc7f18a5c55e0dcb36f4f187a7c429d6aaa1693a17d20422b85081642d9
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/tokenizer.model493 KBdadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
onnx/cpu_and_mobile/mistral-7b-instruct-v0.2-cpu-int4-rtn-block-32/tokenizer_config.json1.46 KB0b9444de796ce8352bc4c235aa6fc6ca43c63baf98015cdba16bc8b5ad296c88
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/.azDownload-0947fa30-a02d-9f4b-5ab6-bad303580200-mistral-7b-instruct-v0.2-cuda-fp16.onnx.data0 Be3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/.azDownload-0e60df20-e036-c447-766b-79680d038d62-mistral-7b-instruct-v0.2-cuda-fp16.onnx.data0 Be3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/genai_config.json1.66 KB1a0d052ceff3a5c13fa4ef06b7b726305c9418733f34909cdc945f779133f577
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/mistral-7b-instruct-v0.2-cuda-fp16.onnx202 KBe78dbe83f9f32a17253d0342adac44edaff2d560dcbf1c52e1e4a6dabe8ea5e0
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/mistral-7b-instruct-v0.2-cuda-fp16.onnx.data14.5 GB3e21f7bdf820423d60308d97514491fe68b2c2cf4f3a6d492fcc0a488c30b51c
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/special_tokens_map.json414 B6fa06efa2785e450051989a6f8fb4416b10149ded485ddd3f127a40734f5cfd0
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/tokenizer.json1.80 MBfc4f0bd70b3709312d9d1d9e5ba674794b6bc5abc17429897a540f93882f25fc
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/tokenizer.model493 KBdadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
onnx/cuda/mistral-7b-instruct-v0.2-cuda-fp16/tokenizer_config.json1.46 KB0b9444de796ce8352bc4c235aa6fc6ca43c63baf98015cdba16bc8b5ad296c88
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/.azDownload-0947fa30-a02d-9f4b-5ab6-bad303580200-mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx.data0 Be3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/.azDownload-0e60df20-e036-c447-766b-79680d038d62-mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx.data0 Be3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/genai_config.json1.68 KB878112c63eaa571ef2660c1fbe4c3dd020ca7c3c5f186852abdd26b1f29cb816
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx275 KB39e88df693538bc868ad9356bc0ac1bcd66c5596563fa5a0637d27e951ab4b88
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32.onnx.data4.27 GB7546e8999ae2a38ea4c694a3046b06d21cdde70dfb95832349bea6d3da7aa2f1
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/special_tokens_map.json414 B6fa06efa2785e450051989a6f8fb4416b10149ded485ddd3f127a40734f5cfd0
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/tokenizer.json1.80 MBfc4f0bd70b3709312d9d1d9e5ba674794b6bc5abc17429897a540f93882f25fc
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/tokenizer.model493 KBdadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
onnx/cuda/mistral-7b-instruct-v0.2-cuda-int4-rtn-block-32/tokenizer_config.json1.46 KB0b9444de796ce8352bc4c235aa6fc6ca43c63baf98015cdba16bc8b5ad296c88
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/.azDownload-0947fa30-a02d-9f4b-5ab6-bad303580200-model.onnx.data0 Be3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/.azDownload-0e60df20-e036-c447-766b-79680d038d62-model.onnx.data0 Be3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/genai_config.json1.66 KBfa17c52d4880198083ae3d8f7f536564a23285814b1db2599d5283f65a0ad95f
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/model.onnx12.0 MB298f8cc3cf1b3dfeddcc3912a1900f235499fb3e4354389cbbf08fac99666e55
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/model.onnx.data3.96 GBca1244eb774db3cc813d308bb56e6671387c33380fa9145a61fdf40cc62df891
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/special_tokens_map.json414 B6fa06efa2785e450051989a6f8fb4416b10149ded485ddd3f127a40734f5cfd0
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/tokenizer.json1.80 MBfc4f0bd70b3709312d9d1d9e5ba674794b6bc5abc17429897a540f93882f25fc
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/tokenizer.model493 KBdadfd56d766715c61d2ef780a525ab43b8e6da4de6865bda3d95fdef5e134055
onnx/directml/mistralai_Mistral-7B-Instruct-v0.2/tokenizer_config.json1.46 KB0b9444de796ce8352bc4c235aa6fc6ca43c63baf98015cdba16bc8b5ad296c88
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
6d524105ea pinned now2026-11-15 03:58 UTC45 files · 32.7 GB

Use it

$ fw get microsoft/mistral-7b-instruct-v0.2-ONNX
# manifest sha256 eb1f6857aa1d… · 32.7 GB · 45 files
$ fw verify ./mistral-7b-instruct-v0.2-ONNX microsoft/mistral-7b-instruct-v0.2-ONNX
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/microsoft/mistral-7b-instruct-v0.2-ONNX.json --dir ./mistral-7b-instruct-v0.2-ONNX $ python3 fw.py verify ./mistral-7b-instruct-v0.2-ONNX ./microsoft__mistral-7b-instruct-v0.2-ONNX.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/mistral-7b-instruct-v0.2-ONNX --revision 6d524105ea40470ca1897c97cc467f1f96a8a077 --local-dir ./mistral-7b-instruct-v0.2-ONNX
Run at home

License

Name
Apache License 2.0
Id
Apache-2.0
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
License.txt
Text sha256
9d03887a7c24d4ea5dfdb7eec5386133b786837077e7ecce79a37aa708c5b7c1
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.mistralSubject: mistral-7b-instruct-v0.2-ONNXDate: 2026-11-15Commit: 6d52410Signed: 2DC28CC7B26A283B ✓
Upstream
microsoft/mistral-7b-instruct-v0.2-ONNX
Pinned commit
6d524105ea40470ca1897c97cc467f1f96a8a077
Manifest
mistral-7b-instruct-v0.2-ONNX.json · .minisig
Manifest sha256
eb1f6857aa1d973b81025c1c1c897b14292ba53f21f6bbd9dec74b0ceaec10a1
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 03:58 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/microsoft/mistral-7b-instruct-v0.2-ONNX.json $ curl -O https://freeweights.allternit.com/manifests/microsoft/mistral-7b-instruct-v0.2-ONNX.json.minisig $ minisign -Vm mistral-7b-instruct-v0.2-ONNX.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/mistral-7b-instruct-v0.2-ONNX
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. mistral-7b-instruct-v0.2-ONNX.json. Provenance. Takedown history: none.
Source
huggingface.co/microsoft/mistral-7b-instruct-v0.2-ONNX

Related models

All of alt.freeweights.mistral →
From: bartowski <huggingface.co/bartowski>Subject: alamios_Mistral-Small-3.1-DRAFT-0.5B-GGUF
From: microsoft <huggingface.co/microsoft>Subject: chatbench-mistral-7b
From: NousResearch <huggingface.co/NousResearch>Subject: DeepHermes-3-Mistral-24B-Preview
From: mistralai <huggingface.co/mistralai>Subject: Devstral-Small-2-24B-Instruct-2512
From: mlx-community <huggingface.co/mlx-community>Subject: Devstral-Small-2-24B-Instruct-2512-4bit
From: mlx-community <huggingface.co/mlx-community>Subject: Devstral-Small-2-24B-Instruct-2512-6bit