alt.freeweights.qwen / Qwen3-8B-ONNX

From:

onnx-community <huggingface.co/onnx-community>Newsgroups: alt.freeweights.qwenSubject:

Qwen3-8B-ONNX

Date: 2026-11-15Commit: eab7c5bSigned: 2DC28CC7B26A283B ✓

Model. Onnx weights, 18.5 GB in all. Published by onnx-community under Apache-2.0.

qwen

18.5GBQwen3-8B-ONNXonnxpermissive

FactsParamsnot statedSize18.5 GBFormatsonnxTasknot taggedLicenseApache-2.0From sourcepermissive

$ fw get onnx-community/Qwen3-8B-ONNX
# manifest sha256 dac8d38bd020… · 18.5 GB · 28 files
$ fw verify ./Qwen3-8B-ONNX onnx-community/Qwen3-8B-ONNX

From sourceHosted (signed)onnx-community/Qwen3-8B-ONNXLast check: 2026-11-15 05:23 UTC

Run at home Get help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Quantized of Qwen3-8B.

Works with

Run2 Convert3 Scan1

Runs with

Ranked by fit to this model (onnx), then by GitHub stars.

Ultralytics OpenVINO

Convert with

ONNX

Optimum

Olive

Scan with

ModelScan

Overview Model card Files Use it License Provenance Model tree

Overview

Size: 18.5 GB
Weights: 486 KB (smallest of 3 ONNX variants)
Parameters: not stated
Formats: onnx
Task: not tagged
License: Apache-2.0 (permissive)
Files: 28, each with a sha256
Publisher: onnx-community
Added: 2026-11-15
Section: alt.freeweights.qwen

Lineage

Original release from the publisher.

Base model: Qwen/Qwen3-8B

Where these bytes come from

Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources

Badge	Host	Last check	Result
From source	huggingface.co	2026-11-15 05:23 UTC	passed

Model card

From the publisher at commit eab7c5b. Shown as plain text with links; images open at the publisher.

Qwen3-8B ONNX Models#

This repository hosts the optimized versions of Qwen3-8B to accelerate inference with ONNX Runtime. Model optimizations refer to techniques and methods used to improve the runtime performance, efficiency, and resource utilization of machine learning models.

Optimized models are published here in ONNX format to run with ONNX Runtime with the precision best suited to this target.

Here are some of the optimized configurations we have added:

ONNX model for CPU and mobile using int4 quantization via KLD gradient and block size 128.
ONNX model for CUDA using int4 quantization via KLD gradient and block size 128.
ONNX model for WebGPU using int4 quantization via KLD gradient and block size 32.

Model Create#

You can see how to create the ONNX models by following the Olive recipes here for your target hardware.

Model Run#

You can install ONNX Runtime GenAI to run the model. You can then run the inference example here.

For CPU:

# Download the model directly using the Hugging Face CLI
hf download onnx-community/Qwen3-8B-ONNX --include onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/* --local-dir .

# Install the CPU package of ONNX Runtime GenAI
pip install onnxruntime-genai

# Please adjust the model directory (-m) accordingly
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-chat.py -o model-chat.py
python model-chat.py -m onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/ -e follow_config

For CUDA:

# Download the model directly using the Hugging Face CLI
hf download onnx-community/Qwen3-8B-ONNX --include onnxruntime/cuda/cuda-int4-kld-block-128/* --local-dir .

# Install the CUDA package of ONNX Runtime GenAI
pip install onnxruntime-genai-cuda

# Please adjust the model directory (-m) accordingly
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-chat.py -o model-chat.py
python model-chat.py -m onnxruntime/cuda/cuda-int4-kld-block-128 -e follow_config

For WebGPU:

# Download the model directly using the Hugging Face CLI
hf download onnx-community/Qwen3-8B-ONNX --include onnxruntime/webgpu/webgpu-int4-kld-block-32/* --local-dir .

# Install the WebGPU packages of ONNX Runtime and ONNX Runtime GenAI
pip install onnxruntime-webgpu
pip install onnxruntime-genai --no-deps

# Please adjust the model directory (-m) accordingly
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py
curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-chat.py -o model-chat.py
python model-chat.py -m onnxruntime/webgpu/webgpu-int4-kld-block-32 -e follow_config

Model Description#

Developed by: Microsoft
Model type: ONNX
License: Apache 2.0
Model Description: This is a conversion of the Qwen3-8B model for local inference on CPUs and GPUs.
Disclaimer: Model is only an optimization of the base model. Any risk associated with the model is the responsibility of the user of the model. Please verify and test for your scenarios. There may be a slight difference in output from the base model with the optimizations applied. Note that optimizations applied are distinct from fine tuning and thus do not alter the intended uses or capabilities of the model.

Files

28 files · 18.5 GBWeights 486 KB (smallest of 3 ONNX variants). Sizes and sha256 from the signed manifest.

Show all 28 files with their sha256

File	Size	sha256
.gitattributes	1.62 KB	6916ac4a9fd92a5964d49b45d4c87999e8dc7287e6a3073ecc6cba692d7a6615
LICENSE	11.3 KB	832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
README.md	4.00 KB	df14edb2ec2771b09959e9b7e1cf8f4871de7c4aab3f1202fa76f024a3022cc7
config.json	728 B	f7c4eadfbbf522470667b797a3c89be2524832d2d599797248dc304fff447c30
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/chat_template.jinja	4.17 KB	a55ee1b1660128b7098723e0abcd92caa0788061051c62d51cbe87d9cf1974d8
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/config.json	10.1 KB	24c139e93c8511b48498e8367b7d6c71f79e94f12704138812a5f11ab699f73b
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/genai_config.json	1.52 KB	b34eed7e1e816402a3a8266eabfc7bd7f7307b045a07a44ee15456699978b784
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/model.onnx	486 KB	f3a99f7d372fa52646e5506c7e539070883be9a07c49312c1f89a23439343f1b
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/model.onnx.data	6.08 GB	5c678fc9776f144d766ec969d344a35d7bdc6b1461caeaa07c6dc231ae263fd4
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/model_config.json	41.6 KB	0ff29fae14e9cafde4db9ba91a4cdd83d5eddc2b48ae5e0170851cff4c365cb2
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/tokenizer.json	11.4 MB	979d160e081df25a1bf7f4e2e8f4c441b5dfdc9a8e84aec9f32e80445e1b59b8
onnxruntime/cpu_and_mobile/cpu-int4-kld-block-128/tokenizer_config.json	663 B	bfd13b57b2e0c2cb582f311c0895f077d16a868c0dc61e4dbbba544a835341d6
onnxruntime/cuda/cuda-int4-kld-block-128/chat_template.jinja	4.17 KB	a55ee1b1660128b7098723e0abcd92caa0788061051c62d51cbe87d9cf1974d8
onnxruntime/cuda/cuda-int4-kld-block-128/config.json	10.1 KB	24c139e93c8511b48498e8367b7d6c71f79e94f12704138812a5f11ab699f73b
onnxruntime/cuda/cuda-int4-kld-block-128/genai_config.json	1.76 KB	49b70e1d9e957452a390e7a6c00ef100192f665be04ceb8b4e5f0201df38f165
onnxruntime/cuda/cuda-int4-kld-block-128/model.onnx	486 KB	733ec0d373a36953554b800cd30dbc7a277c8a071736497dbee679f94e1d9bf4
onnxruntime/cuda/cuda-int4-kld-block-128/model.onnx.data	5.94 GB	c8b4fdf4952081b983732598fc3f553fd2435f833d924491d79216e0c2056f3f
onnxruntime/cuda/cuda-int4-kld-block-128/model_config.json	41.6 KB	0410e54edc452249eb32ac631062567f659961744f4ba2ae903bf8eaf9cf3f8c
onnxruntime/cuda/cuda-int4-kld-block-128/tokenizer.json	11.4 MB	979d160e081df25a1bf7f4e2e8f4c441b5dfdc9a8e84aec9f32e80445e1b59b8
onnxruntime/cuda/cuda-int4-kld-block-128/tokenizer_config.json	663 B	bfd13b57b2e0c2cb582f311c0895f077d16a868c0dc61e4dbbba544a835341d6
onnxruntime/webgpu/webgpu-int4-kld-block-32/chat_template.jinja	4.17 KB	a55ee1b1660128b7098723e0abcd92caa0788061051c62d51cbe87d9cf1974d8
onnxruntime/webgpu/webgpu-int4-kld-block-32/config.json	9.92 KB	643ef5e1ada1a7294b898d885e5c2b507e558bd9e745f9b888d26893e015a6a5
onnxruntime/webgpu/webgpu-int4-kld-block-32/genai_config.json	1.75 KB	5ae0d88c9648d8c5f376312fd3c0848e4e5abfdb8cc9b919f07fd47ebe49b6ae
onnxruntime/webgpu/webgpu-int4-kld-block-32/model.onnx	487 KB	dd11227b49ff020600ef9cb52cfd5e6f62cadb191b53dfc1975c87d9477206b9
onnxruntime/webgpu/webgpu-int4-kld-block-32/model.onnx.data	6.43 GB	9fc6adbf230c2187570ab843a9b05d480cd317a6f74422bc26af621b87080df5
onnxruntime/webgpu/webgpu-int4-kld-block-32/model_config.json	40.6 KB	fd1224ac6d321465e81458219e84ff3a100c8dd2cd83ae9c16bd0df0d404a24f
onnxruntime/webgpu/webgpu-int4-kld-block-32/tokenizer.json	11.4 MB	979d160e081df25a1bf7f4e2e8f4c441b5dfdc9a8e84aec9f32e80445e1b59b8
onnxruntime/webgpu/webgpu-int4-kld-block-32/tokenizer_config.json	663 B	bfd13b57b2e0c2cb582f311c0895f077d16a868c0dc61e4dbbba544a835341d6

VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.

Upstream commit	Date	Contents
eab7c5b1ac pinned now	2026-11-15 05:23 UTC	28 files · 18.5 GB

Use it

$ fw get onnx-community/Qwen3-8B-ONNX
# manifest sha256 dac8d38bd020… · 18.5 GB · 28 files
$ fw verify ./Qwen3-8B-ONNX onnx-community/Qwen3-8B-ONNX

fw.pyChecks the signature and every sha256, tries each source in order, resumes.

$ python3 fw.py get https://freeweights.allternit.com/manifests/onnx-community/Qwen3-8B-ONNX.json --dir ./Qwen3-8B-ONNX
$ python3 fw.py verify ./Qwen3-8B-ONNX ./onnx-community__Qwen3-8B-ONNX.json

hfThe Hugging Face CLI with one setting. Same repo id.

$ HF_ENDPOINT=https://freeweights.allternit.com hf download onnx-community/Qwen3-8B-ONNX --revision eab7c5b1ac4637f6745e5e1d21ffa0bec4228186 --local-dir ./Qwen3-8B-ONNX

Run at home

License

Name: Apache License 2.0
Id: Apache-2.0
Class: Permissive
Redistribution: yes
Listing: Hosted (signed)
Decided: License as declared by the publisher. No person read it. 2026-11-15
Text in: LICENSE
Text sha256: 832dd9e00a68dd83b3c3fb9f5588dad7dcf337a0db50f7d9483f310cd292e92e
Terms gate: No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From:

onnx-community <huggingface.co/onnx-community>Newsgroups: alt.freeweights.qwenSubject: Qwen3-8B-ONNXDate: 2026-11-15Commit: eab7c5bSigned: 2DC28CC7B26A283B ✓

Upstream: onnx-community/Qwen3-8B-ONNX
Pinned commit: eab7c5b1ac4637f6745e5e1d21ffa0bec4228186
Manifest: Qwen3-8B-ONNX.json · .minisig
Manifest sha256: dac8d38bd0202a8119da5bef353af1f25ccfb33d60412fa94227780eb44a20a6
Signature: minisign, Ed25519, key 2DC28CC7B26A283B
Created: 2026-11-15 05:23 UTC
Generator: freeweights-manifest/0.1.0
Takedowns: None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/onnx-community/Qwen3-8B-ONNX.json
$ curl -O https://freeweights.allternit.com/manifests/onnx-community/Qwen3-8B-ONNX.json.minisig
$ minisign -Vm Qwen3-8B-ONNX.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq
$ shasum -a 256 <file>   # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

Parent

Qwen/Qwen3-8B · this model is a quantization of it

No model in the catalog names this one as its parent.

Get it

Download

$ HF_ENDPOINT=https://freeweights.allternit.com hf download onnx-community/Qwen3-8B-ONNX

Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest

minisign, key 2DC28CC7B26A283B. Qwen3-8B-ONNX.json. Provenance. Takedown history: none.

Source

huggingface.co/onnx-community/Qwen3-8B-ONNX

Related models

All of alt.freeweights.qwen →

From:

BAAI <huggingface.co/BAAI>Subject: Aquila-VL-2B-llava-qwen

From: Perciqa <huggingface.co/Perciqa>Subject: Aurora-Code-1

From:

BAAI <huggingface.co/BAAI>Subject: bge-reasoner-embed-qwen3-8b-0923

From: amazon <huggingface.co/amazon>Subject: BMOJOF-primed-HQwen3-8B-Instruct

From: nbeerbower <huggingface.co/nbeerbower>Subject: Bubba-Qwen3.6-27B-LoRA

From: bartowski <huggingface.co/bartowski>Subject: bytkim_Qwen3.8-27B-pi-GGUF