ONNX Runtime GenAI

Release: v0.19.0 (2026-11-04)Commit: d01b244License: MIT

Generation loop for ONNX models: run language models on CPU, GPU and NPUs with ONNX Runtime.

inference engine

v0.19.0releaseONNX Runtime GenAIC++MIT

FactsLicenseMITLanguageC++Kindinference engineRelease2026-11-04PlatformsLinux, macOS, WindowsLinks to the projectpermissive

$ pip install onnxruntime-genai

Overview Install Use with Freeweights License Provenance

Overview

Generation loop for ONNX models: run language models on CPU, GPU and NPUs with ONNX Runtime.

Kind: inference engine
License: MIT
Language: C++
Platforms: Linux, macOS, Windows
Latest release: v0.19.0 (2026-11-04)
Repository: microsoft/onnxruntime-genai

Freeweights links to the project and does not host tool binaries.

Install

$ pip install onnxruntime-genai

$ pip install onnxruntime-genai

Checked against the project's README on 2026-11-14. The project's own instructions win when they differ: install docs

Freeweights links to the project and does not host tool binaries.

Use with Freeweights

Run a catalog model

Download the weights from Freeweights first (HF_ENDPOINT or fw.py), then point the tool at the folder.

Catalog models it can run (218)

AFM-4.5B-ONNX AMD-OLMo-1B AMD-OLMo-1B-SFT AMD-OLMo-1B-SFT-DPO Apertus-8B-Instruct-2509-ONNX Apertus-v1.1-0.5B-Instruct-ONNX Apertus-v1.1-0.5B-Instruct-QAD-INT4-ONNX Apertus-v1.1-1.5B-Instruct-ONNX Apertus-v1.1-1.5B-Instruct-QAD-INT4-ONNX Apertus-v1.1-4B-Instruct-ONNX Apertus-v1.1-4B-Instruct-QAD-INT4-ONNX Baguettotron-ONNX Biggie-SmoLlm-0.4B-ONNX Bonsai-1.7B-ONNX Carbon-3B-ONNX Carbon-500M-ONNX DeepScaleR-1.5B-Preview-ONNX DeepSeek-R1-Distill-Qwen-1.5B-ONNX DialoGPT-small-ONNX Dolphin-ONNX ERNIE-4.5-0.3B-ONNX Fara-7B-onnx Fara-7B-Onnx Florence-2-base Florence-2-base-ft Florence-2-large Florence-2-large-ft gpt-oss-20b-ONNX gpt2 gpt2-large gpt2-large-ONNX gpt2-medium gpt2-medium-ONNX gpt2-mini-ONNX gpt2-xl-ONNX granite-3.0-2b-instruct granite-4.0-1b-ONNX-web granite-4.0-350m-ONNX-web granite-4.0-h-1b-ONNX granite-4.0-h-350m-ONNX granite-4.0-h-micro-ONNX granite-4.0-micro-ONNX-web granite-docling-258M-ONNX helium-1-preview-2b-ONNX Jan-nano-ONNX LightOnOCR-2-1B-ONNX Ministral-3-3B-Instruct-2512-ONNX mistral-7b-instruct-v0.2-ONNX Monad-ONNX NuExtract-1.5-tiny-ONNX OLMo-1B-hf Olmo-3-7B-Instruct-ONNX Olmo-Hybrid-Think-SFT-7B-ONNX OpenMath-Nemotron-1.5B-ONNX OpenReasoning-Nemotron-1.5B-ONNX Phi-3-medium-128k-instruct-onnx-cpu Phi-3-medium-128k-instruct-onnx-cuda Phi-3-medium-128k-instruct-onnx-directml Phi-3-medium-4k-instruct-onnx-cpu Phi-3-medium-4k-instruct-onnx-cuda

and 158 more.

License

License: MIT
Class: Permissive

Read the license

Freeweights links to the project and does not host tool binaries.

Provenance

From:

microsoft <github.com/microsoft>Newsgroups: alt.freeweights.toolsSubject: ONNX Runtime GenAIRelease: v0.19.0 (2026-11-04)Commit: d01b244License: MIT

Repository: microsoft/onnxruntime-genai
Pinned release: v0.19.0, 2026-11-04
Release commit: d01b2441a72d4e3223415b9b08474c6e65fbc028
Default branch: main da29a76b66e7d69658e2dfd1026322a3b370c2a1
Fetched: 2026-11-14
Takedowns: None.

Freeweights links to the project and does not host tool binaries.

Get it

Install Project repository

Works with

218 catalog models. See the list