CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.tools / ExLlamaV3
From: turboderp-org <github.com/turboderp-org>Newsgroups: alt.freeweights.toolsSubject:

ExLlamaV3

Release: v1.6.7 (2026-11-13)Commit: 8c593e4License: MIT

An optimized quantization and inference library for running LLMs locally on modern consumer-class GPUs

inference enginev1.6.7releaseExLlamaV3PythonMIT
FactsLicenseMITLanguagePythonKindinference engineRelease2026-11-13PlatformsLinux (NVIDIA), Windows (NVIDIA)Links to the projectpermissive
$ pip install exllamav3
OverviewInstallUse with FreeweightsLicenseProvenance

Overview

Fast local inference for language models quantized to the EXL3 format on NVIDIA GPUs.

Kind
inference engine
License
MIT
Language
Python
Platforms
Linux (NVIDIA), Windows (NVIDIA)
Latest release
v1.6.7 (2026-11-13)
Repository
turboderp-org/exllamav3

Freeweights links to the project and does not host tool binaries.

Install

$ pip install exllamav3
$ pip install exllamav3

Checked against the project's README on 2026-11-14. The project's own instructions win when they differ: install docs

Freeweights links to the project and does not host tool binaries.

Use with Freeweights

Run a catalog model

Download the weights from Freeweights first (HF_ENDPOINT or fw.py), then point the tool at the folder.

Catalog models it can run (3137)

135M-lc-100k-rope-12Ba2-rl-e2egit_large-50-32BAesCode-32BAesCode-8BAFM-4.5BAFM-4.5B-BaseAgentCPM-ReportAgents-A1-4BAI21-Jamba-Reasoning-3BAI21-Jamba2-MiniAlchemistCoder-L-7BAlexandria-Gemma-4-12B-it-Ornith9-Summaries-LoRA-r128Alexandria-Gemma-4-12B-it-Ornith9-Summaries-LoRA-r64Alexandria-Gemma-4-12B-it-Qwen27-Knowledge-Units-LoRA-r128Alexandria-Gemma-4-12B-it-Qwen27-Summaries-LoRA-r128Alexandria-Gemma-4-12B-it-Qwen27-Summaries-LoRA-r64Alexandria-Gemma-4-E4B-it-Qwen27-Knowledge-Units-LoRA-r128ALLaM-7B-Instruct-previewAM-Thinking-v1AMD-OLMo-1BAMD-OLMo-1B-SFTAMD-OLMo-1B-SFT-DPOApertus-70B-2509Apertus-70B-Instruct-2509Apertus-8B-2509Apertus-8B-Instruct-2509Apertus-8B-Instruct-2509-unsloth-bnb-4bitApertus-v1.1-0.5BApertus-v1.1-0.5B-InstructApertus-v1.1-1.5BApertus-v1.1-1.5B-InstructApertus-v1.1-4BApertus-v1.1-4B-InstructApex-35B-V1.0Apodex-1.1-miniApriel-1.5-15b-ThinkerApriel-1.6-15b-ThinkerApriel-Nemotron-15b-ThinkerAquila-135M-IntermediateAquila-VL-2B-IntermediateAquilaDense-16BAquilaMoEAquilaMoE-SFTArcee-Maestro-7B-PreviewArete-27BAREX-2AREX-2-6bitAREX-BaseAREX-TurboAstaBrief_8BAstaBrief_8B_SFTAurora-Code-1BaguettotronBaichuan-M2-32BBAR-2x7B-BaseBAR-2x7B-CodeBAR-2x7B-Code-SFTBAR-2x7B-MathBAR-2x7B-Math-SFTBAR-2x7B-Safety

and 3077 more.

License

License
MIT
Class
Permissive

Read the license

Freeweights links to the project and does not host tool binaries.

Provenance

From: turboderp-org <github.com/turboderp-org>Newsgroups: alt.freeweights.toolsSubject: ExLlamaV3Release: v1.6.7 (2026-11-13)Commit: 8c593e4License: MIT
Repository
turboderp-org/exllamav3
Pinned release
v1.6.7, 2026-11-13
Release commit
8c593e4e5c88da76a4aaf8f1b1838bb7c1fce4e0
Default branch
master 8c593e4e5c88da76a4aaf8f1b1838bb7c1fce4e0
Fetched
2026-11-14
Takedowns
None.

Freeweights links to the project and does not host tool binaries.

Get it
InstallProject repository
Works with

3137 catalog models. See the list