alt.freeweights.tools / Triton Inference Server

From: triton-inference-server <github.com/triton-inference-server>Newsgroups: alt.freeweights.toolsSubject:

Triton Inference Server

Release: v2.74.0 (2026-11-02)Commit: 46a0d50License: BSD-3-Clause

The Triton Inference Server provides an optimized cloud and edge inferencing solution.

inference enginev2.74.0releaseTriton Inference ServerPythonBSD-3-Clause

FactsLicenseBSD-3-ClauseLanguagePythonKindinference engineRelease2026-11-02PlatformsLinuxLinks to the projectpermissive

$ git clone -b r26.10 https://github.com/triton-inference-server/server.git

Overview Install Use with Freeweights License Provenance

Overview

NVIDIA's server for deploying models from many frameworks on GPUs and CPUs.

Kind: inference engine
License: BSD-3-Clause
Language: Python
Platforms: Linux
Latest release: v2.74.0 (2026-11-02)
Repository: triton-inference-server/server

Topicsclouddatacenterdeep-learningedgegpuinferencemachine-learning

Freeweights links to the project and does not host tool binaries.

Install

$ git clone -b r26.10 https://github.com/triton-inference-server/server.git

$ git clone -b r26.10 https://github.com/triton-inference-server/server.git

Checked against the project's README on 2026-11-15. The project's own instructions win when they differ: install docs

Freeweights links to the project and does not host tool binaries.

Use with Freeweights

Run a catalog model

Download the weights from Freeweights first (HF_ENDPOINT or fw.py), then point the tool at the folder.

Catalog models it can run (6105)

135M-lc-100k-rope-12B a2-rl-e2egit_large-50-32B a3-rl-DCAgent_exp_rpt_e2egit-large-15-8B a3-rl-DCAgent_exp_rpt_e2egit-v2-10-8B a3-rl-DCAgent_exp_rpt_pymethods2test-large-80-8B a3-rl-DCAgent_exp_rpt_unitsyn-python-large-10-8B a3-rl-DCAgent_mix_h2_language_proportional-65-8B a3-rl-DCAgent_mix_h4_binary_easy-50-8B a3-rl-laion_exp_rpt_ghactions-v3-20-8B a3-rl-laion_nemotron-gym-agent-calendar-80-8B ablation-pymethods2test-seqmean-arm0-tis-15-8B ACE-Step ACE-Step-v1-3.5B AesCode-32B AesCode-8B AFM-4.5B AFM-4.5B-Base AgentCPM-Report Agents-A1-4B ai_vs_real_image_detection AI-Anime-Image-Detector-ViT AI21-Jamba-Reasoning-3B AI21-Jamba2-Mini AlchemistCoder-L-7B Alexandria-Gemma-4-12B-it-Ornith9-Summaries-LoRA-r128 Alexandria-Gemma-4-12B-it-Ornith9-Summaries-LoRA-r64 Alexandria-Gemma-4-12B-it-Qwen27-Knowledge-Units-LoRA-r128 Alexandria-Gemma-4-12B-it-Qwen27-Summaries-LoRA-r128 Alexandria-Gemma-4-12B-it-Qwen27-Summaries-LoRA-r64 Alexandria-Gemma-4-E4B-it-Qwen27-Knowledge-Units-LoRA-r128 all-MiniLM-L6-v1 all-MiniLM-L6-v2 all-MiniLM-L6-v2-4bit all-mpnet-base-v1 ALLaM-7B-Instruct-preview allenai-specter AM-Thinking-v1 AMD-OLMo-1B AMD-OLMo-1B-SFT AMD-OLMo-1B-SFT-DPO answerdotai-ModernBERT-base-ai-detector Apertus-70B-2509 Apertus-70B-Instruct-2509 Apertus-8B-2509 Apertus-8B-Instruct-2509 Apertus-8B-Instruct-2509-unsloth-bnb-4bit Apertus-v1.1-0.5B Apertus-v1.1-0.5B-Instruct Apertus-v1.1-1.5B Apertus-v1.1-1.5B-Instruct Apertus-v1.1-4B Apertus-v1.1-4B-Instruct Apex-35B-V1.0 Apodex-1.1-mini Apriel-1.5-15b-Thinker Apriel-1.6-15b-Thinker Apriel-Nemotron-15b-Thinker Aquila-135M-Intermediate Aquila-VL-2B-Intermediate Aquila-VL-2B-llava-qwen

and 6045 more.

License

License: BSD-3-Clause
Class: Permissive

Read the license

Freeweights links to the project and does not host tool binaries.

Provenance

From: triton-inference-server <github.com/triton-inference-server>Newsgroups: alt.freeweights.toolsSubject: Triton Inference ServerRelease: v2.74.0 (2026-11-02)Commit: 46a0d50License: BSD-3-Clause

Repository: triton-inference-server/server
Pinned release: v2.74.0, 2026-11-02
Release commit: 46a0d50f54259cf82c6119bd74afaf7f200281b3
Default branch: main 1751a2c84904ef21d7c424179891ab7e0f7f5a89
Fetched: 2026-11-15
Takedowns: None.

Freeweights links to the project and does not host tool binaries.

Get it

Install Project repository

Works with

6105 catalog models. See the list