CatalogWantedMirrorsLedgerOperating StatementDocsStudio statusCard system
alt.freeweights.phi / Phi-3-medium-128k-instruct-onnx-cpu
From: microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.phiSubject:

Phi-3-medium-128k-instruct-onnx-cpu

Date: 2026-11-15Commit: 7996d85Signed: 2DC28CC7B26A283B ✓

Text generation model. Onnx weights, 9.35 GB in all. Published by microsoft under MIT.

phi9.35GBPhi-3-medium-128k-instruct-onnx-cpuonnxpermissive
FactsParamsnot statedSize9.35 GBFormatsonnxTasktext generationLicenseMITFrom sourcepermissive
$ fw get microsoft/Phi-3-medium-128k-instruct-onnx-cpu
# manifest sha256 be2de2573ccc… · 9.35 GB · 15 files
$ fw verify ./Phi-3-medium-128k-instruct-onnx-cpu microsoft/Phi-3-medium-128k-instruct-onnx-cpu
From sourceHosted (signed)microsoft/Phi-3-medium-128k-instruct-onnx-cpuLast check: 2026-11-15 03:47 UTC
Run at homeGet help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run5Convert3Evaluate13Scan12Apps58

Runs with

Ranked by fit to this model (onnx), then by GitHub stars.

Transformers.jsONNX Runtime GenAIUltralyticsgpu-unlockOpenVINO

Convert with

ONNXOptimumOlive
OverviewModel cardFilesUse itLicenseProvenanceModel tree

Overview

Size
9.35 GB
Weights
69.5 MB (ONNX)
Parameters
not stated
Formats
onnx
Task
text generation
License
MIT (permissive)
Files
15, each with a sha256
Publisher
microsoft
Added
2026-11-15
Section
alt.freeweights.phi
Lineage

Original release from the publisher.

Where these bytes come from
Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources
BadgeHostLast checkResult
From sourcehuggingface.co2026-11-15 03:47 UTCpassed

Model card

From the publisher at commit 7996d85. Shown as plain text with links; images open at the publisher.

Phi-3 Medium-128K-Instruct ONNX CPU models#

This repository hosts the optimized versions of Phi-3-medium-128k-instruct to accelerate inference with ONNX Runtime for your CPU.

Phi-3 Medium is a 14B parameter, lightweight, state-of-the-art open model trained with the Phi-3 datasets, which include both synthetic data and the filtered publicly available websites data, with a focus on high-quality and reasoning dense properties. The model belongs to the Phi-3 family with the medium version in two variants: 4K and 128K, which are the context lengths (in tokens) that they can support.

The base model has undergone a post-training process that incorporates both supervised fine-tuning and direct preference optimization for the instruction following and safety measures. When assessed against benchmarks testing common sense, language understanding, math, code, long context, and logical reasoning, Phi-3-Medium-128K-Instruct showcased a robust and state-of-the-art performance among models of the same-size and next-size-up.

Optimized variants of the Phi-3 Medium models are published here in ONNX format and run with ONNX Runtime on CPU and GPU across devices, including server platforms, Windows, and Linux, with the precision best suited to each of these targets.

ONNX Models#

Here are some of the optimized configurations we have added:

  1. ONNX model for INT4 CPU: ONNX model for CPUs using int4 quantization via RTN.

How do you know which is the best ONNX model for you:

How to Get Started with the Model#

To support the Phi-3 models across a range of devices, platforms, and EP backends, we introduce a new API to wrap several aspects of generative AI inferencing. This API makes it easy to drag and drop LLMs straight into your app. To run the early version of these models with ONNX, follow the steps here.

Hardware Supported#

The models are tested on:

  • Intel(R) Core(TM) i9-10920X CPU @ 3.50GHz

Minimum Configuration Required:

  • CPU machine with 16GB RAM
Model Description#
  • Developed by: Microsoft
  • Model type: ONNX
  • Language(s) (NLP): Python, C, C++
  • License: MIT
  • Model Description: This is a conversion of the Phi-3 Medium-128K-Instruct model for ONNX Runtime inference.

Additional Details#

Performance Metrics#

The model runs at ~20 tokens/sec on a Intel(R) Core(TM) i9-10920X CPU @ 3.50GHz.

Appendix#

Model Card Contact#

parinitarahi, kvaishnavi, natke

Contributors#

Kunal Vaishnavi, Sunghoon Choi, Yufeng Li, Akshay Sonawane, Sheetal Arun Kadam, Rui Ren, Edward Chen, Scott McKay, Emma Ning, Natalie Kershaw, Parinita Rahi

Files

15 files · 9.35 GBWeights 69.5 MB (ONNX). Sizes and sha256 from the signed manifest.
Hide all 15 files with their sha256
FileSizesha256
.gitattributes1.57 KB1403a1a07764f40b81196144fc294c8daff49cbc0f1d0a79fcfbf5e35a027311
LICENSE12.5 KBc6186c688d3a12fdad30e4fe4962de3af2897bd47944702962f8cac94a646f94
README.md4.50 KB7ed04044adbd8f2b41b8fb3b636658ce881644dc31f13aa9054f539b7d64208c
config.json3.19 KBa4891a2693084c920487521061b75979e5e23b71699d18c3fa5e3b60c7e683a1
configuration_phi3.py10.4 KB18b06d379f5199bfb8b7da26da14c580f3852e792895d2bf6158b2ef33c92d52
cpu-int4-rtn-block-32-acc-level-4/added_tokens.json293 B5f5c94d30901fe9c961d776d51eadbda4454991a77b53d36e0be0571a8e72a7d
cpu-int4-rtn-block-32-acc-level-4/config.json3.19 KBa4891a2693084c920487521061b75979e5e23b71699d18c3fa5e3b60c7e683a1
cpu-int4-rtn-block-32-acc-level-4/configuration_phi3.py10.4 KB18b06d379f5199bfb8b7da26da14c580f3852e792895d2bf6158b2ef33c92d52
cpu-int4-rtn-block-32-acc-level-4/genai_config.json1.58 KB7470207cacad25ec320b01dac6f1be5f7fa06df552a7dbb295153fd4767bae91
cpu-int4-rtn-block-32-acc-level-4/phi3-medium-128k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx69.5 MB19cb27ede9da507d1e5d09e6fa6ab512d6ef33045cc3084fa3aec4e0fe030310
cpu-int4-rtn-block-32-acc-level-4/phi3-medium-128k-instruct-cpu-int4-rtn-block-32-acc-level-4.onnx.data9.28 GB7d6e56e9192de5d5f16134e230cee72142570aa3b4e778a2c1b71811bef2a09f
cpu-int4-rtn-block-32-acc-level-4/special_tokens_map.json569 B474d699677e264edd3851559548cbbb61801ad8da6b249c81455007602313c34
cpu-int4-rtn-block-32-acc-level-4/tokenizer.json1.84 MB9e2ae3d66819f163cdcfedba5078f3a3af8118c2712491ed60912f77886bda6f
cpu-int4-rtn-block-32-acc-level-4/tokenizer.model500 KB9e556afd44213b6bd1be2b850ebbbd98f5481437a8021afaf58ee7fb1818d347
cpu-int4-rtn-block-32-acc-level-4/tokenizer_config.json3.16 KBc7aad94765fa33f2927557e2f9fb410ce307c98661b4115cc0c36d55a7f8639f
VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.
Upstream commitDateContents
7996d8594b pinned now2026-11-15 03:47 UTC15 files · 9.35 GB

Use it

$ fw get microsoft/Phi-3-medium-128k-instruct-onnx-cpu
# manifest sha256 be2de2573ccc… · 9.35 GB · 15 files
$ fw verify ./Phi-3-medium-128k-instruct-onnx-cpu microsoft/Phi-3-medium-128k-instruct-onnx-cpu
fw.pyChecks the signature and every sha256, tries each source in order, resumes.
$ python3 fw.py get https://freeweights.allternit.com/manifests/microsoft/Phi-3-medium-128k-instruct-onnx-cpu.json --dir ./Phi-3-medium-128k-instruct-onnx-cpu $ python3 fw.py verify ./Phi-3-medium-128k-instruct-onnx-cpu ./microsoft__Phi-3-medium-128k-instruct-onnx-cpu.json
hfThe Hugging Face CLI with one setting. Same repo id.
$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/Phi-3-medium-128k-instruct-onnx-cpu --revision 7996d8594bf78d43363e124b1c4330189b986564 --local-dir ./Phi-3-medium-128k-instruct-onnx-cpu
Run at home

License

Name
MIT License
Id
MIT
Class
Permissive
Redistribution
yes
Listing
Hosted (signed)
Decided
License as declared by the publisher. No person read it. 2026-11-15
Text in
LICENSE
Text sha256
c6186c688d3a12fdad30e4fe4962de3af2897bd47944702962f8cac94a646f94
Terms gate
No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From: microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.phiSubject: Phi-3-medium-128k-instruct-onnx-cpuDate: 2026-11-15Commit: 7996d85Signed: 2DC28CC7B26A283B ✓
Upstream
microsoft/Phi-3-medium-128k-instruct-onnx-cpu
Pinned commit
7996d8594bf78d43363e124b1c4330189b986564
Manifest
Phi-3-medium-128k-instruct-onnx-cpu.json · .minisig
Manifest sha256
be2de2573ccc6078221a4c83386ef044add095c64e947a14e05ecd81375f909b
Signature
minisign, Ed25519, key 2DC28CC7B26A283B
Created
2026-11-15 03:47 UTC
Generator
freeweights-manifest/0.1.0
Takedowns
None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/microsoft/Phi-3-medium-128k-instruct-onnx-cpu.json $ curl -O https://freeweights.allternit.com/manifests/microsoft/Phi-3-medium-128k-instruct-onnx-cpu.json.minisig $ minisign -Vm Phi-3-medium-128k-instruct-onnx-cpu.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq $ shasum -a 256 <file> # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.
Get it
Download
$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/Phi-3-medium-128k-instruct-onnx-cpu
Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest
minisign, key 2DC28CC7B26A283B. Phi-3-medium-128k-instruct-onnx-cpu.json. Provenance. Takedown history: none.
Source
huggingface.co/microsoft/Phi-3-medium-128k-instruct-onnx-cpu

Related models

All of alt.freeweights.phi →
From: BAAI <huggingface.co/BAAI>Subject: bunny-phi-2-siglip-lora
From: microsoft <huggingface.co/microsoft>Subject: MediPhi
From: microsoft <huggingface.co/microsoft>Subject: MediPhi-Clinical
From: microsoft <huggingface.co/microsoft>Subject: MediPhi-Guidelines
From: microsoft <huggingface.co/microsoft>Subject: MediPhi-Instruct
From: microsoft <huggingface.co/microsoft>Subject: MediPhi-MedCode