alt.freeweights.phi / Phi-4-multimodal-instruct-onnx

From:

microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.phiSubject:

Phi-4-multimodal-instruct-onnx

Date: 2026-11-15Commit: 295df4cSigned: 2DC28CC7B26A283B ✓

Automatic speech recognition model. Onnx weights, 5.14 GB in all. Published by microsoft under MIT.

phi

5.14GBPhi-4-multimodal-instruct-onnxonnxpermissive

FactsParamsnot statedSize5.14 GBFormatsonnxTaskautomatic speech recognitionLicenseMITFrom sourcepermissive

$ fw get microsoft/Phi-4-multimodal-instruct-onnx
# manifest sha256 d9102288c8be… · 5.14 GB · 29 files
$ fw verify ./Phi-4-multimodal-instruct-onnx microsoft/Phi-4-multimodal-instruct-onnx

From sourceHosted (signed)microsoft/Phi-4-multimodal-instruct-onnxLast check: 2026-11-15 04:02 UTC

Run at home Get help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run17 Convert3 Scan1

Runs with

Ranked by fit to this model (onnx), then by GitHub stars.

Whisper Ultralytics faster-whisper WhisperX FunASR NeMo Speech Pipecat LiveKit Agents

Speech-to-Speech SpeechBrain Moshi OpenVINO

Show all 17

pyannote.audio RealtimeSTT ESPnet WhisperKit WhisperLive

Convert with

ONNX

Optimum

Olive

Scan with

ModelScan

Overview Model card Files Use it License Provenance Model tree

Overview

Size: 5.14 GB
Weights: 12.0 KB (smallest of 4 ONNX variants)
Parameters: not stated
Formats: onnx
Task: automatic speech recognition
License: MIT (permissive)
Files: 29, each with a sha256
Publisher: microsoft
Added: 2026-11-15
Section: alt.freeweights.phi

Lineage

Original release from the publisher.

Where these bytes come from

Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources

Badge	Host	Last check	Result
From source	huggingface.co	2026-11-15 04:02 UTC	passed

Model card

From the publisher at commit 295df4c. Shown as plain text with links; images open at the publisher.

Phi-4 Multimodal Instruct ONNX models#

Introduction#

This is an ONNX version of the Phi-4 multimodal model that is quantized to int4 precision to accelerate inference with ONNX Runtime.

Model Run#

For CPU: stay tuned or follow this tutorial to generate your own ONNX models for CPU!

<!-- ```bash

Download the model directly using the Hugging Face CLI#

huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/* --local-dir .

Install the CPU package of ONNX Runtime GenAI#

pip install --pre onnxruntime-genai

Please adjust the model directory (-m) accordingly#

curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/common.py -o common.py curl https://raw.githubusercontent.com/microsoft/onnxruntime-genai/main/examples/python/model-mm.py -o model-mm.py python model-mm.py -m cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4 -e cpu


For CUDA:

Download the model directly using the Hugging Face CLI#

huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include gpu/* --local-dir .

Install the CUDA package of ONNX Runtime GenAI#

pip install --pre onnxruntime-genai-cuda

Please adjust the model directory (-m) accordingly#


For DirectML:

Download the model directly using the Hugging Face CLI#

huggingface-cli download microsoft/Phi-4-multimodal-instruct-onnx --include gpu/* --local-dir .

Install the DML package of ONNX Runtime GenAI#

pip install --pre onnxruntime-genai-directml

Please adjust the model directory (-m) accordingly#


You will be prompted to provide any images, audios, and a prompt.

The performance of the text component is similar to the [Phi-4 mini ONNX models](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx/blob/main/README.md)

### Model Description

- Developed by: Microsoft
- Model type: ONNX
- License: MIT
- Model Description: This is a conversion of Phi4 multimodal model for ONNX Runtime inference.

Disclaimer: Model is only an optimization of the base model, any risk associated with the model is the responsibility of the user of the model. Please verify and test for you scenarios. There may be a slight difference in output from the base model with the optimizations applied.

### Base Model

Phi-4-multimodal-instruct is a lightweight open multimodal foundation model that leverages the language, vision, and speech research and datasets used for Phi-3.5 and 4.0 models. The model processes text, image, and audio inputs, generating text outputs, and comes with 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning, and direct preference optimization to support precise instruction adherence and safety measures.

See details [here](https://huggingface.co/microsoft/Phi-4-multimodal-instruct/blob/main/README.md)

Files

29 files · 5.14 GBWeights 12.0 KB (smallest of 4 ONNX variants). Sizes and sha256 from the signed manifest.

Show all 29 files with their sha256

File	Size	sha256
.gitattributes	1.66 KB	80f03004afedeb2d28c10ee5e42d26298fee39e4768b2a7d70d29f32ca2f8604
LICENSE	1.14 KB	c2cfccb812fe482101a8f04597dfc5a9991a6b2748266c47ac91b6a5aae15383
README.md	3.79 KB	3e40e8fe47cbe6e50a98ada870119bacc28e35c238ec751e032f7bac4497c60f
config.json	4.63 KB	49e1c05f93d43d7f17715b779a2576235b019f587285d7d914e5b05156253f62
gpu/gpu-int4-rtn-block-32/added_tokens.json	249 B	d4f2aceb0f20b71dd1f4bcc7e052e4412946bf281840b8f83d39f259571af486
gpu/gpu-int4-rtn-block-32/genai_config.json	2.92 KB	d674802d88d850c2f8cf2361f419160f2c739376c78361f00736740c9e6658e3
gpu/gpu-int4-rtn-block-32/merges.txt	2.42 MB	856ce61180bb689282eed6b3a6838bb1f438399be23aefe9d20eb379791fb4ad
gpu/gpu-int4-rtn-block-32/phi-4-mm-embedding.onnx	12.0 KB	e201c5981608c06f93b15860993bf6029cfc3f60401290926c404ab84946dddd
gpu/gpu-int4-rtn-block-32/phi-4-mm-embedding.onnx.data	1.23 GB	85e0389fd112f00584016e1429588410e07f548cbedc5111c84bafabb14c825b
gpu/gpu-int4-rtn-block-32/phi-4-mm-speech.onnx	429 KB	7b3d608bbc3affc2228f065f38962efc59cfb219ab21bcb3add01267b6e97ce3
gpu/gpu-int4-rtn-block-32/phi-4-mm-speech.onnx.data	520 MB	d28dc86d07fe3469e307bf06d6ad05dde254f58b7e5625d9b6f88ac65284878e
gpu/gpu-int4-rtn-block-32/phi-4-mm-speech.onnx_adapter	313 MB	67d2ef0f27d22477f5c48f750b25c67e728b79c72d7c72ae5dd5c2090dd48ddd
gpu/gpu-int4-rtn-block-32/phi-4-mm-text.onnx	237 MB	f84c26e4745958d98628619dbc9f6bc17cf72d9642a34053197301a9f6e10b18
gpu/gpu-int4-rtn-block-32/phi-4-mm-text.onnx.data	2.18 GB	e64a9c88de1e7e316e11a6a86e063a533ce26cc8e92e65003e2daebf6e0106b6
gpu/gpu-int4-rtn-block-32/phi-4-mm-vision.onnx	377 KB	c5ef47554450d7bf96f9dcfe28e6bbf2c99beddcd5dfc8786d69713b7f70e9fd
gpu/gpu-int4-rtn-block-32/phi-4-mm-vision.onnx.data	384 MB	0faf7f9b430c3092737e96e5f98b5509022e8cd5063cc5c05e4987503fbd7728
gpu/gpu-int4-rtn-block-32/phi-4-mm-vision.onnx_adapter	250 MB	b596e59e6092dfda593f449abc7c6437cf46cc1f366e5283475299ae2e7629ab
gpu/gpu-int4-rtn-block-32/special_tokens_map.json	587 B	aff38493227d813e29fcf8406e8e90062f1f031aa47d589325e9c31d89ac7cc3
gpu/gpu-int4-rtn-block-32/speech_processor.json	1.80 KB	77ac9c85f4cc8b947bdd19d77ba71de814f76ef6658130e344ee4ecc2eef7014
gpu/gpu-int4-rtn-block-32/tokenizer.json	15.5 MB	4c1b9f641d4f8b7247b8d5007dd3b6a9f6a87cb5123134fe0d326f14d10c0585
gpu/gpu-int4-rtn-block-32/tokenizer_config.json	3.27 KB	60d63678e0cfc6c6c3deb98419b4326a3ada8680165aa0318389355383208b41
gpu/gpu-int4-rtn-block-32/vision_processor.json	2.01 KB	e40a4384c7f530f73514fd87d3fe9219be2a012fded0037bfa06a5468067623a
gpu/gpu-int4-rtn-block-32/vocab.json	3.91 MB	6cb65a857824fa6615bb1782d95d882617a8bbce1da0317118586b36f39e98bd
onnx/builder.py	24.5 KB	8213d127546124ff6cda7ee2d6ac1181af7b68254c567325d588aae4753cf860
onnx/config.json	4.63 KB	6a51e83822b43cce5531974027de76f26710fee3095c065213607bb3557276c9
onnx/modeling_phi4mm.py	127 KB	90bb1481a02b4ac4d2e49600ad7b7845d6dbe76fa6e83a5a125ada6621692306
onnx/processing_phi4mm.py	32.9 KB	25bfd083965f22a05f1ea499dd063c98e5fcecde9c4e24c24478837ae7c0a079
onnx/speech_conformer_encoder.py	114 KB	7b5a021f96ca3e7938a678f2bea435208fd998640ccb2bce7c64230f8e961eea
onnx/vision_siglip_navit.py	78.5 KB	a45f0ada444ef91756d3776b65a9908f0359b2a45f10e1695d48999afc94cc6c

VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.

Upstream commit	Date	Contents
295df4cddf pinned now	2026-11-15 04:02 UTC	29 files · 5.14 GB

Use it

$ fw get microsoft/Phi-4-multimodal-instruct-onnx
# manifest sha256 d9102288c8be… · 5.14 GB · 29 files
$ fw verify ./Phi-4-multimodal-instruct-onnx microsoft/Phi-4-multimodal-instruct-onnx

fw.pyChecks the signature and every sha256, tries each source in order, resumes.

$ python3 fw.py get https://freeweights.allternit.com/manifests/microsoft/Phi-4-multimodal-instruct-onnx.json --dir ./Phi-4-multimodal-instruct-onnx
$ python3 fw.py verify ./Phi-4-multimodal-instruct-onnx ./microsoft__Phi-4-multimodal-instruct-onnx.json

hfThe Hugging Face CLI with one setting. Same repo id.

$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/Phi-4-multimodal-instruct-onnx --revision 295df4cddf8c3740df7cd66e5d05f06a872fcaf5 --local-dir ./Phi-4-multimodal-instruct-onnx

Run at home

License

Name: MIT License
Id: MIT
Class: Permissive
Redistribution: yes
Listing: Hosted (signed)
Decided: License as declared by the publisher. No person read it. 2026-11-15
Text in: LICENSE
Text sha256: c2cfccb812fe482101a8f04597dfc5a9991a6b2748266c47ac91b6a5aae15383
Terms gate: No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From:

microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.phiSubject: Phi-4-multimodal-instruct-onnxDate: 2026-11-15Commit: 295df4cSigned: 2DC28CC7B26A283B ✓

Upstream: microsoft/Phi-4-multimodal-instruct-onnx
Pinned commit: 295df4cddf8c3740df7cd66e5d05f06a872fcaf5
Manifest: Phi-4-multimodal-instruct-onnx.json · .minisig
Manifest sha256: d9102288c8be00ce8d3f774a26ab1868be51f1b9f9d299e7a10824ddac5f1820
Signature: minisign, Ed25519, key 2DC28CC7B26A283B
Created: 2026-11-15 04:02 UTC
Generator: freeweights-manifest/0.1.0
Takedowns: None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/microsoft/Phi-4-multimodal-instruct-onnx.json
$ curl -O https://freeweights.allternit.com/manifests/microsoft/Phi-4-multimodal-instruct-onnx.json.minisig
$ minisign -Vm Phi-4-multimodal-instruct-onnx.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq
$ shasum -a 256 <file>   # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.

Get it

Download

$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/Phi-4-multimodal-instruct-onnx

Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest

minisign, key 2DC28CC7B26A283B. Phi-4-multimodal-instruct-onnx.json. Provenance. Takedown history: none.

Source

huggingface.co/microsoft/Phi-4-multimodal-instruct-onnx

Related models

All of alt.freeweights.phi →

From:

BAAI <huggingface.co/BAAI>Subject: bunny-phi-2-siglip-lora

From:

microsoft <huggingface.co/microsoft>Subject: MediPhi

From:

microsoft <huggingface.co/microsoft>Subject: MediPhi-Clinical

From:

microsoft <huggingface.co/microsoft>Subject: MediPhi-Guidelines

From:

microsoft <huggingface.co/microsoft>Subject: MediPhi-Instruct

From:

microsoft <huggingface.co/microsoft>Subject: MediPhi-MedCode