From:

microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.visionSubject:

colipri

Date: 2026-11-15Commit: 892e42eSigned: 2DC28CC7B26A283B ✓

Zero shot image classification model with 258M parameters. Safetensors weights, 1.04 GB in all. Published by microsoft under MIT.

258M params

1.04GBcoliprisafetensorspermissive

FactsParams258MSize1.04 GBFormatssafetensorsTaskzero shot image classificationLicenseMITFrom sourcepermissive

$ fw get microsoft/colipri
# manifest sha256 bef637752d14… · 1.04 GB · 42 files
$ fw verify ./colipri microsoft/colipri

From sourceHosted (signed)microsoft/colipriLast check: 2026-11-15 02:50 UTC

Run at home Get help

Get help: custom fine-tune or private deploy. We review your setup before we quote.

Works with

Run8 Fine-tune7 Convert8 Scan2

Runs with

Ranked by fit to this model (safetensors), then by GitHub stars.

Transformers LocalAI tinygrad Triton Inference Server BentoML CTranslate2 ExLlamaV2 LitServe

Overview

Size: 1.04 GB
Weights: 1.03 GB (safetensors)
Parameters: 258M
Formats: safetensors
Task: zero shot image classification
License: MIT (permissive)
Files: 42, each with a sha256
Publisher: microsoft
Added: 2026-11-15
Section: alt.freeweights.vision

Lineage

Original release from the publisher.

Where these bytes come from

Publisher at the pinned commit→You

Sources are tried in order. A file counts only when its size and sha256 match the manifest. Sources are checked when you ask for them, not on a timer.

Sources

Badge	Host	Last check	Result
From source	huggingface.co	2026-11-15 02:50 UTC	passed

Model card

From the publisher at commit 892e42e. Shown as plain text with links; images open at the publisher.

COLIPRI#

COLIPRI is a 3D vision–language transformer model trained to encode chest CT scans and radiological reports.

Model description#

COLIPRI was trained using tens of thousands of chest CT scans and reports, without any annotations, using multiple objectives to learn strong joint representations of 3D images and text. The procedure is described in detail in Comprehensive language-image pre-training for 3D medical image understanding (Wald et al., ECCV 2026).

The weights shared here correspond to COLIPRI-CRM, our best-performing model in the paper.

Developed by: Microsoft Research
Model type: 3D vision–language encoder
License: MIT

Uses#

COLIPRI is shared for research purposes only. It is not meant to be used for clinical practice.

The encoders be plugged to other models, or used independently or jointly for many downstream tasks, such as:

Image classification with text prompts
Image clustering
Text clustering
Text-to-image retrieval
Image-to-image retrieval
Image-to-text retrieval
Text-to-text retrieval
Image classification with a classifier
Text classification with a classifier
Image segmentation with a decoder
Report generation with a language decoder

Fine-tuning COLIPRI is typically not necessary to obtain good performance in downstream tasks.

Getting started#

Installation#

pip install colipri

Usage examples#

Below we share some usage snippets to get started with COLIPRI. A more complete Jupyter notebook is also available.

First, let's get a 3D chest CT we can use for demonstration. The plotted slices intersect a lung nodule near the heart.

>>> from colipri import load_sample_ct
>>> image = load_sample_ct()
>>> image
ScalarImage(shape: (1, 512, 512, 139); spacing: (0.76, 0.76, 2.50); orientation: LPS+; dtype: torch.IntTensor; memory: 139.0 MiB)

The image looks like this:

Image: Input CT

Now, let's instantiate the model and processor.

>>> from colipri import get_model
>>> from colipri import get_processor
>>> model = get_model().cuda()
>>> processor = get_processor()

Zero-shot classification#

>>> from colipri import ZeroShotImageClassificationPipeline
>>> pipeline = ZeroShotImageClassificationPipeline(model, processor)
>>> pipeline(image, ["No lung nodules present", "Lung nodules present"])
[
    {'score': 0.0002, 'label': 'No lung nodules present'},
    {'score': 0.9998, 'label': 'Lung nodules present'}
]

Feature extraction#

>>> import torch
>>> preprocessed_images = processor.process_images(image)
>>> preprocessed_images[0]
ScalarImage(shape: (1, 192, 192, 192); spacing: (2.00, 2.00, 2.00); orientation: SAR+; dtype: torch.FloatTensor; memory: 27.0 MiB)
>>> images_batch = processor.to_images_batch(preprocessed_images)
images_batch.shape
torch.Size([1, 1, 192, 192, 192])
>>> with torch.no_grad():
...     patch_embeddings = model.encode_image(images_batch)
>>> patch_embeddings.shape
torch.Size([1, 768, 24, 24, 24])
>>> with torch.no_grad():
...     pooled_embeddings = model.encode_image(images_batch, pool=True, project=True)
>>> pooled_embeddings.shape
torch.Size([1, 768])

Biases, risks, and limitations#

COLIPRI was trained with data from Turkey and the USA only, therefore it might be biased towards population in the training data. Underlying biases of the training datasets may not be well characterized.

Environmental impact#

Hardware type: NVIDIA A100 GPUs
Hours used: 72 hours × 4 GPUs = 288 GPU-hours
Cloud provider: Azure
Compute region: West US 2
Carbon emitted: 21.6 kg CO₂ eq.

Compute infrastructure#

COLIPRI was trained on Azure Machine Learning.

Hardware#

Stage	Node type	Num. nodes	GPU type	GPUs per node
Pre-training	`Standard_NC96ads_A100_v4`	1	NVIDIA A100 (80 GB)	4
Evaluation	`Standard_NC24ads_A100_v4`	1	NVIDIA A100 (80 GB)	1

Software#

The main software libraries used in this work were nnSSL for training, TorchIO for preprocessing and augmentation, nifti-zarr-py for data loading, and nnU-Net for segmentation evaluation.

Community#

Below are some works derived from COLIPRI.

Name	Description
NV-Reason-CT	Open 3D CT VLM for Radiologist Chain-of-Thought Reasoning
ORCA-3DCT	Training-free ORCA compression of CT-RATE embeddings

Citation#

BibTeX#

@inbook{Wald2026,
  title = {Comprehensive Language–Image Pre-training for 3D Medical Image Understanding},
  ISBN = {9783032376244},
  ISSN = {1611-3349},
  url = {http://dx.doi.org/10.1007/978-3-032-37624-4_24},
  DOI = {10.1007/978-3-032-37624-4_24},
  booktitle = {Computer Vision – ECCV 2026},
  publisher = {Springer Nature Switzerland},
  author = {Wald,  Tassilo and Hamamci,  Ibrahim Ethem and Gao,  Yuan and Bond-Taylor,  Sam and Sharma,  Harshita and Ilse,  Maximilian and Lo,  Cynthia and Melnichenko,  Olesya and Schwaighofer,  Anton and Codella,  Noel C. F. and Wetscherek,  Maria Teodora and Maier-Hein,  Klaus H. and Korfiatis,  Panagiotis and Salvatelli,  Valentina and Alvarez-Valle,  Javier and Pérez-García,  Fernando},
  year = {2026},
  pages = {388–406}
}

APA#

Wald, T., Hamamci, I. E., Gao, Y., Bond-Taylor, S., Sharma, H., Ilse, M., … Pérez-García, F. (2026). Comprehensive language–image pre-training for 3D medical image understanding. In Lecture Notes in Computer Science. Lecture Notes in Computer Science (pp. 388–406). doi:10.1007/978-3-032-37624-4_24

Model card contact#

Fernando Pérez-García (fperezgarcia@microsoft.com).

Files

42 files · 1.04 GBWeights 1.03 GB (safetensors). Sizes and sha256 from the signed manifest.

Show all 42 files with their sha256

File	Size	sha256
.gitattributes	1.56 KB	fc949d01d5d893d52edbab7b3b879f98c9c23fac5cc9e60989548f2195239bba
.gitignore	119 B	fb8f306e209d1100ab491442d5a09859c97a26235bced8a435423b0801a16cb9
.python-version	5 B	02e735b3dfe1c32833eb550b7ff8ffa17f5f2bc3fa1e7bae61a8f5a3883ce398
COLIPRI_demo.ipynb	1.46 MB	dd763ebd2af1e9f168f55a03e78961f8f8da0163050304d07287bd9fcfe776d3
LICENSE	1.07 KB	18d3bb3458bb9dabcdf5e74dacf256a339b8a6486338ab09a97ba865d3214928
README.md	7.16 KB	7a03b194361fcfdccd480cdd0a5d62e50cfec247a66c0bb5841de73d177ba5ee
assets/input.png	928 KB	9cb8413eaa5628f5499c6fc3a349561c60dfde81351afbb4a3c19b33cd54c547
mise.toml	648 B	62ab8fe3630faed69ec0a8f4792958a87fe6fea52e700460224877463ad5aa4f
model.safetensors	1.03 GB	97d66586507d1f48577b5149777d02e12d25787435a2a295cac5563df9291c14
model_card.md	5.66 KB	f0d3315c62dddc32024b5d8c3fe834545c817eee6c7d8b117c528f461ddfeab6
pyproject.toml	2.14 KB	1983ba17783bdcd10f64d6b3996bac65cce751e9b650422c80565f19a6989f74
src/colipri/__init__.py	317 B	a04b05de1c33a2f1612134dcf3289df4af7535adfd5d1852203ad5b9d8f24638
src/colipri/checkpoint.py	2.56 KB	7f37feae4f0515ef48862f6a1a040bfb49681d62821596474eaff960d1b0fe4a
src/colipri/configs/config.yaml	176 B	9e1564ec31d46b34fe58662ab5fd83d0c3c3032d46d590821b555049e3498153
src/colipri/configs/model/default.yaml	105 B	74130be799048d2197dad13e8a9966fafc0700995650b5779523b062df2e3532
src/colipri/configs/model/image_encoder/backbone/default.yaml	395 B	8e36980a28a2d73e8fdbc5b89a308f1fdc27779adf650d757d2a569102db0e37
src/colipri/configs/model/image_encoder/default.yaml	119 B	9e3d9e7f0cb272830237f10abc308c5403b9f2f705aa305320fa90a2c47b2eab
src/colipri/configs/model/image_encoder/pooler/default.yaml	85 B	6b5d236b1b522b1ba7e6a7541c5f173906808573c4f467770929ce0f9a915b90
src/colipri/configs/model/image_encoder/projector/default.yaml	105 B	7b073d239a2a44eb99b3cecf0141d4d90a364d9d77dc8cc8167e9c81d0ef303f
src/colipri/configs/model/text_encoder/backbone/default.yaml	195 B	20b971697ad89da2aa835886218fb0e185c6fed6bf3bcdcc8ab7a39cfab47758
src/colipri/configs/model/text_encoder/default.yaml	94 B	e7b6eca2b57377063306d9288e0cf850ba67fade152f901b570ba07b78532c53
src/colipri/configs/model/text_encoder/pooler/default.yaml	85 B	6b5d236b1b522b1ba7e6a7541c5f173906808573c4f467770929ce0f9a915b90
src/colipri/configs/processor/default.yaml	101 B	ae707e6a9fd647660134bd3da324835c45e79eaf2ca4f5ef479609d9e58f9c81
src/colipri/configs/processor/image_transform/default.yaml	1.42 KB	ea5fb9053a11d7e9814de19a55b9629f7b73e32202679843b7e0012753aef2f9
src/colipri/configs/processor/tokenizer/default.yaml	221 B	9fae63b8219c319d6de2f55c4dc5585daeda2969aecd7a01f041a9b72e6d2415
src/colipri/defaults.py	452 B	20f2ad37b7e98d3e7f3e2844fcc584471ad5aec68294fdff87a591849ed7adfd
src/colipri/model/__init__.py	0 B	e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
src/colipri/model/base.py	169 B	34d6d0dc0ccd4c6b10a37f39c68f1e9f0811320a7a4e20c5ed64e6c7d7efdb5d
src/colipri/model/image.py	3.87 KB	f1a6f957c21e11d64839ac15da9d548872732d0d9fb70051e23fa15aa9f2db97
src/colipri/model/multimodal.py	7.17 KB	8bd96c6f0aed823b620973c9c6e7fac138a9ba30c2aaa6e3126753a5ab051054
src/colipri/model/text.py	1.51 KB	11ba1b7f54f33b0faca496cb940aaf0b048890e21d5a8f3e507e6fac532fe989
src/colipri/pipelines.py	1.97 KB	2b21fe09404617eec56809242b699f07ac6ac857f5f8afa41f3a887c237c4c56
src/colipri/pooling.py	5.04 KB	05fe47e1edfe23466482223549a3f8c210eb06eced03f7d742082377a85b655e
src/colipri/processor.py	4.42 KB	5f399169342ab0198a49b09500620e195fb4be4e552ec7b436c3354d8b4f479f
src/colipri/sample_data.py	628 B	635a027172ef461ba8436e78d1b8e6da0939d28601f24a98a8cdeb4be34bb92c
src/colipri/types.py	1.61 KB	b0b60943ffa71cbf9c25b67f500ab2ff42f5e5a66a83ae26f9f7c2498fcbd8fd
tests/test_config.py	2.30 KB	c8fba308fa342871f37dfa655aba6c3f7a2d34ddbbe0e9e320040b41eaf0c906
tests/test_custom_config.py	5.08 KB	61351a697769231e2e614d3b2cb6f6d590c208bb193fd9c425f63e700e5f5fea
tests/test_pooling.py	1.58 KB	67d41601c90c98c6d881455d9ca4e41918127281ebeecdcaa814d4f004dd2d88
tests/test_text.py	1.55 KB	48bb3528f5e989f478297c5ab081fd91f9ca72c8930c110e17b89359422ad832
tox.ini	749 B	f7106e48b5033717115f7266fa22a2659b23f24729267254a07d65d8fad194b1
uv.lock	580 KB	6e218213379867cc662402cae5becce04b8b0be9e0a985870bc55ca1db2806db

VersionsEach time the pinned upstream commit or file list changed in this repo. Newest first.

Upstream commit	Date	Contents
892e42ec36 pinned now	2026-11-15 02:50 UTC	42 files · 1.04 GB

Use it

$ fw get microsoft/colipri
# manifest sha256 bef637752d14… · 1.04 GB · 42 files
$ fw verify ./colipri microsoft/colipri

fw.pyChecks the signature and every sha256, tries each source in order, resumes.

$ python3 fw.py get https://freeweights.allternit.com/manifests/microsoft/colipri.json --dir ./colipri
$ python3 fw.py verify ./colipri ./microsoft__colipri.json

hfThe Hugging Face CLI with one setting. Same repo id.

$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/colipri --revision 892e42ec363f6c561b89c459d60fcd95ac1a99bf --local-dir ./colipri

Run at home

License

Name: MIT License
Id: MIT
Class: Permissive
Redistribution: yes
Listing: Hosted (signed)
Decided: License as declared by the publisher. No person read it. 2026-11-15
Text in: LICENSE
Text sha256: 18d3bb3458bb9dabcdf5e74dacf256a339b8a6486338ab09a97ba865d3214928
Terms gate: No extra terms beyond the license.

Read the license text

Licenses are checked against the license table. A custom license passes only after a person reads it.

Provenance

From:

microsoft <huggingface.co/microsoft>Newsgroups: alt.freeweights.visionSubject: colipriDate: 2026-11-15Commit: 892e42eSigned: 2DC28CC7B26A283B ✓

Upstream: microsoft/colipri
Pinned commit: 892e42ec363f6c561b89c459d60fcd95ac1a99bf
Manifest: colipri.json · .minisig
Manifest sha256: bef637752d143aa404c983ac085f59141d26f326af7e4204e43ba3c00caeb092
Signature: minisign, Ed25519, key 2DC28CC7B26A283B
Created: 2026-11-15 02:50 UTC
Generator: freeweights-manifest/0.1.0
Takedowns: None.

How to verify

$ curl -O https://freeweights.allternit.com/manifests/microsoft/colipri.json
$ curl -O https://freeweights.allternit.com/manifests/microsoft/colipri.json.minisig
$ minisign -Vm colipri.json -P RWQ7KGqyx4zCLXtUFOqvfv8L3g+8fpKJ1FsuxPyOYMF+4KufbUxKq1Oq
$ shasum -a 256 <file>   # compare with the Files tab

A manifest whose signature fails must be rejected. A file whose size or sha256 does not match must be discarded. Verify a download

Model tree

No related models in this catalog.Nothing here lists this model as its parent, and it names no parent.

Get it

Download

$ HF_ENDPOINT=https://freeweights.allternit.com hf download microsoft/colipri

Open in Freeweights Studio

Pull, verify and run it in the Allternit app on Desktop, web or phone.

Signed manifest

minisign, key 2DC28CC7B26A283B. colipri.json. Provenance. Takedown history: none.

Source

huggingface.co/microsoft/colipri

Related models

All of alt.freeweights.vision →

From: dima806 <huggingface.co/dima806>Subject: ai_vs_real_image_detection

From: legekka <huggingface.co/legekka>Subject: AI-Anime-Image-Detector-ViT

From:

onnx-community <huggingface.co/onnx-community>Subject: AI-Anime-Image-Detector-ViT-ONNX

From: AICodexLab <huggingface.co/AICodexLab>Subject: answerdotai-ModernBERT-base-ai-detector

From:

onnx-community <huggingface.co/onnx-community>Subject: answerdotai-ModernBERT-base-ai-detector-ONNX

From:

onnx-community <huggingface.co/onnx-community>Subject: bart-large-cnn-ONNX

colipri

Works with

Runs with

Fine-tune with

Convert with

Scan with

Overview

Model card

COLIPRI#

Model description#

Uses#

Getting started#

Installation#

Usage examples#

Zero-shot classification#

Feature extraction#

Biases, risks, and limitations#

Environmental impact#

Compute infrastructure#

Hardware#

Software#

Community#

Citation#

BibTeX#

APA#

Model card contact#

Files

Use it

License

Provenance

How to verify

Model tree

Related models