Ministral-3-14B-Base-2512-bnb-4bit

Unsloth Dynamic 2.0 achieves superior accuracy & outperforms other leading quants.

https://github.com/unslothai/unsloth/ https://discord.gg/unsloth https://docs.unsloth.ai/

Ministral 3 14B Base 2512#

The largest model in the Ministral 3 family, Ministral 3 14B offers frontier capabilities and performance comparable to its larger Mistral Small 3.2 24B counterpart. A powerful and efficient language model with vision capabilities.

This model is the base pre-trained version, not fine-tuned for instruction or reasoning tasks, making it ideal for custom post-training processes. For instruction and chat based use cases, we recommend using Ministral 3 14B Instruct 2512.

The Ministral 3 family is designed for edge deployment, capable of running on a wide range of hardware. Ministral 3 14B can even be deployed locally, fitting in 32GB of VRAM in BF16, and less than 24GB of RAM/VRAM when quantized.

Key Features#

Ministral 3 14B consists of two main architectural components:

• 13.5B Language Model
• 0.4B Vision Encoder

The Ministral 3 14B Base model offers the following capabilities:

• Vision: Enables the model to analyze images and provide insights based on visual content, in addition to text.
• Multilingual: Supports dozens of languages, including English, French, Spanish, German, Italian, Portuguese, Dutch, Chinese, Japanese, Korean, Arabic.
• Edge-Optimized: Delivers best-in-class performance at a small scale, deployable anywhere.
• Apache 2.0 License: Open-source license allowing usage and modification for both commercial and non-commercial purposes.
• Large Context Window: Supports a 256k context window.

Use Cases#

Private AI deployments where advanced capabilities meet practical hardware constraints:

• Private/custom chat and AI assistant deployments in constrained environments
• Advanced local agentic use cases
• Fine-tuning and specialization
• And more...

Bringing advanced AI capabilities to most environments.

Ministral 3 Family#

Other formats available here.

Benchmark Results#

We compare Ministral 3 to similar sized models.

Reasoning#

Instruct#

Base#

Usage#

The model can be used with the following frameworks;

• vllm: See here
• transformers: See here

vLLM#

We recommend using this model with vLLM.

Installation#

Make sure to install most recent vllm:

uv pip install -U vllm \
    --torch-backend=auto \
    --extra-index-url https://wheels.vllm.ai/nightly

Doing so should automatically install mistral_common >= 1.8.6.

To check:

python -c "import mistral_common; print(mistral_common.__version__)"

You can also make use of a ready-to-go docker image or on the docker hub.

Serve#

To fully exploit the Ministral-3-14B-Base-2512 we recommed using 2xH200 GPUs for deployment due to its large context. However if you don't need a large context, you can fall back to a single GPU.

A simple launch command is:

vllm serve mistralai/Ministral-3-14B-Base-2512 --tensor-parallel-size 2 \
  --tokenizer_mode mistral --config_format mistral --load_format mistral

Additional flags:

• You can set --max-model-len to preserve memory. By default it is set to 262144 which is quite large but not necessary for most scenarios.
• You can set --max-num-batched-tokens to balance throughput and latency, higher means higher throughput but higher latency.

Usage of the model#

Here we asumme that the model mistralai/Ministral-3-14B-Base-2512 is served and you can ping it to the domain localhost with the port 8000 which is the default for vLLM.

Test Base

Quick test with the base model.

from openai import OpenAI

# Modify OpenAI's API key and API base to use vLLM's API server.
openai_api_key = "EMPTY"
openai_api_base = "http://localhost:8000/v1"

TEMP = 0.15
MAX_TOK = 256

client = OpenAI(
    api_key=openai_api_key,
    base_url=openai_api_base,
)

models = client.models.list()
model = models.data[0].id

response = client.completions.create(
    model=model,
    prompt="What is the best thing in the universe ?",
    temperature=TEMP,
    max_tokens=MAX_TOK,
)

print(response.choices[0].text)

Transformers#

You can also use Ministral 3 14B Base 2512 with Transformers ! Make sure to install Transformers from its first v5 release candidate or from "main":

pip install transformers==5.0.0rc0

To make the best use of our model with Transformers make sure to have installed mistral-common >= 1.8.6 to use our tokenizer.

pip install mistral-common --upgrade

Then load our tokenizer along with the model and generate:

Python snippet

from transformers import Mistral3ForConditionalGeneration, MistralCommonBackend, FineGrainedFP8Config

model_id = "mistralai/Ministral-3-14B-Base-2512"
model = Mistral3ForConditionalGeneration.from_pretrained(
    model_id,
    device_map="auto",
)
tokenizer = MistralCommonBackend.from_pretrained(model_id)

input_ids = tokenizer.encode("Once about a time, France was a", return_tensors="pt")
input_ids = input_ids.to("cuda")

output = model.generate(
    input_ids,
    max_new_tokens=30,
)[0]

decoded_output = tokenizer.decode(output[len(input_ids[0]):])
print(decoded_output)

License#

This model is licensed under the Apache 2.0 License.

You must not use this model in a manner that infringes, misappropriates, or otherwise violates any third party’s rights, including intellectual property rights.