LFM3-VL-2B

Image: Liquid AI

API Try Docs Blog

LFM3-VL-2B#

LFM3-VL-2B is a vision-language model for on-device visual understanding. It takes text, images, and video as input, and combines the LFM3 language model with a new in-house vision encoder.

New in LFM3-VL-2B:

• Video understanding: the first LFM vision model that takes video as input. Ask questions about a recorded lecture, a walkthrough filmed on your phone, or a clip from a camera feed.
• Thinking: the model reasons step by step before it answers. This helps most with harder visual questions, such as math, charts, and multi-step problems.
• Spatial and embodied reasoning: the model finds objects and reasons about how they are placed relative to each other, a strong foundation for robotics applications.

Find more information about the LFM3 family in our release blog post.

Model details#

The LFM3 family includes base, text, vision, audio, omni, and decision models.

| Property | Specification

| Checkpoint | Post-trained vision-language model

| Parameters | 1.92B: 1.73B language model, 164M vision encoder, 25M projector

| Language model | LFM3, 24 layers: 13 gated short convolutions, 5 sliding-window attention, 6 global attention

| Vision encoder | 14 layers: 8 gated depthwise 2D convolutions, 6 bidirectional attention, 16×16 patches with 2D rotary embeddings

| Inputs | Text, images, and video at native resolution. Each 2×2 block of patches becomes one language token. Video is sampled at 2 fps, with 4 to 768 frames by default.

| Context length | 128K tokens (input + output)

| Vocabulary | 128,000 tokens

| Languages | English, Arabic, Chinese, French, German, Italian, Japanese, Korean, Portuguese, Spanish, Vietnamese, Thai, Indonesian, Hindi, Russian, Polish

| License | LFM Open License v1.0

LFM3-VL-2B is designed for on-device visual tasks: questions about video, reading documents, charts, and app screens, finding objects in an image, and spatial reasoning for robots. It is not recommended for coding, multi-turn conversations, or complex reasoning.

Inference#

Choose a checkpoint variant for your inference framework and hardware.

| Checkpoint | Framework | Description

| BF16 | vLLM, SGLang, Transformers | BF16 weights for GPU inference.

| GGUF | llama.cpp, LM Studio | Local inference on CPUs and GPUs.

| MLX | MLX-VLM | 4-bit quantization for Apple Silicon.

| ONNX | ONNX Runtime | Cross-platform inference with ONNX Runtime.

| OpenVINO | OpenVINO | 4-bit quantization for Intel hardware.

Sampling parameters: temperature=0.1, top_p=1.0, top_k=50, min_p=0.0, repetition_penalty=1.1. For image and video preprocessing, use the values in processor_config.json.

Transformers quick start (transformers>=5.21.0):

pip install "transformers>=5.21.0" torch torchvision pillow

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "LiquidAI/LFM3-VL-2B"

model = AutoModelForImageTextToText.from_pretrained(
    model_id, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True
)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://placecats.com/300/200"},
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(model.device)

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        do_sample=True,
        temperature=0.1,
        top_k=50,
        repetition_penalty=1.1,
        max_new_tokens=16384,  # generate() needs an explicit limit; reasoning can be long
    )

generated_ids = output_ids[:, inputs["input_ids"].shape[1] :]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

On-device inference with llama.cpp

For most on-device use cases, we recommend llama.cpp b11960 or newer with the GGUF checkpoints. To run it in a desktop app such as Llama.app or LM Studio, click Use this model on the GGUF page and pick the app.

From a terminal, start a web chat UI and an OpenAI-compatible API at http://localhost:8080. With -hf, llama.cpp downloads the model and the matching vision projector (mmproj):

llama serve -hf LiquidAI/LFM3-VL-2B-GGUF:Q4_0 \
  --temp 0.1 --top-p 1.0 --top-k 50 --min-p 0.0 --repeat-penalty 1.1 \
  --no-reasoning-preserve --port 8080

To ask one question from the command line, use llama cli with the same flags, --image <IMAGE PATH> or --video <VIDEO PATH>, and -p "<PROMPT>". Video input needs ffmpeg in PATH.

GPU inference with vLLM and SGLang

For high-throughput serving on GPU, use vLLM or SGLang. Both serve an OpenAI-compatible API. Support for LFM3-VL in SGLang is in review in sgl-project/sglang#46858, and support in vLLM is in progress. We will update this card when their releases support it.

vLLM:

vllm serve LiquidAI/LFM3-VL-2B \
  --reasoning-parser qwen3 \
  --enable-auto-tool-choice --tool-call-parser qwen3_coder \
  --chat-template-content-format openai \
  --structured-outputs-config '{"backend": "xgrammar", "disable_any_whitespace": true}' \
  --media-io-kwargs '{"video": {"video_backend": "qwen3_vl"}}'

SGLang:

sglang serve --model-path LiquidAI/LFM3-VL-2B --trust-remote-code \
  --reasoning-parser auto \
  --tool-call-parser auto \
  --constrained-json-disable-any-whitespace

Send an image-and-text request with the OpenAI client. Change base_url to http://localhost:30000/v1 for SGLang, or http://localhost:8080/v1 for llama.cpp:

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8000/v1", api_key="EMPTY")

response = client.chat.completions.create(
    model="LiquidAI/LFM3-VL-2B",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "image_url", "image_url": {"url": "https://placecats.com/300/200"}},
                {"type": "text", "text": "Describe this image."},
            ],
        }
    ],
    temperature=0.1,
    extra_body={"top_k": 50, "repetition_penalty": 1.1},
)
print(response.choices[0].message.content)

Video frame rate and frame limit

By default, LFM3-VL-2B samples video at 2 fps, with at least 4 and at most 768 frames per video. Each frame adds vision tokens, so for long videos a lower frame rate or frame limit uses less context and memory.

• Transformers: set the frame rate per call with processor.apply_chat_template(..., processor_kwargs={"fps": 1}). Set the frame limit when you load the processor: AutoProcessor.from_pretrained(model_id, trust_remote_code=True, max_frames=128).
• vLLM: set a server default in --media-io-kwargs, for example '{"video": {"video_backend": "qwen3_vl", "fps": 1, "max_frames": 128}}'. Override it per request with extra_body={"media_io_kwargs": {"video": {"fps": 1, "max_frames": 128}}}.
• SGLang: change fps, min_frames, and max_frames under video_processor in processor_config.json in a local copy of the checkpoint.
• llama.cpp: always uses 2 fps and 4 to 768 frames for this model. --video-fps has no effect.

Chat template

LFM3-VL uses this ChatML format, generated from the checkpoint's chat template:

<|startoftext|><|im_start|>system
You are a helpful assistant trained by Liquid AI.<|im_end|>
<|im_start|>user
<|vision_start|><|image|><|vision_end|>What species is in this picture?<|im_end|>
<|im_start|>assistant
<think>

Use processor.apply_chat_template() with add_generation_prompt=True to format messages automatically. The model thinks in a <think>...</think> block before it answers.

The template inserts <|vision_start|><|image|><|vision_end|> for each image and <|vision_start|><|video|><|vision_end|> for each video. Do not put these tags in your message content.

Reasoning from turns before the latest user message is removed unless preserve_thinking=True.

Tool calling

Pass function schemas through tools=... to apply_chat_template(). The template formats tool definitions and results automatically. Generated calls use XML:

<tool_call>
<function=get_weather>
<parameter=city>
Paris
</parameter>
</function>
</tool_call>

Warning. LFM3's tool call format has changed from the custom LFM2/LFM2.5 format to a superset of qwen3_coder.

Execute the call in your application. Add the parsed assistant call to history as a tool_calls entry, then append the result as a tool message. Keep passing the tool schemas on subsequent requests and preserve the assistant's content and reasoning from the current tool-use turn.

function.arguments must be a Python mapping. Decode JSON strings with json.loads() before applying the template.

See the Tool Use documentation for the full guide.

Tool search

LFM3 supports deferred tools. Set defer_loading=True to keep tool definitions out of the prompt until your harness's search tool returns their names as tool_reference parts. The chat template then inserts their definitions into <additional_tools>.

tools = [
    search_tool,
    {"type": "function", "defer_loading": True, "function": get_weather_schema},
]
messages.append({
    "role": "tool",
    "tool_call_id": call_id,
    "content": [
        {"type": "text", "text": "Found 1 tool."},
        {"type": "tool_reference", "name": "get_weather"},
    ],
})

Your harness implements and runs the search. Keep the full tools list, including deferred tools, on every request.

See Deferred tools and tool search for the full guide.

Fine-tuning#

LFM3-VL-2B is designed for easy customization for a given domain, task, or application via TRL.

| Training | Use case | Notebook

| SFT | Extract structured JSON from receipt photos | Open in Colab

| GRPO | Transcribe images of equations to LaTeX | Open in Colab

Performance#

Benchmarks#

Across 62 vision benchmarks, LFM3-VL-2B scores at least 8 points ahead of all models of similar size. It matches Gemma 4 12B, is within 2 points of GPT-5.6 Luna, and scores higher than Gemini 3.5 Flash Lite.

| Category | LFM3-VL 2B | LFM3-VL 2B (no think) | LFM2.5-VL 3B | MiniCPM-V 4.7 1B | Qwen3.5 2B | Qwen3.5 4B | InternVL3.5 4B | Gemma 4 E2B | Gemma 4 12B | GPT-5.6 Luna | Gemini 3.5 Flash Lite

| General | 66.0 | 56.4 | 57.6 | 59.8 | 64.0 | 73.0 | 61.2 | 56.9 | 74.0 | 72.2 | 74.1

| Perception | 75.6 | 68.9 | 67.9 | 65.5 | 71.2 | 77.1 | 63.2 | 59.1 | 73.5 | 71.0 | 73.0

| Document, OCR & Chart | 83.9 | 71.6 | 71.2 | 73.0 | 79.1 | 84.3 | 72.1 | 72.3 | 85.7 | 84.4 | 82.3

| Math & logical reasoning | 68.8 | 41.0 | 40.6 | 51.8 | 61.0 | 78.4 | 46.4 | 51.8 | 79.3 | 77.3 | 66.7

| Visual grounding | 77.3 | 72.6 | 68.3 | 51.1 | 61.2 | 67.0 | 63.6 | 41.2 | 74.1 | 72.7 | 63.5

| Embodied reasoning | 64.4 | 57.9 | 48.0 | 45.3 | 51.2 | 63.2 | 48.9 | 41.5 | 59.1 | 61.5 | 50.3

| GUI understanding | 83.1 | 81.5 | 66.0 | 45.3 | 67.2 | 81.1 | 68.5 | 44.3 | 76.1 | 82.5 | 71.0

| Video | 61.2 | 54.0 | 41.3 | 53.7 | 57.0 | 67.5 | 59.7 | 46.6 | 58.9 | 72.8 | 70.0

| Multilingual | 84.4 | 76.2 | 81.2 | 77.4 | 78.6 | 83.9 | 79.0 | 73.4 | 86.1 | 82.1 | 85.8

| Agentic | 42.5 | 32.9 | 21.1 | 16.9 | 15.2 | 31.3 | N/A | 23.8 | 43.6 | 56.1 | 62.4

| Average | 71.4 | 62.1 | 57.6 | 57.5 | 64.0 | 73.4 | 61.1 | 53.6 | 71.7 | 73.4 | 70.0

All 62 benchmarks

| Benchmark | LFM3-VL 2B | LFM3-VL 2B (no think) | LFM2.5-VL 3B | MiniCPM-V 4.7 1B | Qwen3.5 2B | Qwen3.5 4B | InternVL3.5 4B | Gemma 4 E2B | Gemma 4 12B | GPT-5.6 Luna | Gemini 3.5 Flash Lite

| General | | |

| MME | 86.9 | 76.7 | 74.3 | 82.7 | 83.9 | 88.3 | 82.2 | 79.6 | 85.7 | 74.8 | 89.2

| MMStar | 74.4 | 62.2 | 63.3 | 68.1 | 71.5 | 76.6 | 65.3 | 61.2 | 77.7 | 72.2 | 73.1

| SimpleVQA | 36.3 | 27.7 | 32.0 | 36.5 | 38.0 | 45.8 | 33.4 | 28.4 | 42.5 | 53.2 | 56.9

| SEED-Bench (image) | 78.6 | 76.8 | 77.7 | 76.6 | 76.9 | 78.9 | 76.4 | 75.2 | 80.6 | 76.2 | 79.5

| MMBench (dev EN v1.1) | 86.1 | 76.5 | 81.0 | 83.4 | 80.0 | 86.5 | 81.1 | 70.9 | 88.2 | 84.4 | 88.2

| MMMU (val) | 55.7 | 48.6 | 47.2 | 51.0 | 58.1 | 74.6 | 57.8 | 57.3 | 76.7 | 75.2 | 72.0

| MMMU-Pro | 45.7 | 35.0 | 33.2 | 37.3 | 50.3 | 67.2 | 38.4 | 42.7 | 69.1 | 70.9 | 63.2

| MM-Vet (v1) | 57.4 | 44.3 | 48.2 | 46.8 | 53.4 | 59.9 | 65.3 | 44.8 | 64.1 | 67.1 | 73.3

| MM-IFEval | 72.8 | 54.7 | 60.6 | 58.9 | 62.0 | 79.5 | 57.3 | 61.6 | 83.0 | 82.0 | 73.2

| MuirBench | 66.4 | 61.2 | 58.2 | 57.0 | 65.5 | 72.8 | 54.8 | 47.5 | 71.9 | 66.0 | 72.7

| Perception | | |

| BLINK | 71.1 | 68.7 | 61.0 | 60.7 | 63.5 | 69.8 | 57.4 | 52.9 | 70.9 | 68.6 | 68.9

| CountBenchQA | 93.0 | 86.4 | 87.1 | 86.4 | 87.1 | 92.6 | 81.3 | 78.2 | 95.9 | 91.0 | 90.1

| HallusionBench | 65.6

The publisher's README is longer; it is cut here. Read the rest at the source.