LFM2.5-Encoder-350M

Image: Liquid AI

Try LFM • Docs • LEAP • Discord

LFM2.5-Encoder-350M#

LFM2.5-Encoder is a family of multilingual bidirectional encoders built on the LFM2 architecture, available in two sizes:

• LFM2.5-Encoder-230M — a lightweight encoder for tight latency and memory budgets, punching above its size class.
• LFM2.5-Encoder-350M (this model) — a larger sibling for maximum downstream quality.

Both are masked language models with full bidirectional attention, designed to be fine-tuned into task-specific models (classification, token classification, retrieval, reranking, and semantic similarity) across 15 languages, and to run efficiently on-device.

Find more details about our encoders in our blog post.

Key highlights:

• Top quality for its size. Ahead of every model its size or smaller, and ~5 points above our own retrieval siblings.
• General-purpose. 8k context, strong across NLI, paraphrase, sentiment, and multilingual tasks.
• Fast and on-device. Matches or beats ModernBERT throughput, with a long-context edge on CPU.

Note. 💻 Demos: We built the demos below from fine-tuned LFM2.5-Encoders. Each one runs in a CPU-only Hugging Face space:

• Zero-shot prompt routing — define your own routing lanes as free text. The model scores the whole prompt against every lane in one pass.
• Zero-shot policy linting — check text against your company's rules, written as free text. It scores every token against every rule in one pass.
• Spell checking — correct misspellings token by token.
• PII detection — spot and remove 40 kinds of personal information across 16 languages.
• Masked-diffusion text generation — bonus: run the encoder as a chatbot that generates text by iteratively unmasking instead of left to right.

📄 Model details#

Supported languages: English, German, Spanish, French, Italian, Dutch, Polish, Portuguese, Arabic, Hindi, Japanese, Russian, Turkish, Vietnamese, Chinese (15).

Architecture. LFM2.5-Encoder is built on the LFM2 hybrid backbone, which interleaves gated short-convolution blocks with grouped-query attention. For encoder use, the causal mask is replaced with full bidirectional (non-causal) attention and the model is trained with a masked language modeling head. The encoder body is exposed as Lfm2BidirectionalModel; masked-LM loading uses Lfm2BidirectionalForMaskedLM. Both are wired through auto_map and require trust_remote_code=True.

Lfm2BidirectionalForMaskedLM(
  (lfm2): Lfm2BidirectionalModel
  (lm_head): Linear(in_features=1024, out_features=65536, bias=False)
)

Training. LFM2.5-Encoder-350M is adapted from the LFM2 base and trained with a masked language modeling objective on a large multilingual corpus. Pre-training uses a two-stage schedule that extends the context window to up to 8,192 tokens.

We recommend fine-tuning LFM2.5-Encoder-350M for a range of downstream tasks, such as:

• Text classification: sentiment, topic, intent/routing, moderation, and business-text linting.
• Token classification: named-entity recognition, span extraction, and sequence labeling.
• Retrieval and reranking: a backbone for dense embedding or late-interaction (ColBERT-style) retrievers.
• Semantic similarity: STS, paraphrase, and duplicate detection.
• Natural language inference and extractive QA: sentence-pair reasoning and answer-span extraction.

🏃 How to run#

Install the latest version of transformers:

pip install -U transformers

Run masked-token prediction:

from transformers import AutoModelForMaskedLM, AutoTokenizer
import torch

tok = AutoTokenizer.from_pretrained("LiquidAI/LFM2.5-Encoder-350M", trust_remote_code=True)
mlm = AutoModelForMaskedLM.from_pretrained("LiquidAI/LFM2.5-Encoder-350M", trust_remote_code=True)

text = f"The capital of France is {tok.mask_token}."
enc = tok(text, return_tensors="pt")
with torch.no_grad():
    logits = mlm(**enc).logits
pos = (enc["input_ids"][0] == tok.mask_token_id).nonzero()[0].item()
print([tok.decode([t]).strip() for t in logits[0, pos].topk(5).indices.tolist()])
# -> ['Paris', 'Strasbourg', 'Paris', 'Lyon', 'Versailles']

For downstream tasks, load the encoder body and attach your own head (classification, token classification, regression, retrieval):

from transformers import AutoModel
body = AutoModel.from_pretrained("LiquidAI/LFM2.5-Encoder-350M", trust_remote_code=True)

If your GPU supports it, we recommend using LFM2.5-Encoder-350M with Flash Attention 2 to reach the highest efficiency. To do so, install Flash Attention as follows, then use the model as normal:

pip install flash-attn

📊 Performance#

For each benchmark task, we run a full supervised fine-tune and report that fine-tuned model's score. The results below span 14 models across 17 tasks from GLUE, SuperGLUE, and multilingual classification tasks. The full evaluation harness is open-sourced in the eurobert-repro repository.

Image: benchmark_ranking

17-task results (avg@5 fresh seeds ± std)#

Click to expand per-task results — all 17 tasks (avg@5 fresh seeds ± std)

Per-task results — all 17 tasks (avg@5 fresh seeds ± std)#

* = dev split (GLUE/SuperGLUE test labels hidden). The 5 multilingual columns are labeled test. SeaHorse & STS-B are Spearman×100. All other tasks are accuracy.

Inference speed#

The LFM2 backbone was built for fast inference, and the encoders inherit it. While ModernBERT-base is faster at short sequences in Apple GPU inputs, LFM2.5-Encoders overtake it as inputs grow. At long input sequences of 8k on CPU, the encoders run 3.3× faster than ModernBERT-base.

Image: inference_perf_cpu

Image: inference_perf_gpu

🔧 Fine-tuning#

LFM2.5-Encoder-350M follows standard BERT-style fine-tuning. Attach a task head to the encoder body and train end-to-end. Suggested starting points (tune per task):

📬 Contact#

• Got questions or want to connect? Join our Discord community
• If you are interested in custom solutions with edge deployment, please contact our sales team.

Citation#

@article{liquidAI2026Encoders,
  author = {Liquid AI},
  title = {LFM2.5-Encoders: Fast at Long Context, Even on CPU},
  journal = {Liquid AI Blog},
  year = {2026},
  note = {www.liquid.ai/blog/lfm2-5-encoders},
}