clef

Clef#

• Announcement: Clef decision models on the Cloudflare blog
• Decision Index leaderboard: clef-evals.workers-ai-mle.workers.dev

Clef is a 27B multimodal model that turns a state and a schema of typed questions into decisions. It reads the state as text, JSON, images, or video, and returns a probability for every allowed option of every question in a single forward pass. There is no free-form text generation and no output parsing.

The Clef API is fully compatible with Jev and SystemOne.

Clef is post-trained from Qwen/Qwen3.8-27B. See Clef-Flash for the smaller, faster variant.

Model#

• Backbone: Qwen/Qwen3.8-27B with its vision encoder, stored as standard sharded safetensors.
• Joint schema head: a small transformer head that reads the backbone's final hidden states, routes evidence from the state to each question, and scores all options of all questions jointly.
• Output: one logit per allowed option for each question. Apply a softmax per question to get probabilities.

Files#

Usage#

Tested with torch 2.11 and transformers 5.10.2 on a single H200. Image and video inputs also need pillow.

import sys

import torch
from huggingface_hub import snapshot_download

path = snapshot_download("Cloudflare/clef")
sys.path.insert(0, path)
from joint_schema_model import collate_records, encode_record, load_release_model

model, processor = load_release_model(path, device="cuda")

record = {
    "state": {"invoice": {"vendor": "Acme", "total": 1250.0, "currency": "USD", "status": "overdue"}},
    "questions": {
        "status": {
            "type": "choice",
            "instructions": "What is the invoice status?",
            "criteria": {"paid": "Invoice is paid.", "overdue": "Invoice is past due.", "draft": "Not sent."},
        },
        "large": {"type": "noul", "instructions": "Is the total above 1000 USD?"},
    },
}

encoded = encode_record(processor.tokenizer, record, processor=processor)
batch = collate_records([encoded], processor.tokenizer.pad_token_id, torch.device("cuda"))
with torch.inference_mode():
    logits = model(batch)[0]

for question, question_logits in zip(encoded.questions, logits):
    probabilities = question_logits.float().softmax(-1).tolist()
    print(question.question_id, dict(zip(question.option_ids, probabilities)))

SGLang#

Clef can be served with SGLang. See the SGLang cookbook for launch commands on H200, B200, and B300. On a single H200:

docker run --gpus all \
  --shm-size 32g \
  -p 30000:30000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  --ipc=host \
  lmsysorg/sglang:dev-clef \
  sglang serve \
    --model-path Cloudflare/clef \
    --host 0.0.0.0 \
    --port 30000

The server answers Jev/SystemOne requests at /v1/systemone:

curl http://localhost:30000/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{"model":"Cloudflare/clef","state":"My Stripe integration keeps failing. Please help ASAP.","questions":{"urgency":{"type":"noul","instructions":"Does this message express urgency?"}}}'

Jev / SystemOne API#

systemone takes a Jev/SystemOne POST /v1/systemone request body and returns the same response body: model, answers keyed by question ID, and usage. A choice answer has choice, confidence, and probabilities; a score answer has the expected score, confidence, legend, and probabilities; a noul answer has the probability of true. instructions is optional, and images and videos may be added to the request.

from joint_schema_model import systemone

response = systemone(model, processor, {
    "model": "clef",
    "state": "Our checkout started returning errors and orders are blocked.",
    "questions": {
        "department": {
            "type": "choice",
            "instructions": "Which team should handle the message?",
            "criteria": {"billing": "Payments or invoices", "technical": "Bugs or outages"},
        },
        "urgency": {"type": "score", "criteria": ["Can wait", "This week", "Today"]},
        "outage": {"type": "noul", "instructions": "Is a service down?"},
    },
})
print(response["answers"])

Images and video#

Add images (PIL images) or videos (frame arrays) to the record and pass the processor to encode_record. Optional processor arguments go in media_kwargs.

from PIL import Image

record = {
    "state": {"task": "Review the attached receipt."},
    "images": [Image.open("receipt.jpg")],
    "questions": {
        "legible": {"type": "noul", "instructions": "Is the receipt total legible?"},
    },
}
encoded = encode_record(processor.tokenizer, record, processor=processor)

Text-only and multimodal records can be mixed in the same batch.

Input format#

Each question has:

• type: noul (true/false), choice (named options), or score (ordered options)
• instructions: what to decide; optional, and the question ID is used when it is omitted
• criteria: for choice, a mapping of option ID to description; for score, a list of option descriptions indexed from 0; for noul, optional descriptions for true and false

encode_record accepts max_length (default 16,384 tokens) and max_state_tokens to bound the input.

Results#

Decision Index#

Per-benchmark results from our internal run of the Decision Index 0.2.1 suite. Scores are percentages; ForecastBench is a Brier score, where lower is better. The last two rows are request latency in milliseconds, where lower is better. The best value in each row is in bold.

Workflow evals#

Decision accuracy on four end-to-end business workflows from Typesafe Evals, scored against consensus reference labels. All models are scored on the same dataset revision and case cohort.

License#

Released under the Apache-2.0 license, following the base model Qwen/Qwen3.8-27B.