laya-multilingual-ONNX

laya-multilingual (ONNX)#

ONNX conversion of Convai Innovations' convaiinnovations/laya-multilingual, a non-autoregressive decision model (jhu-clsp/mmBERT-base encoder with a decision head). Give it a state (text or JSON) and typed questions (choice, score, noul); it returns a calibrated distribution per question in one forward pass. For the model itself, its training and its limits, see the original model card and the laya repository.

Files#

One graph holds encoder and head (laya's own laya-ts export splits them in two):

• inputs: input_ids [b, s] int64, attention_mask [b, s] int64, marker_pos [b, k] int64, marker_mask [b, k] bool, qtype [b] int64 (choice 0, score 1, noul 2)
• outputs: logits [b, k] (one per [MASK] option marker, masked slots -1e4), act_logits [b, 2]

Each question is its own sequence ([CLS] {type} question: {question} [SEP] [MASK] opt … [SEP] state [SEP]). config.json carries the encoder config plus a laya section with everything a runtime needs: max_len (1024), head_max_len (256), the calibrated temperatures per question type and option count, and whether the tokenizer needs word-by-word encoding in Transformers.js (split_words).

Use it in the browser#

With open-jev (Transformers.js, WebGPU):

import { OpenJev, choice, score, noul } from "open-jev";

const jev = await OpenJev.load({ model: "laya-multilingual" });
const answers = await jev.decide("Hi, we were billed twice for March. Please refund the duplicate.", {
  department: choice("Which department should handle this?", ["billing", "technical", "other"], {
    billing: "invoices, payments, refunds",
    technical: "bugs, outages, system errors",
  }),
  urgency: score("How urgent is this?", ["not urgent", "soon", "blocking"]),
  refund: noul("Does the user ask for a refund?"),
});

The open-jev laya family rebuilds laya's build_sequence (option texts, 48-token option cap, head budget) and applies the checkpoint's calibrated temperatures, as laya.Agent does.

Conversion and parity#

torch.onnx.export (dynamo, opset 18) of DecisionModel.forward from laya.common, with dynamic batch, sequence and marker axes. Scripts are in conversion/.

• ONNX Runtime (CPU) against PyTorch on real typed-decisions sequences, up to 2,048 tokens (past the 128-token sliding window): max |Δlogit| 3.1e-5.
• open-jev in Node (onnxruntime-node) against laya's own Python Agent.predict on 15 cases (typed-decisions states; double spaces; the mask token inside text; over-long options; the head budget; Hindi, Chinese, Japanese and Spanish): token ids identical for every question; fp32 max |Δp| 1e-4; fp16 max |Δp| 0.035, no top answer changed.

Accuracy in the browser#

typed-decisions test split (400 states, 2,000 questions), WebGPU on an M3 Pro, via open-jev:

fp16 is the default on WebGPU: it matches fp32 within 0.2 points and is faster. The base checkpoints are zero-shot here; Laya's card reports 0.362 for the English one.

License#

Apache-2.0, as the original. Converted by @shreyask; all credit for the model to Convai Innovations.