Model Summary: Granite Switch 4.1 8B Preview is a modular LLM built on IBM Granite 4.1 8B with embedded adapters from the Granite Libraries collection. A single checkpoint supports multiple specialized capabilities — RAG, safety, explainability, and more — that are activated on demand via control tokens in the chat template.
For full details on model composition and adapter configuration, see BUILD.md.
Base Model:ibm-granite/granite-4.1-8b (8B params, 128K context)
Adapters: 12 adapters from granitelib-rag-r1.0, granitelib-core-r1.0, and granitelib-guardian-r1.0
Motivation: Traditional multi-task LLM deployments require either separate model copies per capability (multiplying memory and compute) or weight merging that permanently blends adapters and destroys task specialization. Granite Switch takes a different approach: independently trained activated LoRA adapters are embedded in a single checkpoint and dynamically selected at inference time via control tokens. KV cache normalization ensures adapters share no internal KV cache state as each adapter sees prior tokens only through the base model's representation. That way, adapters can build on each other's outputs, but never through another adapter's cached activations. This allows adapters to be developed independently and composed without accuracy loss. This makes it possible to implement LLM capabilities very efficiently and very accurately.
Mellea is the preferred way to run Granite Switch adapters in applications. It standardizes the interface for building with adapters like answerability checking, hallucination detection, requirement checker and harmful language detection easily and reliably. Constrained decoding and input/output pre-processing are handled automatically, improving accuracy and reliability. When running Granite Switch models through Mellea, embedded adapters function as high-level API calls. This allows you to use direct operations instead of raw prompt engineering.
from mellea.backends.openai import OpenAIBackend
from mellea.formatters import TemplateFormatter
from mellea.stdlib.components import Document, Message
from mellea.stdlib.components.intrinsic import rag
from mellea.stdlib.context import ChatContext
SWITCH_MODEL_ID = "ibm-granite/granite-switch-4.1-8b-preview"
backend = OpenAIBackend(
model_id=SWITCH_MODEL_ID,
formatter=TemplateFormatter(model_id=SWITCH_MODEL_ID),
base_url="http://localhost:8000/v1",
api_key="EMPTY",
load_embedded_adapters=True,
)
context = ChatContext().add(Message("assistant", "Hello there, how can I help you?"))
question = "What is the square root of 4?"
documents = [Document("The square root of 4 is 2.")]
result = rag.check_answerability(question, documents, context, backend)
print(f"Answerability: {result}")
from mellea.stdlib.components.intrinsic import core
context = ChatContext().add(
Message("user", "Invite for an IBM office party.")
).add(
Message("assistant", "Dear Team, you are cordially invited to a team social...")
)
result = core.requirement_check(context, backend, requirement="Use a professional tone.")
print(f"Requirements Satisfied: {result}") # float between 0.0 and 1.0
from mellea.stdlib.components.intrinsic import guardian
context = ChatContext().add(
Message("user", "How can I hack my friend's email?")
)
score = guardian.guardian_check(context, backend, criteria="harm", target_role="user")
verdict = "Risk detected" if score >= 0.5 else "Safe"
print(f"Score: {score:.4f} ({verdict})")
See the mellea examples/ directory for more examples, including manual adapter loading.
The following examples demonstrate low-level adapter invocation via the HuggingFace and vLLM backends directly. Check Granite Switch For additional tutorials.
import granite_switch.hf # Register the model architecture
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("ibm-granite/granite-switch-4.1-8b-preview", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("ibm-granite/granite-switch-4.1-8b-preview")
Granite Switch uses coarse-grained expert switching — one adapter is active across all layers for a contiguous span of tokens. A lightweight switch layer (standard attention) detects control tokens in the input and produces per-position adapter indices that tell every decoder layer which LoRA weights to apply.
Each adapter is activated by passing its name to the chat template using an argument. The template inserts the appropriate control token automatically — callers just pass adapter_name.
This model inherits the safety profile of the base Granite 4.1 model. The Guardian Library adapters (guardian core, factuality detection/correction, policy guardrails) provide additional safety layers but are not a substitute for application-level safety testing. Deployers should:
Test adapter behavior on their specific use cases before production deployment
Apply appropriate content filtering for their domain
Monitor adapter outputs, especially for safety-critical applications
Use the uncertainty adapter to assess model confidence on important decisions
import os
os.environ["HF_ENDPOINT"] = "https://freeweights.allternit.com"
from transformers import AutoModelForCausalLM, AutoTokenizer
tok = AutoTokenizer.from_pretrained("ibm-granite/granite-switch-4.1-8b-preview", revision="8a22410af7cbbfa2cbbd39a3af281ba98c2df846")
model = AutoModelForCausalLM.from_pretrained("ibm-granite/granite-switch-4.1-8b-preview", revision="8a22410af7cbbfa2cbbd39a3af281ba98c2df846", device_map="auto")
