Transformers v5.16.0 lands Qwen4-Exp hybrid attention, ESMC + ESMFold2, GLM 5.3 Flash support in v5.16.1

Hugging Face

Tools official 2 src. ~1 min

v5.16.0 (Aug 26) adds Qwen4-Exp's hybrid sparse+linear attention, GraniteSpeech5 (~470M conformer encoder for ASR), Step3p7 (198B sparse MoE VLM), CohereCompass base for Cohere's small VLMs, ESMC and ESMFold2 protein language/folding models from BioHub. v5.16.1 adds GLM 5.3 Flash support (320B total / 18B active, first natively multimodal GLM-5 model). Breaking change: legacy tensor-parallel implementation replaced with a DTensor-native backend.

Why it matters

Qwen4-Exp's hybrid architecture becomes first-class in the reference impl; the DTensor-only TP backend forces downstream code to upgrade, but unlocks cleaner parallelism stacks.

Importance: 2/5

default

huggingface transformers qwen4 esm release glm

Sources

official Transformers v5.16.0 release

official Transformers v5.16.1 release