llama.cpp Sep 1-2 wave: +4.9% generation from n-gram lookup fix, fused CUDA MoE reduction

ggml

Tools official 1 src. ~1 min

Across b10739-b10753 (Sep 1-2), llama.cpp landed a KV-cache n-gram lookup fix that uses the sequence position index - text generation on one Qwen3 model rose from 69.3 to 72.7 tok/s (+4.9%) at 71k context (b10750) - plus a fused CUDA MoE weighted expert-reduction kernel for k=2..15 experts that cuts global-memory traffic (b10751). Metal work dominated the rest: fa-vec tuning for M2 Pro, M2 Max, and A18 Pro, autoreleasepool leak fixes, and xcframework metallib build support (b10752).

Why it matters

Free generation-speed win for long-context local inference

Importance: 2/5

Measurable performance wins for local inference

Sources