MTP on MoE matters

Explain Like I'm Normal

A community developer discovered that Multi-Token Prediction (MTP) significantly boosts speed on Mixture of Experts (MoE) models despite previous skepticism. By fine-tuning n-max parameters in llama.cpp, they achieved up to a 50% increase in tokens per second for local generation. This suggests that hardware-constrained users can squeeze much more performance out of medium-sized models like Gemma and Qwen.