Xolotl (Strata based) - DeepSeek-V4.1-Flash (552B) on a single 4090 at ~27 tok/s, Uncensored 1017 quant coming in a couple of days

Explain Like I'm Normal

A new inference engine called Xolotl uses the Strata architecture to run massive MoE models on consumer hardware by aggressively offloading experts to RAM and NVMe. It claims to achieve 27 tokens per second on a DeepSeek-V4.1-Flash model using only 24GB of VRAM and system memory. This effectively lowers the hardware floor for running SOTA-scale models by orders of magnitude.