toolsWTF 7.2via r/LocalLLaMA
2 gaming PCs + an M4 Pro MacBook over Ethernet + 2 iPhone 17 Pro Maxes over USB-C: Qwen3.8 Flash Next at 37-39 tok/s at 60k context (262k layout in the works)
"Frankenstein’s LLM: Stitching five random devices together to run a model that shouldn't fit."
Explain Like I'm Normal
A developer successfully ran the Qwen 3.8 Flash Next model by chaining two gaming PCs, an M4 MacBook, and two iPhones into a single distributed inference cluster. By splitting model layers across disparate hardware via Ethernet and USB-C, they achieved respectable token speeds on a setup that usually requires enterprise-grade VRAM. This demonstrates the increasing viability of 'scavenged' compute for running massive Mixture-of-Experts models at home.
#distributed-inference#localllama#edge-computing#qwen#hardware-gore
GET THE DAILY CHAOS
The only newsletter for people who read AI news at 3am and feel things. One email a day.