toolsWTF 4.9via r/LocalLLaMA
We open-sourced Paddock, our Rust/C++ inference engine with its own CUDA kernels (MIT/Apache-2.0)
"Rust developers try not to rewrite every CUDA kernel in existence challenge (IMPOSSIBLE)"
Explain Like I'm Normal
Truesparco has open-sourced Paddock, a high-performance Rust and C++ inference engine that replaces standard kernels with custom implementations. It benchmarks slightly faster than vLLM and SGLang on Qwen models while offering a single binary that handles GGUF and Safetensors. The release provides a production-tested alternative for builders tired of python-heavy inference stacks.
#rust#inference#cuda#open-source
GET THE DAILY CHAOS
The only newsletter for people who read AI news at 3am and feel things. One email a day.