SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT • SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT •
toolsWTF 4.9via r/LocalLLaMA

We open-sourced Paddock, our Rust/C++ inference engine with its own CUDA kernels (MIT/Apache-2.0)

"Rust developers try not to rewrite every CUDA kernel in existence challenge (IMPOSSIBLE)"

Explain Like I'm Normal

Truesparco has open-sourced Paddock, a high-performance Rust and C++ inference engine that replaces standard kernels with custom implementations. It benchmarks slightly faster than vLLM and SGLang on Qwen models while offering a single binary that handles GGUF and Safetensors. The release provides a production-tested alternative for builders tired of python-heavy inference stacks.

Read original ↗
#rust#inference#cuda#open-source

GET THE DAILY CHAOS

The only newsletter for people who read AI news at 3am and feel things. One email a day.