SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT • SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT •
toolsWTF 6.4via r/LocalLLaMA

Basalt: NVFP4 + faster prefill for Qwen3.8 Flash-Next on Blackwell (single 5090)

"Local LLM bros are already redlining the 5090 with NVFP4 native support."

Explain Like I'm Normal

The Basalt inference engine has achieved native NVFP4 support on NVIDIA's new Blackwell architecture, drastically improving speed for the Qwen 3.8 Flash-Next model. By utilizing a single RTX 5090, the developer hit over 300 tokens per second for structured tasks and a prefill speed of 8,500 tokens per second. This represents a significant optimization for running large models on consumer-grade hardware with minimal quality loss.

Read original ↗
#llm#quantization#blackwell#inference#locally-hosted

GET THE DAILY CHAOS

The only newsletter for people who read AI news at 3am and feel things. One email a day.