SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT • SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT •
toolsWTF 6.9via r/LocalLLaMA

DeepSeek V4.1 Flash at 2.61 bpw + Engram: 26 t/s on dual RTX 3090s with dsx

"Local AGI go brrr: Running a massive MoE on consumer GPUs at usable speeds."

Explain Like I'm Normal

A community member successfully optimized DeepSeek V4.1 Flash to run on two consumer RTX 3090s using the DSX engine and custom 2.6-bit quantization. By utilizing 'Engram' tables, they achieved generation speeds of 26 tokens per second even with a massive 262k context window. This proves that high-end agentic models are becoming increasingly accessible to developers without needing enterprise-grade H100 clusters.

Read original ↗
#quantization#deepseek#localllama#hardware

GET THE DAILY CHAOS

The only newsletter for people who read AI news at 3am and feel things. One email a day.