SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT • SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT •
breakthroughsWTF 6.4via r/LocalLLaMA

I ran Ternary-Bonsai-27B (2-bit) and Bonsai-27B (1-bit) on Terminal-Bench 2.0, in 8GB VRAM

"27B parameters on a laptop GPU? Your VRAM budget just got a stimulus check."

Explain Like I'm Normal

Independent testing confirms that extremely low-bit quantization (ternary/2-bit) allows hefty 27B parameter models to run on consumer hardware with 8GB VRAM. While performance currently lags behind larger 8-bit models like Qwen, the successful execution and clean tool-calling prove that 'weight-shedding' is a viable path for local AGI.

Read original ↗
#quantization#1-bit#edge-ai#localllama

GET THE DAILY CHAOS

The only newsletter for people who read AI news at 3am and feel things. One email a day.