SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT • SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT •
toolsWTF 4.8via r/LocalLLaMA

I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest

"Your 1-bit LLM is hallucinating because of a one-line SiLU bug, stay winning CPU bros."

Explain Like I'm Normal

Independent testing revealed that the official bitnet.cpp implementation for 1-bit models yields incorrect outputs due to using the wrong activation functions. While the text might look okay, the perplexity is six times worse than the reference model unless specific community fixes are applied. For those running local models on CPU, ik_llama.cpp with specific flags currently offers the best balance of speed and accuracy.

Read original ↗
#quantization#bitnet#cpu-inference#llama-cpp

GET THE DAILY CHAOS

The only newsletter for people who read AI news at 3am and feel things. One email a day.