toolsWTF 4.8via r/LocalLLaMA
I checked BitNet and Bonsai on CPU against references: bitnet.cpp main still gives wrong output, Bonsai 1.7B is correct wherever it loads, and ik_llama.cpp with -rtr is usually fastest
"Your 1-bit LLM is hallucinating because of a one-line SiLU bug, stay winning CPU bros."
Explain Like I'm Normal
Independent testing revealed that the official bitnet.cpp implementation for 1-bit models yields incorrect outputs due to using the wrong activation functions. While the text might look okay, the perplexity is six times worse than the reference model unless specific community fixes are applied. For those running local models on CPU, ik_llama.cpp with specific flags currently offers the best balance of speed and accuracy.
#quantization#bitnet#cpu-inference#llama-cpp
GET THE DAILY CHAOS
The only newsletter for people who read AI news at 3am and feel things. One email a day.