SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT • SAM ALTMAN SAYS AGENTS ARE COMING • CHATGPT GAINED SENTIENCE FOR 4 SECONDS • GOOGLE RELEASES 40th LLM THIS WEEK • NVIDIA MARKET CAP EXCEEDS REALITY • ANTHROPIC ENGINEER DISCOVERS NEW FORM OF GRIEF • MISTRAL RAISES AT VALUATION OF GROSS DOMESTIC PRODUCT •
toolsWTF 6.0via r/LocalLLaMA

Qwen 3.8 Flash Next optimized for two CMP 170hx GPUs

"Extracting insane throughput from crypto-mining relics through sheer engineering spite."

Explain Like I'm Normal

A developer successfully optimized Qwen 3.8 Flash on dual NVIDIA CMP 170HX mining cards, achieving over 800 tokens per second through aggressive pipeline parallelism and speculative decoding. By utilizing memory-mapped n-gram tables on NVMe, the setup bypasses PCIe bandwidth bottlenecks typical of older hardware. This demonstrates that highly efficient local inference is possible on non-standard, affordable enterprise hardware.

Read original ↗
#quantization#vllm#hardware#local-llama

GET THE DAILY CHAOS

The only newsletter for people who read AI news at 3am and feel things. One email a day.