A 10 year old Xeon is all you need

What people landed on is that this is a memory-bandwidth story, not a raw-core-count story. The reason old Xeons can be interesting is not that they are secretly fast, it is that server platforms give you large RAM capacity and multiple memory channels very cheaply. That makes them viable for running larger open-weight models locally when the alternative is paying for expensive GPUs or cloud tokens. Several people reported similar results on old Mac Pros, first-gen Ryzen boxes, workstation Xeons, and mixed CPU plus cheap GPU setups. The recurring pattern was the same. Slow but acceptable decode speeds can be enough for batch jobs, automation, overnight document extraction, privacy-sensitive internal use, or “read along” chat. They are not enough for everyone’s coding assistant or long-context interactive work. A lot of the highest-signal discussion was really about what the benchmark omitted. Prompt processing and time to first token matter more than decode speed for many real workloads, especially coding, long documents, and anything with image inputs. People pointed out that the posted prompt was only seven tokens long, so the prompt-eval number was not representative. Others noted that speculative decoding can help decode throughput without changing model output, but it does not erase the fundamental bandwidth ceiling, and acceptance rates need to be interpreted correctly. Several readers also wanted proper llama-bench or sweep-bench results instead of vibe-based “reading speed.” The biggest factual issue in the comments was hardware consistency. The post says Xeon E5-2620 v4 and DDR3. Many readers flagged that this CPU is documented by Intel as DDR4-only, while a handful of unusual boards and OEM-only SKUs can pair some v3 or v4 Xeons with DDR3. The consensus was that the specific CPU listed does not support DDR3, so either the RAM type, CPU model, or motherboard story is wrong. That did not kill the main point that old server-class hardware can run current open models, but it did make people distrust the exact hardware recipe. The broader strategic angle drew almost as much attention as the benchmark itself. Many readers see local inference getting “good enough” for a meaningful share of coding, office, and private-data workflows, which pressures the economics of API-first AI companies. Others pushed back that cloud models are still far better for hard tasks, and that old Xeon boxes often lose badly on power, heat, noise, and total cost versus paying for hosted inference. The practical conclusion was narrower than the hype. Reused servers are compelling when you already have them, when privacy matters, or when you can tolerate latency. They are a bad default if you are buying hardware from scratch expecting a cheap frontier-model replacement.