DeepSeek V4 Flash on a Single AMD MI300X

The repo demonstrates a practical deployment recipe for DeepSeek V4 Flash 0731 on one AMD MI300X. That matters because this model is large enough that full-precision serving usually pushes teams toward multi-GPU Nvidia setups or hosted APIs. Here, the claim is that one rented MI300X can hold the model at original weights and produce usable throughput, while trading the full 1 million token context window down to 256k. People reading it as a cheap way to beat DeepSeek’s API mostly came away unconvinced. At roughly $2 per hour for MI300X cloud access, the raw token economics look tight or outright worse than just buying DeepSeek tokens, especially for a single user and especially once you account for actual sustained generation rates instead of peak aggregate numbers. The stronger case was operational, not financial. Several comments sharpened the point that self-hosting buys privacy, control over prompts and decoding, guaranteed access to unquantized weights and KV cache behavior, and more predictable latency than low-cost shared inference endpoints that may batch aggressively or quietly quantize for margin. That made the project interesting as a small-team dedicated server rather than a general-purpose money printer. A second thread corrected the hardware intuition. An MI300X is not a consumer card you can slot into a workstation. It is typically an OAM server module, sold in expensive systems, though cloud rentals make one-off access possible. People pointed to MI350P as the more realistic future target because it is a PCIe card, but also noted that it is still a server part with serious cooling and power demands. Performance comparisons also stayed grounded. Nobody thought this matched DeepSeek’s own serving stack on H800-class systems, and several comments said that gap is expected because large distributed deployments get better scaling and much deeper software optimization. So the real signal is narrower and more useful: a single MI300X can now run a frontier open model at intended weights and respectable speed, but you do it for privacy, control, and dedicated capacity, not because it beats the API on pure cost.

If you are evaluating self-hosted frontier models, stop treating cost per token as the only metric. Privacy, stable throughput, control over decoding, and avoiding provider-side quantization or data retention can justify a setup that looks worse on paper than the API price.