My first local AI rig: 8 GPUs, 256 GB of VRAM, $880
- 8× AMD MI50 32GB - 256 GB of VRAM
- GPU cost: $880 (early 2025)
- MiniMax-M2.1: 26.8 tokens/s generation
- GLM 4.7: 15.6 tokens/s · GLM 4.6: 12 tokens/s
- 280 W idle / 1,200 W inference
- Context up to 196,000 tokens (MiniMax)
A bit of backstory: the AMD rig was not really my first AI rig. It all started with the NVIDIA GTX 1060, then 2060, cards in my laptops, running Gemma models (from Google) with ollama, then llama.cpp. Then I stepped things up and built my first AI server with 6 NVIDIA RTX 3090 (144 GB of VRAM) running vLLM.
Then came the question: do I stay on the NVIDIA path with 8 RTX 3090 and more, knowing that these cards keep getting more expensive and harder to find (on every market, new and second-hand)? Or do cheaper alternatives exist, little known to the general public and with almost equivalent capability (good memory bandwidth, powerful cores and plenty of VRAM: the winning trio for high-performance AI)?
In short: can you run serious large AI models at home, at a reasonable speed and without breaking the bank? Not a toy, not a demo - something you can work with every day with an intelligent model, fully local.
My answer was a bit of a gamble: AMD MI50s. These are 2019 datacenter cards at the end of their commercial life, resold by datacenters for next to nothing. Each one carries 32 GB of VRAM at 1,024 GB/s HBM2 and 26 TFLOPS FP16. In early 2025 they went for around $110 each. Eight cards, and there it is: 256 GB of VRAM for $880 - far below the price tag of a single high-end card at the time.
The real challenge was never the hardware: it was the software. AMD no longer officially supports these cards. But the open source community has worked miracles, and I contributed my own adaptations of vLLM for these GPUs. Big thanks again to the global open source community for all of it.
What it feels like day to day
With MiniMax-M2.1, I get 26.8 tokens/s generation with the maximum 196,000-token context. With GLM 4.7, 15.6 tokens/s. All of this runs under 1,200 W at full inference, 280 W idle. For a model of this size running entirely at home, it was one of the most cost-effective setups of its time - that was even my stated goal: reach the most cost-effective solution for fast, intelligent local inference.
I mostly used it for code: Roo Code (the GitHub repo is archived now) connected to the rig, and off we went. Full disclosure: some of the numbers I first published were too good - the prompt processing speeds shown in vLLM logs did not reflect reality. The real measured values are below; I would rather correct myself than leave misleading numbers out there.
What the community asked me
The Reddit post got over 300 upvotes and more than 120 comments. Here are the questions that keep coming back.
- "What motherboard fits 8 GPUs?"
- An ASRock Rack ROMED8-2T (7 PCIe 4.0 x16 slots). But I also ran GLM 4.6 on a mining motherboard for under $200 with x1 risers: stable up to ~60k tokens of context, at a slightly lower speed (~7 tokens/s). Everything is in the guides.
- "Is it loud?"
- The cards come with their stock 50 mm fans. It stays under 70 °C, but it is loud: a separate closed room is the way to go, in my opinion.
- "Where do you buy these cards and the related hardware, and what's the catch?"
- Alibaba, Amazon, eBay, AliExpress and the like. The catch: MI50 prices have climbed a lot since 2025, and AMD dropped official support. In practice the community runs ROCm 7.2+ on them and it works rather well.