40× RTX 5070 Ti: the rig behind AgentVUE

After the 1 TB monster, back to reason - but not back to small. The current rig drops the aging datacenter cards for recent consumer GPUs: 40 RTX 5070 Ti, 640 GB of VRAM, shelved like all the previous ones.

And the difference is striking. On a single 20-GPU vLLM command, one request reaches 5,200 tokens/s prompt processing and 142 tokens/s generation. Where this rig really changes the game is agentic usage: with 8 agents working in parallel, the command sustains 5,500+ tokens/s prompt processing and 800 tokens/s total generation, with 315,000 tokens of context. And since the 40 GPUs share two commands of this kind, throughput simply doubles, reaching 10,000+ tokens/s prompt processing and 1,600 tokens/s generation.

That kind of throughput is what makes what I am building with AgentVUE possible: several agents reading, writing and checking at the same time, on hardware I own, with no data leaving the house. The photo above is from October 2026: since then, the rig has received a few more improvements, which I will cover in an upcoming article.

Why 5070 Tis?

In two words: efficiency. The MI50s were a brilliant bet at €150 a card, but their idle draw and their age were starting to show. Recent consumer GPUs offer a far better balance of performance, power and noise - and above all, a software stack that is maintained every day. The detailed build guide is coming; meanwhile, all the key numbers are in the box above.

The interconnect: InfiniBand, not Ethernet

One detail changes everything: the 4 nodes do not talk to each other over Ethernet. Each one is wired to the Mellanox switch with its own 56 Gb/s InfiniBand cable, through ConnectX-3 Pro cards. The result: P2P latency between cards on different nodes collapses. In practice, vLLM tensor parallelism spread across several nodes loses only ~5% inference speed, versus more than 50% over an equivalent Ethernet link.

The two 20-GPU vLLM commands are split differently across these nodes: one spans 3 nodes, the other 4. And between the two commands, the performance loss is almost nil - the best proof of how effective InfiniBand is.

And what's next?

The rack can take 44 GPUs: today I use 40, and the next 4 are planned for a future build. The loop is closed: from 8 second-hand GPUs at $880 to here, every step answered the same question - how far can you go with affordable hardware? Further and further, apparently.

"What does 40 GPUs draw?"

Exact measurements will come with the full guide. What I can already say: with recent cards, the performance-per-watt ratio has nothing to do with the previous generation - and the machine is switched off when unused, as always.

"Do you sell the rigs?"

No - but I share everything: guides, forks and benchmarks are free on the AI-INFOS links below. And what I build with all of this, AgentVUE, is available to everyone.