For a team building a private AI service, useful capacity is a business question. How much work can the deployment process, and how long will people wait for a response when traffic rises?
In our October DeepSeek V4.1 Flash study, an updated serving configuration delivered 60.1% more output throughput per GPU than the September 22 InferenceX MI325X reference, at the same two-GPU allocation and 32 concurrent benchmark sessions. Against our own baseline on the same machine, the improvement was 60.8%.
The hardware allocation stayed the same. The opportunity was in how the serving software used it.
The result: more output from the same GPU allocation
| Configuration | Output throughput, tokens/s/GPU |
|---|---|
| Published InferenceX MI325X reference | 227.8 |
| ProsGrow local baseline | 226.7 |
| ProsGrow optimized configuration | 364.6 |
DeepSeek V4.1 Flash; two GPUs assigned to one model instance; 32 concurrent trace sessions. The public reference is dated September 22, 2026; ProsGrow's local measurements are from October 20.
The local comparison also showed a substantial change in response starts. Average time to first token fell from 4.70 to 1.09 seconds, a 76.8% reduction. The optimized run completed 2,785 profiling requests, compared with 1,896 in the baseline run, during the one-hour benchmark.
Those measures describe different benefits: more output capacity, less waiting before generation begins, and more requests processed. The completed requests are benchmark requests, rather than a count of finished customer tasks.
What this means for private AI capacity
An assistant can revisit the same documents, instructions, and conversation history across many turns. If the serving stack repeatedly processes work it could reuse, available GPU capacity does less useful work for the application.
Our result suggests a practical step before expanding an inference deployment: measure the existing system with representative traffic and evaluate its serving configuration. For workloads with similar reuse patterns, software changes may provide additional capacity within the same GPU allocation.
That is capacity headroom to investigate. Converting it into a cost-per-request estimate requires the actual hosting price, utilization, CPU and memory resources, and application quality targets. We did not measure customer cost savings in this study.
How we found the improvement
ProsGrow combined an updated open-source serving stack with better reuse of eligible context across agent turns. We established a local baseline, evaluated candidate configurations, and compared complete benchmark runs.
The work builds on DeepSeek and the vLLM ROCm ecosystem. Our contribution was to identify the deployment opportunity, integrate the configuration, and measure its behavior on MI325X. The 60.8% local gain reflects the combined software and configuration update.
This extends the workload-focused approach in our earlier DeepSeek latency study. That study and these October measurements are separate experiments; their percentage gains should not be added together.
Measurement notes
These are ProsGrow's local engineering results using the archived InferenceX AgentX protocol: recorded agentic traces, one-hour scheduled runs, and simulated speculative acceptance. The simulation controls a performance-test condition; production responses require real verification and separate validation. Throughput counts generated output over the observed request window, including trace pauses.
The headline compares our measurement with a dated public reference run, retained as row 442868 in our October 19 snapshot. It is not a claim of an updated InferenceX ranking. The local before-and-after pair provides the same-machine comparison. Each local configuration reported here has one formal run, with no confidence interval.
A separate, small quality check used a different GPU allocation and concurrency. It does not establish quality for this two-GPU, 32-session configuration. Customer-task quality and real-serving response times remain part of production qualification.
Find the capacity in your deployment
ProsGrow can help evaluate private DeepSeek hosting against your request mix, traffic, response-time targets, and infrastructure budget. Start with a measured baseline, then test how much capacity the right serving configuration can recover.