We benchmarked the Gulf's sovereign models on an H100. Here is what we found.

There is a lot of pride in the new wave of sovereign models, the Arabic and Indic systems built in the Gulf and India, and there should be. But pride is not a number. Before we commit capital to serving these models, we wanted to know one boring thing precisely: how many tokens a second do they actually produce on a real GPU, and what does that mean for the cost of a token? So we rented a single H100 for about an hour, served three of them with vLLM, and measured. Here is the data, the gotchas, and the economics it implies. No vendor slides, just what the cards reported.

The setup

One GPU, clean conditions, repeatable method. The goal was a defensible throughput figure per model, not a leaderboard.

A note on honesty: we wanted an A100, the cheaper card, but on the day the cheap on-demand stock was gone, so we ran on an H100. That is its own small lesson about this market, and we will come back to it. The benchmark itself is a one-line tool that ships with vLLM:

The numbers

Here is what one H100 produced, in aggregate output tokens per second under full load.

If you serve open models, two of those numbers will look low, and you are right. A plain 7B Transformer on an H100 will happily push several thousand output tokens a second; Falcon-H1 7B gave us under two thousand. The 34B, at 251, is in a different category of slow. The reasons are interesting, and they are not about model quality.

Why the hybrid models are slower to serve (today)

Falcon-H1 is a hybrid: it mixes Transformer attention with Mamba, a state-space layer that trades the attention cache for a small fixed-size recurrent state. On paper that is a serving dream, because the memory per sequence stops growing with context. In practice, in mid-2026, the serving stack has not caught up with the architecture. Two things bit us:

First, concurrency is capped by Mamba cache blocks. vLLM allocates a fixed pool of these state slots, and each in-flight sequence needs one. The engine refused to start until we lowered the number of concurrent sequences to fit the pool. Fewer concurrent sequences means less batching, and less batching means lower aggregate throughput. That is most of the gap between Falcon-H1 7B and a comparable plain Transformer.

Second, the 34B simply does not leave room to go fast on one card. Its weights take about 68 of the 80 gigabytes, which left too little for the working memory that the fast path needs, and the first run failed with an out-of-memory error during graph capture. It ran only after we disabled that optimization and shrank the batch, which is why 251 tokens a second is a floor, not a verdict. Quantize the weights to 8-bit, or give it a second GPU, and that number moves a lot.

From tokens a second to the cost of a token

Now the part that decides whether any of this is a business. A GPU costs the same whether it is flat out or sitting idle. So the economics of inference are not really about the price of the card, they are about how full you keep it. The decisive number is utilization, and throughput sets how much revenue a full GPU can earn.

Work it through for the 7B. Call a rented H100 roughly $2.70 an hour on the open market. At 1,963 output tokens a second, billing prompt and completion together, a fully loaded card moves on the order of 21 million billable tokens an hour. At open small-model rates of around ten cents per million tokens, that is about $2.10 an hour of revenue when the card is pinned at 100 percent. Against a $2.70 cost, the card does not pay for itself even when it is completely full.

Read that first number again. On rented hardware at those rates, a small sovereign model needs more than a full card's worth of demand to break even, which is impossible. That is not a DesertGrid problem, it is the shape of the whole market: serving frontier-quality open models cheaply, on someone else's expensive GPUs, does not close. There are only three honest ways to make it close, and they are the entire strategy.

The three levers, and the only durable one

Throughput. Quantize the weights, batch harder, wait for the serving stack to mature. This is real and it helps, especially for the 34B, but it is a moving target you do not control.

Price. Charge more. Sovereign models can justify a premium, because they are in-region, data-resident under local law, and tuned for languages the global providers treat as an afterthought. Worth doing, but a premium only stretches so far before a buyer reaches for a cheaper generic model.

Cost base. Lower the denominator. This is the one that compounds. Notice what happened to the break-even number above when the card was owned and run on cheap power instead of rented: it fell from impossible to roughly sixty percent utilization. Most of a rented GPU's hourly price is the chip's own capital cost amortized over its life, and the next biggest line is electricity, paid continuously on every token for as long as the hardware runs. Own the chip, run it where power is cheap, and keep it busy, and the same model that loses money for a renter can make money for you.

Update: we ran the throughput lever

We did not want to leave "quantize and it moves" as a hand-wave, so we went back to the same H100 and re-served every model in fp8, the eight-bit weight format that halves the memory a model occupies. Same card, same workload, before and after:

The pattern is the lesson. fp8 helps a plain Transformer most, because the memory it frees turns straight into more parallel requests, so Sarvam-M jumped by half. The Mamba-heavy Falcon-H1 7B gained the least, because its ceiling is the state-space kernels, not memory. And the throughput-starved 34B more than doubled, exactly where we expected, because halving the weights is the difference between a model that barely fits and one with room to work. It is still memory-bound enough to refuse the fastest execution path on a single card, so a second GPU would push it further again.

What does that do to the economics? It moves the lines, and it does not move the conclusion. The 34B crosses from "cannot break even at any load" to a still-demanding eighty percent or so of a full card, and Sarvam-M now breaks even under half a card's worth of demand. The small 7B-class models stay marginal on rented hardware, the same story the box above told. Optimization narrows the gap. The cost base is what closes it. We folded the winning settings into how we serve each model, so the speed is not a one-off benchmark, it is the default.

One honest caveat: fp8 is a lossy format. The throughput is free, the accuracy is not quite, so a serious deployment validates quality before it ships the speed. We measured the speed here; the quality pass is its own piece of work, and we will not quietly skip it.