RouterPlus (YC F26)

There is an absolute barrage of inference providers popping up that get to $1M/mo extremely quickly. This is the rough economics of inference providers: Even if you rent on-demand B300s at $8/GPU-hr, many open source models (GLM 5.3, Kimi K3, DeepSeek 4.1 Flash / Pro) can get you >$10-20/GPU-hr at reasonable throughputs (100tok/s). DeepInfer from DeepSeek directly shows they can get $38/GPU-hr on off-peak hours. Every chip / model has a different throughput / interactivity curve, but there is a lot of arbitrage between even high rental spot prices and token prices. New chips (Vera Rubins) also change this meaningfully. Your ability to be making gross margin relies on your utilization and how good your stack is. Gateways like OpenRouter, Vercel and more help solve that problem by aggregating demand to you. May inference providers discount their tokens to get longer term commits too. I've even heard stories of selling back Chinese model tokens to Chinese labs! One thorn in the economics of these players is that many Chinese open source model labs are now asking for rev share if you're at a certain scale, espp for pre-release access to their models for day 1 support. That further pushes the threshold to be in the money. The barrier to entry seems low and falling. There are even stories of smaller ones who can "vibe code" a way to stand up OSS models on a cluster they rent. And as a result, we're seeing 10s of such companies pop up almost every week and immediately skyrocketing revenue at low-ish gross margin and raising rounds on the back of that. Unsurprisingly, because the actual service provided seems commodity, this eventually drives up compute prices and drives down token prices. In theory, it would seem like neoclouds who have the lowest cost of compute and themselves have started inference businesses would win the race to the bottom (Crusoe Coreweave Nebius and GMI all have inference businesses), with larger players (Modal Fireworks Baseten) being second because they are well capitalized to make bigger compute commits cheaply and have demand (high utilization), but the reality is the pie is growing so quickly that even upstarts are absolutely crushing it and growing aggressively (inference dot net, DeepInfra, Wafer, Relace, Morph). Strategies vary across the board: neoclouds want to serve inference, inference players want to be neoclouds and do higher margin reserved instances / post training, and some want to go heterogeneous hardware or creatively flexible compute constructs. The risks are: do open source models continue to be competitive? Will their creators rent seek more? Will their shape change compute needs? How do different chips change economics? Who gets compute in a supply crunch? Below is a aggressive sketch of how gross margin scales with utilization for various models / chip sets. For now, there are many inference providers and they’re all ripping!

Krea 3, the new image editing model from @krea_ai, is now on @useRouterPlus . High-quality edits in under 6 seconds. Reimagine the scene while keeping your subject consistent, or blend multiple reference images with a simple text prompt.

We've been monitoring a decline in HuggingFace downloads across the top open LLMs we track for interconnects. It looks like there was a slight change in counting of downloads, as we saw a ~30% download rate drop for many top models, but this is continuing over time. What else do people think it may be?

Yesterday I posted about inference price war pushing margin down. But I since learned inference scammers are a thing with shady price manipulations: As inference providers, - You can lower your input token price 100x cheaper but up your output price 5x. You rank higher in openrouter in price but tots cost is much higher. - You can quantize the model to serve cheaply but users are buying a shittier model. - You can retain and sell user data elsewhere to make more money. - You can secretly swap to DeepSeek v4 flash but say it’s GLM 5.3 10% of the time. Therefore it’s hard to know 1. You get the model you ask for 2. You pay for the tokens you actually burn across cache input, input, output and 3. Your data is safe. I’m sure openrouter is working on it AND baseten, fireworks, together and other enterprise brands will continue to thrive even with higher pricing because they sell peace of mind.