shoshin

Engineers like multi-layer platforms Marketers like circles The reality is that life is spiral

Interesting. Revolut reportedly only started seeing meaningful AI productivity gains after moving away from expensive OpenAI and Anthropic models to self-hosted GLM. Decagon reportedly runs ~90% of its workloads on open weight models. Why? Because when you're running AI agents in production for e.g. customer service, cost and latency matter enormously. You don't need a frontier model with PhD-level reasoning or even a model that knows the capital of Finland to handle every customer interaction. We're seeing similar dynamics across hypergrowth AI startups and increasingly the large enterprises we work with. But most importantly, the future isn't binary. When use cases are new, you want the smartest general-purpose model available. But once workflows mature and become predictable, smaller, faster, specialized models increasingly win. I wouldn't be surprised if 80–90% of routine enterprise AI workflows eventually run on open-weight and smaller models. This doesn't mean frontier labs lose. It means the market is splitting into two massive opportunities: frontier intelligence and specialized execution at scale.

The below argument is conflating software abstraction layers with hardware execution to claim $NBIS s managed services are "weak". This ignores company timelines which shows the absolutely immaculate pace of improvement by $NBIS on their Token factory offerings and the complex unit economics and technical nuances of inference. Look at the maturity of these app tier platforms: 1/ @baseten : Founded in 2019 (~7 years old) 2/ @modal : Founded in 2021 (~5 years old) 3/ @togethercompute & @FireworksAI_HQ : Founded in 2022 (~4 years old) Meanwhile $NBIS Token Factory is barely 1 year old, yet it already competes with all the above players across a range of SOTA models and continues to improve and take market share at an insane pace, not just with software level optimization but proprietary hardware + software co designs. Dismissing the violent pace of their growth because older software wrappers had a multi year head start is shortsighted and ignorant. To dismantle @jiahanjimliu core question "Why sell bare metal to Modal if $NBIS can sell that same capacity via Token Factory?" it assumes that selling tokens/managed inference is always superior to selling bare metal. That is fundamentally false from a first principles economic standpoint: 1/ GPU & Workload Matching: Depending on the specific GPU generation, memory bandwidth constraints, and the exact model architecture being served (dense vs. sparse MoEs), the risk adjusted return of running inference can easily underperform a locked in, high paying bare metal contract. Knowing when to lock in a contract  vs to sell tokens matters here. A great example to how well $NBIS handles this is them renting GPUs to Baseten to specifically serve their video gen model stack instead of using that compute to serve the same video gen model stack on just Token Factory. Vid gen models in this case are significantly less profitable because they require heavy O(N squared) spatial temporal attention through long sequences of frames plus multiple sequential diffusion denoising steps for every single clip processed. This creates heavy VRAM overhead that restricts concurrent request batching which tanks hardware utilization efficiency relative to text models. 2/ Guaranteed Utilization: Selling raw compute to a mature player like Modal which crossed $1B ARR and scaled 15x secures massive, predictable cash flow without taking on app layer demand volatility. It's optimal capital allocation and diversification not a flaw. Also you gotta "Keep your friends close and your enemies closer" lol Furthermore comparing inference platforms to $NBIS misses where the technical moat I had defined several times before truly lives: 1/ Co Design vs. Wrappers: Modal rents multi tenant compute and wraps it in Python abstractions. 2/ Token Factory is engineered from the bare metal up optimizing InfiniBands, custom hardware kernels, co engineering with industry leaders like vLLM and PyTorch alongside hardware end optimizations like integrating Custom ASICS like the Groq 3 LPX on top of the $NVDA stack for their own inference infra to optimize serving from the hardware upwards unlike the rest of the above players who are stuck to software only optimization from the GPUs upwards. 3/  The Long Term Stack: If open source runtimes (like DeepSeek) and vendor optimized stacks commoditize pure software orchestration, app layer margins will compress. Owning the bare metal infra alongside the inference stack ensures ultimate pricing power which only $NBIS has. This ultimate aim of $NBIS Jim is trying hard to ignore:  Diversification across the stack. By monetizing both bare metal supply and managed inference, $NBIS minimizes execution risk and maximizes safety across market cycles rather than being locked into a single vulnerable layer. When one or the other from the bare metal or the software stack gets hit by margin compression collapsing one or the other from players like $IREN , $APLD or the above purely inference platforms, $NBIS will survive due to this exact layer based diversification. Aside from hyperscalers ($GOOG & $AMZN ) no other pure play Al company owns the complete stack from physical data centers, power, and hardware infrastructure all the way up to the cloud layer and managed inference plus even agentic offerings. That isn't a contradiction, that's real vertical integration.

$NBIS Just Did to GPUs What Airlines Did to Seats