AlphaSense

Interview with an industry expert on why GPU utilization decides whether to build or buy AI inference ( $AMZN, $MSFT, $NVDA ): - The expert says GPU compute is not elastic the way CPU compute is. The cloud providers the expert's team works with always want a three-year contract, and adding GPUs in the middle of a quarter takes at least a month of lead time on top of that commitment. Because of the three-year commitment, the team keeps paying for a GPU instance even after turning it down, so it has had to build engineering optimizations to work around the problem, while CPU capacity can simply be bought on spot instances when demand rises. - The expert sees GPU availability by region as a real challenge, since GPUs are only available in certain regions while customers span hundreds of regions, so routing traffic is difficult, especially when customers have data retention or locality requirements. To manage this, GPU capacity is spread across AWS, Azure, and GCP in selected regions. - According to the expert, $SNOW works with the three largest cloud providers in a rough 60%, 30%, and 10% split, with AWS clearly dominant. $SNOW is described as one of AWS's largest customers, having announced a $6 billion renewal of its spending commitment on its latest earnings call. - On hardware, everything is $NVDA, with the mix chosen on price-performance. Today it is mainly A10G, H200, and B200, plus some L40s. Asynchronous batch workloads open up better optimization opportunities, since they favor throughput over latency, so L40s can replace B200S for much better price-performance. The dollar split across SKUs is dynamic and subject to change. - The expert notes that GPU utilization depends on product usage, which fluctuates a lot. According to him, break-even sits at about 40% average utilization, counting quiet periods like nights and weekends, with business hours running higher. At that level, a company is indifferent between sending inference requests to Bedrock and self-hosting an open-weight model, and anything above it makes self-hosting more favorable, which makes utilization the key lever in the build-versus-buy decision.

Former $META employee on why almost any task that involves the web will be moving to a personal assistant ($AMZN, $WMT, $GOOGL): - The expert sees the personal agent space as both powerful and valid, with the experience finally coming together after years of startups trying to build a true assistant. Competition is heating up, with $GOOGL rumored to release its own agent around the end of the year. - According to the expert, retention will depend on the experience, specifically how autonomously an agent handles a task and how well it remembers a user and improves its suggestions over time, while security and privacy remain the current gap. All models are still early on what the expert calls the personal chores quality benchmark, so it is too soon to say which product wins. - The expert's view is that almost any task that involves interacting with the web will move to a personal assistant, and that this applies to personal assistants in general, not only Muse. Muse is described as surprisingly good at handling these kinds of web-based tasks end to end once it is connected to a user's accounts, which the expert sees as an early sign of where the whole category is heading. - The expert says Muse works almost entirely through the browser, operating websites the way a person would, with only an estimated 1-2% of interactions using APIs, mostly for services like Gmail. As far as the expert has seen, $META's partnerships with retailers like $WMT are not special integrations but a way to avoid blocks, such as the one $AMZN put on Muse, and to make retailers' sites discoverable by agents, since the models are already very good with the browser. - The expert highlights $AAPL as the company best placed to benefit from personal agents on the consumer side, if it executes well. Its strong brand, the richest user base in the world, and an ecosystem that ties everything together give it a real edge, with the iPhone as the most powerful device to host an agent. However, the expert is unsure $AAPL can deliver, noting that it has been nearly silent on this for three to four years and that agents would erode its App Store revenue as apps become less necessary.