friendliai (@friendliai) header
friendliai (@friendliai) profile picture

The Frontier Inference Cloud for Agents

San Francisco, CA
Joined April 2022
Same open-weight model. Different provider. Different results. Quantization, kernels, caching, and routing shape quality, speed, and cost. For GLM-5.3, FriendliAI had the lowest median TTFT (266 ms) of 22 providers on Vercel AI Gateway. Read more 👇 friendli.ai/blog/same-model-…
2
82
friendliai retweeted
We had the best time with @friendliai and all the developers that came out to the meetup in Seoul! Thank you all for coming!
13
5
68
7,973
A great evening deserves a little rewind. 🎬 Here's a look back at our @ollama × @friendliai Executive Dinner in Seoul! We brought together about a 100 industry leaders for an evening of insightful conversations on open models, AI inference, and the future of AI in production. Curious about the full conversation? 👀 The full session recording is available upon request. 👉 Request access here: qg2m6.share.hsforms.com/2RtZ… *As this was a private, invitation-only event, access to the full recording is subject to review and approval.
2
66
Liner moved Liner Scholar, its AI research agent, from a closed model to GLM on FriendliAI. 💸 ~30% lower LLM cost per turn ⏱️ ~25% lower avg response time per turn The open model was step one. Fast serving was step two. friendli.ai/customers/liner
73
60 seconds from our Seoul Developer Meetup with @ollama 🎥 Packed room, sharp questions, and a lot of builders working on production agents with open models. Full sessions coming soon 🔜
1
1
7
2,592
Almost time! 🚀 Whether you're in SF for Ignite week or want to learn more about AI inference and open-weight models, join us Wednesday evening at our SF office. One block from Moscone. 🎤 Talks from Victor Su-Ortiz (MiniMax), Salman Quazi (Reflection AI), and Yunmo Koo (FriendliAI). 🍕 Food & drinks on us. No badge needed. 📍 Wed Nov 18, 6-8pm PT RSVP: luma.com/cch7p7pw
3
125
Missed our latest FriendliLink interview? 👀 We've put together a quick 1-minute highlight reel featuring some of the key moments from our conversation. Take a look at the highlight below, and catch the full interview on our YouTube channel! 🎥 Watch the full interview: youtube.com/watch?v=_Euq-VJd…
5
117
friendliai retweeted
Roughly 150 developers joined us and @friendliai for a meetup in Seoul last week, where our CEO @jmorgan talked about the open model ecosystem and where it’s headed. Huge thank you to everyone in the community who showed up – more than 650 registered, and we wish there was room for all of you.
14
12
1
80
10,962
What a great turnout for our Developer Meetup co-hosted with @Ollama in Seoul last week! With over 650 registrations, we were excited to see so much interest from Seoul's developer community in building production AI agents with open models. Together with Ollama and @MiniMax_AI, we explored the latest developments in the open model ecosystem and what it takes to make AI inference faster and more efficient in real-world applications. Huge thanks to everyone who joined us for the talks, Q&A, and conversations afterward. It was great connecting with so many developers and engineers!
2
1
8
1,408
Every step an AI agent takes makes its next request longer. Instructions, messages, and tool results pile up, and the model reprocesses that history on every call. At 40 calls per task, the repeated input reaches 4.3M tokens per task. KV caching lets the inference system reuse that work instead of recomputing it. In our new blog, we break down the four mechanisms that make reuse possible at scale: → KV reuse within a request, so each new token computes only its own state → Prefix caching across requests, which cut input token costs by 80% in our 1,000-task example → Hierarchical caching across GPU memory, host DRAM, and disk to retain more agent histories → Cache-aware routing that sends requests to workers already holding the matching state (0.5s vs. 3.1s queue-plus-prefill in our example) As contexts grow and more agents run concurrently, keeping useful KV state available where it’s needed becomes a key lever for inference cost. Read the full blog: friendli.ai/blog/reducing-ag…
3
86
Connect your AI coding agents to open-weight models with FriendliLink 🔗 FriendliLink makes it easy to integrate FriendliAI Model APIs into your existing AI coding workflows, giving you access to open-weight models powered by FriendliAI inference. Our latest video walks through: 🔹 What FriendliLink is and how it works 🔹 How to get started with FriendliLink 🔹 How it connects coding agents to FriendliAI Model APIs 🔹 The technical challenges behind supporting different agent harnesses Watch the video to learn how FriendliLink brings open models into your coding environment. 📹: youtu.be/_Euq-VJdRVQ?si=i7rb…
2
65
Join us at our SF office alongside MiniMax and Reflection AI during Ignite week for an evening focused on ultra-fast inference and open-weight models! 🌉 We'll be breaking down real-world serving architectures, performance vs. cost trade-offs, and scaling for agentic workloads. 🍕 Food & drinks on us (no conference badge needed) 📍 1 block from Moscone Center 🗓️ Wed, Nov 18 | 6:00 PM – 8:00 PM PT Space is limited. Request to join: luma.com/cch7p7pw
2
106
Open Models. Infinite Possibilities. 🇰🇷 What a great evening in Seoul! Together with @ollama, we hosted Ollama × FriendliAI: An Evening in Seoul, bringing together business and technology leaders to explore how open models and AI inference are shaping the future of enterprise AI. From greater model flexibility and data control to the importance of fast, reliable, and cost-efficient inference, we exchanged perspectives on what it takes to bring open models into production and scale AI applications in real-world business environments. A big thank you to everyone who joined us for the insightful discussions and meaningful connections!
2
12
338
FriendliAI is on Vercel AI Gateway. GLM-5.3, Oct 13–20: • 266 ms median TTFT, lowest of 22 providers • 19.8x faster than the provider average Prefer FriendliAI with fallbacks, route exclusively, or bring your own key. Guide 👇 friendli.ai/blog/friendliai-…
3
111
Friendli Inference vs. vLLM on the same 4× B200 GPUs (agentic GLM-5.3-Flash): → 2.1× throughput at 100 tok/s per request (~53% lower cost per token) → 3.1× faster generation at vLLM’s peak throughput Full benchmark 👇 friendli.ai/blog/more-tokens…
4
1
6
179
Open-weight models are becoming a serious part of production AI. @TheFuturumGroup's latest survey found - 42% of surveyed enterprises use open-weight models in production, up from 31% in July. - Gemma is used by 55% of production respondents, with low-complexity workloads accounting for a growing share of open-model usage. While many enterprises run open-weight models on their own infrastructure, self-hosting comes with significant operational costs. They also found - 46% of production respondents use self-managed cloud infrastructure and 42% run models on-premises. - Yet 50% cite self-hosting costs as a barrier to adoption. This highlights an important opportunity for managed inference providers to make open models easier and more cost-effective to run in production. futurumgroup.com/insights/en…
1
3
189
For teams looking to avoid the overhead of self-hosting, managed inference can be a practical alternative. For example, Gemma 4 31B is available on FriendliAI at just $0.14 per 1M input tokens and $0.40 per 1M output tokens, with no GPU infrastructure to manage. friendli.ai/models/google/ge…
1
1
51
A faster model doesn't always mean a faster coding agent. AI coding performance depends on 4 layers: 🤖 Agents 🧠 Models ⚡ Inference 🖥️ GPUs The real goal? Better task success, faster completion, and lower cost per task. Explore the full stack 👇 friendli.ai/blog/ai-coding-a…
2
1
2
353