MTS @inferact, ex-database engineer @RisingWaveLabs, bachelor @SJTU1896, created MNGA App (https://nitter.cf/t.co/7GD0KRoGhd)
Singapore
Joined September 2011
- Tweets1.6K
- Following548
- Followers3.5K
- Likes11.3K
Yab 确实是体验下来最丝滑的一个，就连 WebKit 最难受的触摸板返回动画也解决掉了 🤔
一直希望世界上有一个好用的 WebKit 浏览器，因此之前还连续赞助了 @OrionBrowser 一两年，谁料你们太不争气了，bug 就没断过 🤡 被现代 vibe coding 几周的东西吊打
Перепробовал кучу браузеров на замену Arc и довольно уверенно заземлился на Yab (@yab_browser).
Шустрый, нормально работает, регулярно обновляется. И у людей хороший вкус.
Пространства, вертикальные вкладки, несколько страниц рядом в split view. Данные из Arc перетащил хорошо. AI встроен классно: можно подключить уже оплаченные подписки Claude или ChatGPT, спрашивать про открытую страницу и поручать ему дела в браузере.
И всё это на WebKit (а WebKit лучше хрома!).
Наконец-то позволил себе окончательно снести Arc.
yetanotherbrowser.com/
Bugen Zhao retweeted
Refactoring gave us the Rule of Three: see similar code a third time, extract it.
That rule was written for humans, who forget to update the other copies. Agents don’t really have that problem.
Talked about this with @BugenZhao from @inferact recently, and we’re on the same page.
So we’re letting our agents figure out the new best practices for code reuse, optimizing for the lowest long-term maintenance cost.
后来发现这样蜂窝基带基本上没法休眠，还是太费电了，而且一旦某个 CDN 被DNS 解析成 SG 的地址，一来一回加载个图片/视频/小程序简直灾难 🤡
果然还是 SG 家里 Mac mini 常驻一个 sing-box 当🪜的最后一跳出口最省心 😇
Nothing tells you how Opus 5.5 and the GPT-6 series are doing this month better than vLLM's contributor graph on GitHub 😇
Bugen Zhao retweeted
Singapore, let's talk inference.
Join the vLLM meetup with engineers from @inferact, @NVIDIA, @alibaba_cloud and the vLLM-Omni community: high-throughput prefill, RL, Blackwell, Vera Rubin, agent infrastructure and interactive world models.
Nov 13 · 1–5:30 PM SGT · SQ Collective
Free, with registration approval.
Bring your serving questions. Meet the people building the stack.
luma.com/0k9243bi
Bugen Zhao retweeted
vLLM matches DeepInfer, DeepSeek's own engine for DeepSeek-V4.1-Flash, on DeepSeek's decode configuration across batch sizes 8 to 256, and does it at DEP32 against DeepInfer’s EP64DP64.
DeepInfer was originally built on vLLM. By integrating DeepSeek's open-source kernels and extensively optimizing prefill and decode, vLLM now matches DeepInfer's performance curves.
或许是以后的日常 🥵 无法想象年底之前还会有多大的网络安全事件发生，到时候或许没人能置身事外
So basically, today PHP got two critical bugs that let anyone run code on your server without even logging in, which is bad, and then the details leaked before the patch was even out, which is worse, and it works on WordPress and MediaWiki and Magento and Joomla and Laravel, and Censys says like 1.5 million servers are running a version with no fix, so the NSA was like "take your system offline," and the FBI was like "take your system offline," and CISA and the UK and CERT-EU were all like "take your system offline," and then the Prime Minister of Japan, the actual Prime Minister, posted at 5:55 in the morning telling 48.8 million people to take their system offline, so everyone took their system offline, like the Dutch cybersecurity agency took its own website offline, and the Dutch parliament went offline, and the Port of Rotterdam went offline, and Finland and France and Denmark started pulling government sites, and Wikipedia went down, and Archive.org went down, and Asana and GOG and Something Awful went down, and the University of Vienna's webmail went down for some reason, and that's just PHP. Because GitLab also dropped a CVSS 10.0, which is the highest score they give out, for a bug that's been there since 2014, so that's every self-hosted GitLab for twelve years, and then LiteLLM 1.107.0 got hijacked and shipped a credential stealer and a remote access trojan in every single wheel and the Docker images, including latest, and you didn't even have to import it, just installing it was enough, and it's the second time this year, and then Railway paused every deployment on the entire platform because someone got into the environment variables, so now everyone has to rotate every secret they've ever stored there, and TiDB Cloud said someone was walking around inside with stolen credentials for basically a month and an extortion group says they took 10.6 terabytes, and Zeabur got breached for the second time in two months and is just shutting down, like the whole company, and Node.js patched remote code execution and a TLS bypass, and CISA says Plex is being actively exploited, and someone took over 300 Tor exit relays to steal SSH passwords, and SafePay ransomware has hit over 600 critical infrastructure orgs, and India's biggest bank filed a data loss incident, and Japan has been in an official state of emergency in cyberspace since October 9, which honestly explains the 5:55 AM post. Oh, and eMudhra, a certificate authority that's issued over 60 million certificates, whose entire job is proving websites are who they say they are, was supposed to be using private keys that are random numbers over a hundred digits long, and one of them was 4. Just 4. And the others were like 66 and 237. Anyway, it's Wednesday.
Bugen Zhao retweeted
🚨🚨🚨 vLLM is all you need to reproduce DeepSeek’s performance.
With Wide-EP and EPLB, vLLM can match DeepInfer’s decode performance within 6% and generate $39/gpu/hr in revenue at DeepSeek’s off-peak pricing, at 200 P90 TPS on the AgentX workload.
Specifically:
• Under DeepSeek’s decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer’s published results across batch sizes 8–256.
• On AgentX, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/gpu/hr in revenue at 196 P90 TPS, based on DeepSeek’s off-peak API pricing.
• Also, vLLM extended low-latency kernel support for DeepSeek-V4.1-Flash, achieving up to 826 TPS per user on AgentX.
Bugen Zhao retweeted
Grateful to @deepseek_ai for releasing DeepInfer, initially built primarily on @vllm_project, under Apache 2.0 and crediting the open-source community!
We've worked hard on DeepSeek V4.1 Flash inference efficiency and will keep improving with ideas from DeepInfer! Stay tuned 🙌
🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!
Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.
🔗 github.com/deepseek-ai/DeepI…
1/4
Bugen Zhao retweeted
Thank you @a16z for telling the Inferact story.
Four years in, inference is still far from solved. Models keep getting bigger, workloads keep getting more varied, and the hardware landscape keeps expanding. Inferact exists to build the future of inference with the vLLM community.
We’re hiring across all teams in SF and Singapore: inferact.ai/careers
In 2022, Berkeley PhD student Woosuk Kwon thought he could make Meta's new open-weight model, OPT-175B, run faster in two weeks.
Four years and 3,000 contributors later, that project is @vllm_project, one of the largest open-source inference engines, running on ~500,000 GPUs at any given time.
He and fellow researchers Simon Mo, Kaichao You, and Roger Wang borrowed "virtual memory" from operating systems, handing out memory only as it's needed so each GPU can juggle requests of wildly different sizes. (The "v" in vLLM is for virtual.)
Their first user, Berkeley neighbor @arena, cut its GPU needs in half. Today Google, Amazon, and Meta all run vLLM in production.
In January, the core maintainers founded @inferact and raised $150M to give vLLM a full-time team.
They're hiring across eng, marketing, and devrel in SF and Singapore: a16zjobs.substack.com/p/infe…
@simon_mo_ @woosuk_k @KaichaoYou @rogerw0108
Bugen Zhao retweeted
Our first TPU megakernel for Kimi K3 reaches 709 tokens/s on low-concurrency decode, against 450 tokens/s for our GB200 baseline, both with DSpark speculative decoding.
To our knowledge, this is the first TPU inference megakernel. The whole model runs in a single Pallas kernel, and without spec decoding it is roughly 1.4 to 2x the GB200 baseline at batch sizes 1 through 8.
We are open sourcing it today.
1/2
Bugen Zhao retweeted
my hot take is specialized inference engines are amateur projects to spend the spare tokens when @thsottiaux gives people one more reset😄
an inference engine is way more than just running a model on a certain hardware, it is an inference ecosystem.
I use this slide several times when I talk to people about vLLM. Models, hardwares, and inference techniques, all of them move quickly. And vLLM lies in the intersection to provide a unified interface to end-users and applications. vLLM is the inference ecosystem.
vLLM not only support current models and hardwares, but we are also working to support new models and hardwares coming in the next few months. It's an ecosystem people can trust and rely on.
Over the past 3 years, I have seen so many projects claiming to be better than vLLM in certain aspects, but in the end either their techniques are contributed to vLLM or they disappear. That's the power of ecosystem.
A specific example would be tilert github.com/tile-ai/TileRT , a megakernel inference engine dedicated for decode. They collaborate with vLLM by using vLLM prefill + tilert decode in vllm.ai/blog/2026-07-14-vllm… , as highlighted in newsletter.semianalysis.com/… from @SemiAnalysis_ .
50000 !
With Kimi K3 Day-0 on vLLM: Open Frontier Intelligence for Everyone 🚀
At 2.8 trillion parameters, Moonshot AI's Kimi K3 is one of the most powerful open-weight models ever released. Starting today, you can serve it on vLLM the moment the weights are public.
What K3 brings:
🧠 2.8T-parameter Mixture-of-Experts (16 of 896 experts active per token)
📚 1M-token context window
🛠️ Native multimodal understanding, including vision
⚡ Kimi Delta Attention: a hybrid of linear and full attention that makes million-token context affordable
Huge thank you to @Kimi_Moonshot AI for the model release and partnership, @inferact for leading the vLLM optimizations, and to our partners at @nvidia, @AMD, and the broader vLLM community.
1/6
Bugen Zhao retweeted
Great writeup from @khluu000 👏
How does vLLM stay production-quality while merging ~2,000 commits/month and shipping every 2 weeks?
The team broke down the three layers that make it possible. A huge community effort—thank you to everyone who is helping along the way.
vllm.ai/blog/2026-07-16-keep…
Bugen Zhao retweeted
Announcing the first-ever vLLM Conference — hosted by @inferact at Ray Summit, Aug 24–26 in San Francisco 🎉🌉
This is where we'll get into the work pushing open, high-performance inference forward, such as:
🗺️ Where the vLLM roadmap is headed
⚡ Getting the most out of accelerators including NVIDIA, AMD, TPU
🔗 Wiring vLLM into training and serving pipelines
🚀 Running inference on production scale
The summit features speakers from Inferact, NVIDIA, AMD, Google TPU, Anyscale, PyTorch, Meta, Red Hat, and more 🎤
Come learn where the future of inference, open source, and AI is heading — and meet the leading builders driving it 👇
vllm.ai/events/vllm-conferen…
帮同事看一间 shophouse 被 60 多岁的房东拉着聊了一个小时，和我探讨美国政府为什么打压 Anthropic，GLM 5.2 体验如何，和能不能用 vLLM 在 Mac Studio 上跑本地模型…🤔
