AI Agent & LLM inference enthusiast | ex Computer Science @USCViterbi｜Software Engineer @awscloud
Los Angeles, CA ⇌ Seattle, WA
Joined June 2022
- Tweets2.4K
- Following863
- Followers8K
- Likes10.5K
Really happy to be part of this🥳
Most agent memory work asks what to remember. We goes a step further and asks where and how that memory should live.
I believe this will become a go-to reference for anyone working on multimodal agents.
Multimodal agent memory is on arXiv!
📈 From static retrieval, to dynamic update, to self evolving!
🤖 Across Image, audio, video, GUI, and Embodied agent.
📱Important for personal agent and RSI…
Explore our observatory and submit papers by website to contribute!
📝 seeing-maintaining-learning.…
📄 arxiv.org/abs/2610.19366
#AgenticAI #RSI #PersonalAgent
SGLang Omni brings interactive superintelligence to everyone. 🤩
MiniMax M3.1 Flash + SGLang Omni: building toward a more interactive generation of voice agents. The shift ahead is from “ask, wait, listen” to conversations people can interrupt, correct, and redirect as a task unfolds.
MiniMax M3.1 Flash brings intelligence and fast, concise responses. SGLang Omni adds streaming, conversation state, and interruption handling so users can redirect conversations in real time.
Together, they give developers a foundation to explore proactive tutors, support agents, and hands-free copilots - pairing Flash’s capabilities with infrastructure for continuous interaction.
Thanks to the SGLang Omni team for exploring this direction with M3.1 Flash! ✨
太丝滑了，真的好用，已卸载我的typeless
很荣幸给大家介绍并演示 Voxt，一款开源的 Mac 语音输入工具。
Voxt 已经持续迭代了半年多，也经历了上千位用户的实际使用和反馈。这次，我们把 SGLang Omni 的能力带入 Voxt 的推理 stack，进一步提升本地推理性能。之后，我们也会持续优化 Voxt 的使用体验。
现在，语音输入工具已经很多了。Typeless、豆包输入法、智谱输入法、微信输入法，都提供了相关功能。Voxt 的优势在于，整套软件 stack 开放、可控，我们可以从应用到推理过程持续优化，兼顾性能和稳定性。如果选择完全本地的推理配置，语音识别和文字润色都在自己的电脑上完成，语音和转写内容不需要上传到远程推理服务。
今天，我们会重点展示 Voxt 在专业术语输入上的表现，特别是 AI 工程里的模型名称、框架名称和英文缩写。这类表达，仅靠基础 ASR 转写，很容易出现识别偏差。
Voxt 把热词识别、带有个人系统提示词的 LLM 润色，以及推理性能优化结合起来，让输入既高效，也能更准确地处理专业词汇，贴合个人的表达习惯。
这次演示用的是我的 M5 Pro Mac。
语音识别使用 SGLang Omni 的 MLX 分支，文字润色则由 Ollama Serve 在本地运行 Qwen3.8 27B。两步都在这台电脑上完成。
当然，Voxt 也为 ASR 和 LLM 提供了接口。你可以根据自己的需要，选择远程识别服务和远程大模型，不必局限于这次演示的本地配置。
我和雪松用了 Voxt 半年多，也从日常用户变成了贡献者。这次接入 SGLang Omni，是我们进一步合作的一个里程碑。
感谢关伟老师和所有参与者。欢迎大家下载试用，也欢迎一起把 Voxt 做得更好。
鸣谢：关伟 · 叶雪松 · 刘崇寒 · 周屿涵 · 邓家昕
项目：github.com/sgl-project/sglan…
下载：github.com/sgl-project/sglan…
Jeffro retweeted
Excited to see what the community has built with SGLang-Omni!
It’s a pleasure to introduce and demo Voxt, an open-source voice input tool for Mac.
Voxt has been actively developed for over six months, shaped by real-world use and feedback from more than a thousand users. With this update, we’re bringing SGLang Omni into Voxt’s inference stack to further improve local inference performance. We’ll also continue refining the overall user experience.
There are already plenty of voice input tools available. Typeless, Doubao Input Method, Zhipu Input Method, and WeChat Input Method all offer these capabilities. Voxt’s strength is its open, controllable software stack: we can continuously optimize everything from the application to inference, improving both performance and stability. With a fully local configuration, speech recognition and text refinement run entirely on your own computer, so your audio and transcriptions don’t need to be uploaded to a remote inference service.
Today, we’ll focus on how Voxt handles technical terminology, particularly model names, framework names, and English acronyms used in AI engineering. Basic ASR alone can easily misrecognize these terms.
Voxt combines hotword recognition, LLM-based text refinement guided by a personal system prompt, and inference performance optimizations. Together, these make voice input efficient, improve its handling of technical vocabulary, and adapt it to your own style of expression.
This demo runs on my M5 Pro Mac.
Speech recognition uses SGLang Omni’s MLX branch, while text refinement uses Qwen3.8 27B running locally through Ollama Serve. Both steps run on this machine.
Voxt also provides interfaces for both ASR and LLMs. You can choose remote speech recognition services and remote language models to suit your needs—you’re not limited to the local setup shown in this demo.
Xuesong and I have been using Voxt for over six months, and we’ve gone from everyday users to contributors. Integrating SGLang Omni marks another milestone in our collaboration.
Thank you to Guan Wei and everyone involved. We invite you to download Voxt, give it a try, and help us make it even better.
Acknowledgments: Guan Wei · Ye Xuesong · Liu Chonghan · Zhou Yuhan · Deng Jiaxin
Project: github.com/sgl-project/sglan…
Download: github.com/sgl-project/sglan…
Jeffro retweeted
We brought SGLang to NVIDIA Vera Rubin and accelerated Kimi K3 inference.
Working closely with @NVIDIA, we optimized attention, MoE, and speculative verification kernels on early-access Rubin hardware.
Highlights:
• Up to 20% faster FP8 MLA at batch 1 / 128K context
• 20% faster KDA verification, with bitwise-identical output
• 5.9% end-to-end inference speedup from MoE tail fusion, removing 276 kernel launches per decode step
SGLang also powers rollouts for Miles' end-to-end RL training on Rubin, including agentic RL with 64 concurrent sandboxes on the Vera CPU.
Full results and engineering details 👉 lmsys.org/blog/2026-10-09-ve…
Jeffro retweeted
This first SGLang Summit will gather people from a broader ecosystem. Join us to meet people, products, users from different AI verticals, frontier research, cool hardware invents for the new era, and hear updates and customer stories around! Huge thanks to the whole community for sponsorships ❤️
Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.
Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.
Register: sglang.io/summit
Jeffro retweeted
Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.
Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.
Register: sglang.io/summit
Jeffro retweeted
快来 SGLang Omni 与 StepAudio3 的活动！
非常感谢阶跃朋友们的邀请，许久前就听说车企有许多与阶跃合作，构造车端一体化交互模型。这次 StepAudio3 发布，期待阶跃的多模态矩阵更上一层楼！
这一次阶跃将发布 StepAudio3 全新模型，并且在旧金山展开一次线下的产品 + 开发者交流会，我们团队的顾景文 Jingwen Gu 会作为嘉宾分享我们 SGLang Omni 在全双工模型的通讯协议和模型设计，也会和阶跃的朋友们共同分享下一代多模态交互式模型的技术愿景，欢迎来玩！！！
Jeffro retweeted
We bumped a dependency and quietly got dumber. Same weights, same inputs for models, the whole diff was the order four numbers get added.
This serves as a reminder that we need to pay attention to every action we might consider harmless in codebase.
