Tsai Li Ming (@tsailiming) profile picture

Engineer

Tiny red dot in apac
Joined May 2011
Photo by Tsai Li Ming (@tsailiming)Photo by Tsai Li Ming (@tsailiming)Photo by Tsai Li Ming (@tsailiming)
Tsai Li Ming retweeted
vLLM Conference is next week, and we have a packed schedule 🎊 📅. Here's the full list of events you should know: Mon: 🔷 4–6PM vLLM × Ray × Google Cloud Happy Hour: rsvp.withgoogle.com/events/r… 🔷 6–9PM vLLM × Dynamo Meetup: luma.com/r8o604o0 Tue: 🔷 11–11:30AM vLLM Keynote from @simon_mo_ 🔷 12–5PM vLLM Track Day 1 🔷 6–9PM vLLM × AMD Happy Hour: luma.com/6nb8coax Wed: 🔷 12–5PM vLLM Track Day 2 🔷 6–8:30PM vLLM x DigitalOcean × NVIDIA Happy Hour: luma.com/fufg0bg5 No ticket needed for the happy hours and meetups, but space is limited. To join the full event, register here: vllm.ai/events/vllm-conferen…
5
31
4
166
23,303
Define your #AI evaluation strategy with precision. EvalHub collections let you set benchmarks, weights, and thresholds that match your deployment context, not generic leaderboard scores. 🔨 Build measurement criteria that answer your actual questions. red.ht/4vtUV5k
2
6
383
Tsai Li Ming retweeted
Speculators v0.5.0 ships DFlash training support. The Red Hat AI team trained a Gemma 4 31B speculator with it. It beats Eagle 3 and a standalone FP8 quantized verifier on inter-token latency. Combine DFlash with FP8 and the gains stack. Serve it: vllm serve -tp 2 RedHatAI/gemma-4-31B-it-speculator.dflash developers.redhat.com/articl…
2
3
1
23
1,535
Tsai Li Ming retweeted
This one has been in the works for a while. @cedricclyburn teaching LLM inference, compression, and benchmarking with @vllm_project -- free course with @DeepLearningAI. Proud of this one.
New short course: Fast & Efficient LLM Inference with vLLM, built in partnership with @RedHat and taught by @cedricclyburn. Learn to quantize an open-source LLM, serve it with vLLM, and benchmark your deployment across speed, cost, and accuracy. Free to enroll: hubs.la/Q04jXfpR0
3
8
42
4,603
Tsai Li Ming retweeted
The biggest @vllm_project meetup yet is starting soon in Beijing! Looking forward to meeting the many talented speakers and developer community here 🚀
2
1
23
1,058
Tsai Li Ming retweeted
How do you make LLMs route queries to the right models or tools, efficiently and with low latency? Join vLLM Office Hours on Oct. 9 to explore the new Semantic Router (github.com/vllm-project/sema…), a lightweight, high-performance router for @vllm_project. red.ht/office-hours
1
2
7
506
Tsai Li Ming retweeted
What's next for Red Hat AI? Join our Red Hat AI leaders @joefern1, @KatarkiTushar, Steven Huels, & Jeff DeMoss for a Q4 sneak peek into the latest releases and key new features that help enterprises address their biggest challenges. Sign up: redhat.com/en/events/webinar…
3
5
394
Tsai Li Ming retweeted
Big congrats to Red Hat’s newest @vllm_project committer, Nicolo Lucchesi (@nick_lucche)! 🎉 Nicolo already made major contributions: ⚡️ Optimized disaggregated serving with KV transfer + Nixl for faster heterogeneous tensor-parallel setups. 🗣️ Designed & implemented vLLM’s speech-to-text interface, powering v1/audio/transcriptions and v1/audio/translations. 🤖 Advanced TPU integration with optimized sampling, efficient compilation, and improved preprocessing. Now he’s leading the charge on "Disaggregated Everything" - building a tokens-in, tokens-out interface to unlock the next wave of vLLM performance. 🚀 Congrats, Nicolo!
3
17
1,241
Tsai Li Ming retweeted
v0.10.2 marks the first release with official aarch64 support for vLLM! You can now install vLLM directly onto @nvidia 's GB200. Along with the PyPI release, our docker image is also multi-platform so pulling the right image just works. More perf enhancements on the way!
6
21
3
262
17,574
Tsai Li Ming retweeted
🚀 Thrilled to announce GuideLLM v0.3.0! This release is highlighted by a brand new Web UI, containerized benchmarking, and powerful dataset preprocessing. GuideLLM GitHub: github.com/vllm-project/guid… (Thread 👇)
1
11
1
22
2,443
Tsai Li Ming retweeted
Qwen3-Next dropped yesterday and you can run it with Red Hat AI today. ✅ Day-zero support in vLLM ✅ Day-one deployment with Red Hat AI Step-by-step guide: developers.redhat.com/articl… The future of AI is open.
5
2
18
2,280
Tsai Li Ming retweeted
On August 23, we are co-hosting the Shanghai vLLM meetup with our friends at MetaX, so if you are in town and want to learn more about how the community is leveraging vLLM, then sign up and join in! @vllm_project lu.ma/3bex3de0
1
6
18
3,222
Happy 4th July! Speed is the Moat & @AnushElangovan & his team Keeps Running Faster & Faster. Still lots of areas where ROCm has gaps but many are already closing
15
34
8
439
148,877
Tsai Li Ming retweeted
Join us: IMDA x Red Hat Technical Sharing Session! 🗓️ Thursday, 19 June 2025, 2.30pm @ Red Hat SG Office ⚡High-performance inference with vLLM ⚙️ Deploying LLMs in resource-limited environments 🌐 Scalable GenAI across hybrid cloud with llm-d Register: red.ht/4jOhVFR
2
2
116
Tsai Li Ming retweeted
Cool to see @vllm_project used as part of @WhatsApp Trusted Execution Environment (TEE) Private Processing! Paper here: ai.meta.com/static-resource/…
9
32
3,864
Tsai Li Ming retweeted
Thank you @AMD @LisaSu @AnushElangovan for Advancing AI together with @vllm_project! We look forward to the continued partnership and pushing the boundary of inference.
4
18
2
210
17,219
Tsai Li Ming retweeted
🇯🇵 Join us for an in-person @vllm_project meetup on Monday, June 16 in Tokyo. Or tune in via live stream! Agenda: -Intro to vLLM -Japanese LLM adoption -Model optimization w/ LLM Compressor -Distributed inference w/ llm-d -Q&A and lightning talks RSVP: ossbyredhat.connpass.com/eve…
2
18
3
47
4,427
Tsai Li Ming retweeted
Blogger can get free haircut sponsorship one ah? How come nair tell me? I also Blogger! twitpic.com/dkz5wn
7
17
8
Tsai Li Ming retweeted
Dear Anonymous, don’t be too hao-lian. We have already put Batman Suparman behind our bars.
2
51
16
Tsai Li Ming retweeted
What do you call the offspring of a Royal Prince and a commoner? A: Half-Blood Prince
1
78
7