Run open LLMs locally on Apple Silicon — fast, private, Apache-2.0. A Mac app or a one-line install.
San Francisco
Joined February 2021
- Tweets137
- Following9
- Followers9.2K
- Likes212
Pinned Tweet
We made local AI on your Mac up to 4× faster. 🐆 Rapid-MLX 1.0 is here.
Run OpenCode, Claude Code, Codex and your own apps on a local model. One command.
• 240+ open models
• OpenAI + Anthropic compatible API
• 4,000+ ⭐, 48 contributors, Apache 2.0
• 400K+ Hugging Face downloads
What's new 👇
Under the hood, the 1.0 server is built to take a beating:
• Out of memory? It recovers without a restart
• Long-prompt bursts queue instead of crashing
• 4 concurrent chats: 2× aggregate throughput
• Safer by default: blocks drive-by browser access and DNS rebinding
Up and running in one line:
curl -fsSL rapidmlx.com/install.sh | bash
Then plug in your agent: rapid-mlx launch opencode
Prefer an app? Rapid-MLX Desktop is free: rapidmlx.com/desktop
Like it? A ⭐ on GitHub cheers up our 48 contributors:
github.com/raullenchai/Rapid…
The community has been hyping up LFM3, so we put it to the test.
We benchmarked @LiquidAI_’s LFM3-VL on a two-generation-old M2 Pro Mac mini, and the results speak for themselves:
⚡ ~2s TTFT (full photo)
⚡ ~75 tok/s
🔒 Fully private
Landing in the next Rapid-MLX release. Stay tuned. 🐆
We used our proprietary automated hardware-in-the-loop architecture search to design LFM3’s architecture for agentic workloads, targeting more capability per parameter and fast inference over long contexts.
For LFM2, the search focused on CPUs on phones and laptops. For LFM3, we expanded to 14 device and backend combinations: phones (Galaxy S26 Ultra and OnePlus 15), laptops (Apple M4 Max and @AMD Ryzen AI), a Raspberry Pi 5, and @Arm and x86 cloud CPUs, plus GPUs and the @Qualcomm Hexagon NPU.
The search evaluates model quality alongside measured inference speed and memory use on target hardware. This makes deployment constraints part of architecture selection, optimizing for stronger capabilities within the latency and memory budgets of the devices running your agents.
3/
🚀 Meet your new fully local AI coding agent.
💻 With a single command
rapid-mlx launch codex --model flint-40b-a2.5b-4bit
it instantly loads a 40B model and fires up Codex.
⚡️ Watch it build and test a Game of Life in 26 seconds flat with zero API keys, zero cloud dependency, and absolutely nothing leaving your Mac.
🐆 Try it at rapidmlx.com.
Rapid-MLX 0.16.9 is out 🐆
⚡ 47× faster first token on repeated long prompts. A ~32K-token prompt that took 18.6 s to start now starts in 0.39 s.
🔥 Flint 40B runs on a 36 GB Mac at 131 tok/s, with tool calls and reasoning built in.
👉 rapid-mlx serve flint-40b-a2.5b-4bit
👥 Parallel chats decode together: 2× aggregate throughput on Qwen3.8-27B.
👉 rapid-mlx serve qwen3.8-27b-4bit
🧠 New: Nemotron 3.5 Bolt E4B, with tool calls and thinking controls.
👉 rapid-mlx serve nemotron-3.5-bolt-e4b
🛠 One command to connect your coding agent, offered as soon as the server is ready.
👉 rapid-mlx launch codex
Give it a try: rapidmlx.com/download
👀👀
Available now: NVIDIA Nemotron 3.5 Bolt E4B 4GB ⚡️
Our newest small open model is built for local assistants, interactive characters and robots that can carry on a conversation and use tools to get things done.
Watch Reachy Mini chat and move, with Bolt handling conversation and NVIDIA Cosmos handling vision. Both models run locally on Jetson Orin Nano 8GB.
Learn more: research.nvidia.com/labs/nem…
Round 2 of the cheetah bicycle test: Qwen3.6-35B-A3B.
68 seconds. Sun, clouds, grass, real spinning wheels, and a "ROAD RAGER" sign. The cheetah's tail is longer than the bike, and it is very clearly late for something. 🐆🚲
94 tok/s, 100% local on a Mac mini with Rapid-MLX. Unedited SVG.
The pelican had its moment. Now, behold the cheetah. 🐆🚲
Flint’s masterpiece: easy-rider handlebars, a very determined spotted torso, and furious pedaling while the bike politely refuses to obey the laws of physics. 😅
40B params, 2.5B active, blazing at 67 tok/s on an M4 Pro Mac mini. Raw & unedited. Judge accordingly.
The pelican had its moment. Now, behold the cheetah. 🐆🚲
Flint’s masterpiece: easy-rider handlebars, a very determined spotted torso, and furious pedaling while the bike politely refuses to obey the laws of physics. 😅
40B params, 2.5B active, blazing at 67 tok/s on an M4 Pro Mac mini. Raw & unedited. Judge accordingly.
v0.16.8 is out: Flint is now in a Rapid-MLX release. 🚀
pip install -U rapid-mlx
rapid-mlx serve flint-40b-a2.5b-4bit
No more building from main. The alias is pinned to the validated checkpoint, so you get the exact weights we benchmarked.
Also in 0.16.8:
⚡ Concurrent MTP chats decode together: 4 parallel Qwen3.8-27B requests went from 35 → 75 tok/s
🖥️ Desktop shows model loading right in the chat composer
🔔 Running servers notice new releases without a restart
Release notes: github.com/raullenchai/Rapid…
Download: rapidmlx.com/download
v0.16.8 is out: Flint is now in a Rapid-MLX release. 🚀
pip install -U rapid-mlx
rapid-mlx serve flint-40b-a2.5b-4bit
No more building from main. The alias is pinned to the validated checkpoint, so you get the exact weights we benchmarked.
Also in 0.16.8:
⚡ Concurrent MTP chats decode together: 4 parallel Qwen3.8-27B requests went from 35 → 75 tok/s
🖥️ Desktop shows model loading right in the chat composer
🔔 Running servers notice new releases without a restart
Release notes: github.com/raullenchai/Rapid…
Download: rapidmlx.com/download
Sneak peek: Flint from @Recursive_SI now runs in Rapid-MLX. 🔥
Measured on a 36 GB M5 Max with 4-bit weights, greedy, 256 output tokens:
⚡ ~1K context: 131 tok/s decode, 0.35 s TTFT
📄 ~8K context: 118 tok/s decode, 2.2 s TTFT
👥 4 concurrent requests: 193 tok/s aggregate
♻️ Prefix cache: repeat-prompt TTFT 0.46 s → 0.15 s
💾 Peak memory: ~24 GB
Tool calls, streaming and 8K+ context all work out of the box with an OpenAI-compatible API.
Ships in the next release. Can't wait? Build from main:
git clone github.com/raullenchai/Rapid… && cd Rapid-MLX
pip install .
rapid-mlx serve flint-40b-a2.5b-4bit
We are incredibly proud to be invited to the @SGLang Summit—a premier gathering for frontier AI! 🎉
Catch talks from AI leaders like Lilian Weng, Soumith Chintala, and Aravind Srinivas in SF (Nov 12–13).
Want to join us in SF? 🎟️ Rapid-MLX members get a HUGE discount on tickets! Hop into our Discord to claim yours: discord.gg/nZcXkUjY5R
See you there 👋
Announcing the first SGLang Summit: November 12–13 at Fort Mason, San Francisco.
Join us for two days of talks, discussions, and deep dives into frontier AI infrastructure, silicon, models, and applications.
Register: sglang.io/summit
Sneak peek: Flint from @Recursive_SI now runs in Rapid-MLX. 🔥
Measured on a 36 GB M5 Max with 4-bit weights, greedy, 256 output tokens:
⚡ ~1K context: 131 tok/s decode, 0.35 s TTFT
📄 ~8K context: 118 tok/s decode, 2.2 s TTFT
👥 4 concurrent requests: 193 tok/s aggregate
♻️ Prefix cache: repeat-prompt TTFT 0.46 s → 0.15 s
💾 Peak memory: ~24 GB
Tool calls, streaming and 8K+ context all work out of the box with an OpenAI-compatible API.
Ships in the next release. Can't wait? Build from main:
git clone github.com/raullenchai/Rapid… && cd Rapid-MLX
pip install .
rapid-mlx serve flint-40b-a2.5b-4bit
Introducing Flint, @Recursive_SI’s first model: a fast local coding model built with our automated research system, the Eureka Machine.
• Best-in-class fast local coding
• Up to 2x faster than comparable models: 125+ tokens/sec on a MacBook Pro, 150+ on RTX Spark
• Built from scratch in under 4 months
Private, unmetered intelligence on your own machine. Open weights available now on @huggingface, run it locally on your favorite framework or try it for free on @opencode or @pidotdev.
Rapid-MLX 0.16.7 is out: DeepSeek V4.1 Flash runs up to ~2x faster on a Mac.
Our quant of DeepSeek-V4.1-Flash-1017 (GPTQ v2) now decodes at 33–45 tok/s single-stream on an M3 Ultra.
Same weights, faster engine: lighter drafting, fused mixed-bit MoE, and a vocab head read once per verify round. Every token is still verified by the full model.
pip install -U rapid-mlx
rapid-mlx serve deepseek-v41-flash-1017
Try it out: rapidmlx.com/download
Running DeepSeek-V4.1-Flash locally shouldn't mean sacrificing quality.
Meet the most faithful MLX quantization of DeepSeek-V4.1-Flash-1017. We built the ultimate version for 256GB+ Macs, maximizing both quality and speed.
👑 The New Standard for MLX:
✅ Highest Quality: 35–44% lower Mean KL. Crushes the leading 2-bit MLX quant (85.1% vs 79.3% top-1).
✅ No Compromises: ZERO experts pruned. All 40x384 kept.
✅ Advanced Tech: Sequential quantization, sensitivity-driven mixed precision, and Engram tables streamed from SSD.
✅ Fastest Path: A major high-speed inference release is imminent.
The best local DeepSeek-V4.1-Flash-1017 on Mac is ready.
🔗 Weights & comparisons: huggingface.co/rapid-mlx/Dee…
Running DeepSeek-V4.1-Flash locally shouldn't mean sacrificing quality.
Meet the most faithful MLX quantization of DeepSeek-V4.1-Flash-1017. We built the ultimate version for 256GB+ Macs, maximizing both quality and speed.
👑 The New Standard for MLX:
✅ Highest Quality: 35–44% lower Mean KL. Crushes the leading 2-bit MLX quant (85.1% vs 79.3% top-1).
✅ No Compromises: ZERO experts pruned. All 40x384 kept.
✅ Advanced Tech: Sequential quantization, sensitivity-driven mixed precision, and Engram tables streamed from SSD.
✅ Fastest Path: A major high-speed inference release is imminent.
The best local DeepSeek-V4.1-Flash-1017 on Mac is ready.
🔗 Weights & comparisons: huggingface.co/rapid-mlx/Dee…
DeepSeek V4.1 Flash 1017 scores 45 on the Artificial Analysis Intelligence Index, improving on DeepSeek V4.1 Flash by 6 points, and is the equal #2 open-weights model
@deepseek_ai has released DeepSeek V4.1 Flash 1017, an update on their flagship model DeepSeek V4.1 Flash. On the first-party API, DeepSeek V4.1 Flash 1017 is priced at $0.30 per 1M input tokens and $1.20 per 1M output tokens during peak hours, with off-peak pricing half of peak rates, the same pricing as DeepSeek V4.1 Flash.
Key results:
➤ DeepSeek V4.1 Flash 1017 makes gains in agentic capabilities, knowledge and scientific reasoning. DeepSeek V4.1 Flash 1017’s Elo rating on GDPval-AA v2.1 is 1746, an increase of 146 points from DeepSeek V4.1 Flash’s rating of 1600. On AA-Briefcase v1.1, its Elo rating increases by 150 points from 1420 to 1570. On AA-Omniscience, its non-hallucination rate is 33%, a 29-point improvement from DeepSeek V4.1 Flash’s 4%, while seeing an improvement in accuracy from 43% to 46%, pushing its overall AA-Omniscience Index score from -5 to +5. On CritPt, DeepSeek V4.1 Flash 1017 scores 24%, up from 14% for DeepSeek V4.1 Flash.
➤ DeepSeek V4.1 Flash 1017 is now the #2 open-weights model alongside GLM-5.3. With a score of 45 on the Artificial Analysis Intelligence Index, DeepSeek V4.1 Flash 1017 is behind MiMo-V2.6-Pro by only 1 point and ahead of other open-source models like Step 5 Preview and Kimi K3 (both at 44).
➤ With a Cost per Task of $0.26, DeepSeek V4.1 Flash 1017 sits in the most attractive quadrant of Capability vs. Cost. With a Cost per Task of $0.26 and Intelligence Index score of 45, DeepSeek V4.1 Flash 1017 compares favorably to GLM-5.3-Flash at $0.28 per task and a score of 42. At the same time, MiMo-V2.6-Pro (46) at $0.13 per task and GPT-6.1 Sol (medium, 48) at $0.21 per task both offer higher capability at a lower Cost per Task.
Additional model details:
➤ Context window: 1M tokens (equivalent to DeepSeek V4.1 Flash)
➤ Pricing: $0.30 per 1M input tokens and $1.20 per 1M output tokens, unchanged from DeepSeek V4.1 Flash. Cache hit price of $0.006 per 1M tokens, a 98% discount
➤ Size: 552B total parameters (16B active)
➤ License: MIT
➤ Providers: Available through DeepSeek’s first-party API, @FireworksAI_HQ , @baseten, @DeepInfra , @wafer_ai , @Cloudflare and others
Rapid-MLX · local AI on your Mac retweeted
🚀 Trio-Spark DI (9B) just hit the Top 10 on the Decision Index for models ≤9B!
Built for bulletproof agent reliability in the real world, not benchmark inflation:
✅ 0 errors on 110k+ requests
✅ 97.04% independent match rate
✅ Zero text generation overhead
Turn action selection into a single, perfectly calibrated forward pass. Live now as a gated 9B GGUF (CC BY-NC 4.0).
📊 Board: huggingface.co/spaces/multim…
📦 Weights: huggingface.co/MachineFi/Tri…
Trio-Spark v1.3 is live ⚡
The SOTA System-1 decision model for autonomous agents.
Most agent pipelines burn compute generating text just to pick an action. Spark turns action selection into a single forward pass: zero generated tokens, perfectly calibrated confidence.
🏆 SOTA Performance Across 13 Benchmarks:
• 90.9% on the ten-task panel (+7.8% over v1.0, overtaking Jev 1.13 and level with Clef)
• #1 among all 7 models on PubMedQA, RTE, TweetEval, and SNLI
• Calibration error crushed to 0.019 (vs 0.234 in v1.0) — true confidence you can actually threshold
Upgrade is drop-in: same API, identical limits, text & multimodal vision support. Just point to "model": "trio-spark-v1.3".
⚡ Get 100 free decisions: platform.machinefi.com/spark
📖 Full benchmark report & transparency note:
machinefi.com/blog/trio-spar…
🚀 Rapid-MLX 0.16.5 is LIVE: Turn your Mac into a top-tier AI compute center!
🌟 DeepSeek-V4.1-Flash-1017-MLX
Run the frontier model locally on a 256GB Apple Silicon Mac—without compromising its soul:
• 🔪 ZERO experts removed: We kept the full 40×384 architecture. No brutal pruning.
• ⚖️ Smart Quantization: Extreme mixed-precision (2/3/4-bit for experts, while crucial routing & attention stay at 8-bit/BF16).
• 📉 Near-Lossless Quality: We crushed the KL divergence down to 0.439 (compared to a catastrophic 2.02 on standard 2-bit). Code, math, and agentic capabilities are preserved!
• ⚡ Speed: ~20 tok/s on an M3 Ultra (206.5GB resident RAM).
🔥 Also dropping today:
• 👁️ MiniCPM-V 4.7: Vision + Tool Calling! Reads screenshots, charts, and photos flawlessly. Available in a tiny 1B for laptops or 35B-A3B for deep reasoning.
• ⚡ LFM2.5-24B-A2B (Liquid AI): 24B parameters of knowledge, but only ~2B active per token. Blazing fast on everyday Macs with out-of-the-box tool calling.
Stop paying for expensive cloud APIs and waiting in queues. Your Mac is all you need. 👇
💻 pip install -U rapid-mlx
🚀 rapid-mlx serve deepseek-v41-flash-1017
Learn more: rapidmlx.com
DeepSeek V4.1 Flash 1017 scores 45 on the Artificial Analysis Intelligence Index, improving on DeepSeek V4.1 Flash by 6 points, and is the equal #2 open-weights model
@deepseek_ai has released DeepSeek V4.1 Flash 1017, an update on their flagship model DeepSeek V4.1 Flash. On the first-party API, DeepSeek V4.1 Flash 1017 is priced at $0.30 per 1M input tokens and $1.20 per 1M output tokens during peak hours, with off-peak pricing half of peak rates, the same pricing as DeepSeek V4.1 Flash.
Key results:
➤ DeepSeek V4.1 Flash 1017 makes gains in agentic capabilities, knowledge and scientific reasoning. DeepSeek V4.1 Flash 1017’s Elo rating on GDPval-AA v2.1 is 1746, an increase of 146 points from DeepSeek V4.1 Flash’s rating of 1600. On AA-Briefcase v1.1, its Elo rating increases by 150 points from 1420 to 1570. On AA-Omniscience, its non-hallucination rate is 33%, a 29-point improvement from DeepSeek V4.1 Flash’s 4%, while seeing an improvement in accuracy from 43% to 46%, pushing its overall AA-Omniscience Index score from -5 to +5. On CritPt, DeepSeek V4.1 Flash 1017 scores 24%, up from 14% for DeepSeek V4.1 Flash.
➤ DeepSeek V4.1 Flash 1017 is now the #2 open-weights model alongside GLM-5.3. With a score of 45 on the Artificial Analysis Intelligence Index, DeepSeek V4.1 Flash 1017 is behind MiMo-V2.6-Pro by only 1 point and ahead of other open-source models like Step 5 Preview and Kimi K3 (both at 44).
➤ With a Cost per Task of $0.26, DeepSeek V4.1 Flash 1017 sits in the most attractive quadrant of Capability vs. Cost. With a Cost per Task of $0.26 and Intelligence Index score of 45, DeepSeek V4.1 Flash 1017 compares favorably to GLM-5.3-Flash at $0.28 per task and a score of 42. At the same time, MiMo-V2.6-Pro (46) at $0.13 per task and GPT-6.1 Sol (medium, 48) at $0.21 per task both offer higher capability at a lower Cost per Task.
Additional model details:
➤ Context window: 1M tokens (equivalent to DeepSeek V4.1 Flash)
➤ Pricing: $0.30 per 1M input tokens and $1.20 per 1M output tokens, unchanged from DeepSeek V4.1 Flash. Cache hit price of $0.006 per 1M tokens, a 98% discount
➤ Size: 552B total parameters (16B active)
➤ License: MIT
➤ Providers: Available through DeepSeek’s first-party API, @FireworksAI_HQ , @baseten, @DeepInfra , @wafer_ai , @Cloudflare and others
This is exactly where local AI shines—Rapid-MLX can easily serve as the on-device brain for hardware like this.
Six months ago, a friend at @seeedstudio handed me this $50 ESP32 AI gadget. I tossed it in a drawer and forgot about it.
@Meta dropped the open-source Muse SDK last week. I dusted the device off, but it still felt restricted to just one "brain." I wanted total control, better privacy, and the freedom to use Claude, Astra, or my own local models.
So I gutted the software and rebuilt it. The result works so well that I decided to open-source the project—turning this dusty gadget into a lightning-fast, multi-model personal assistant. Here is why it actually feels alive:
🗣️ The Ultimate Polyglot: Speak to it in English, Español, 中文, 日本語, or 한국어. It instantly auto-detects your language and replies in the exact same tongue, picking the best voice model automatically.
🧠 Bring Your Own Brain: Plug in ANY OpenAI-compatible API. You aren't locked in anymore. Swap between Claude, GPT, or local models with a 1-line preset.
⚡️ No More Awkward Pauses: We slashed the latency from a clunky 10s+ down to ~3s. It literally starts talking to you while the LLM is still generating the text.
🛑 True Conversationalist: You can interrupt it mid-sentence without breaking its brain (true barge-in), and we added silence guards so it never hallucinates from background room noise.
All the heavy lifting (TLS optimizations, audio compression, crash-recovery) is handled under the hood.
Grab your dusty ESP32 toys and give them a real brain:
🔗 github.com/raullenchai/esp32…