Senior software engineer Automation + applied AI (pharma) Contributor: MLX-Serve, TensorFold, ds4 Local inference · Real hardware · Builders
Joined September 2026
- Tweets1.1K
- Following480
- Followers2.2K
- Likes7.1K
Pinned Tweet
Building automation and applied AI systems for pharma. Contributing to local inference projects (MLX-Serve and TensorFold).
Obsessed with making models run faster and more reliably on real hardware.
Tinkerers and builders — what are you working on? I’d love to connect.
Everyone's best argument for 4x Spark over a Mac was prefill. oMLX just paired an M5 Max with an M3 Ultra and took DeepSeek V4.1 Flash from 1,060 to 3,123 tok/s at 64K. No new silicon. Told you software catches up. nitter.cf/jundotkim/status/21179…
oMLX 0.7.4.dev1 is out!
github.com/jundot/omlx/relea…
With the new Prefill Accelerator, connect an M5 Max to your big Mac and prefill up to 2.9x faster!
Prefill Accelerator (64K prompt, M3 Ultra 512GB + M5 Max 128GB)
- DeepSeek V4.1 Flash: 1,060 -> 3,123 tok/s (+195%)
- Qwen3.8 Flash-Next: 1,309 -> 3,666 tok/s (+180%)
- GLM-5.3 Flash: 540 -> 1,388 tok/s (+157%)
- MiniMax M3.1: 297 -> 569 tok/s (+92%)
- An M5 Max 64GB still adds 30-118%, and decode speed stays the same.
Faster DeepSeek V4.1 with A8 and buffer pool improvements
- Prefill on M5 Ultra is 43-48% faster, 32K goes 2,631 -> 3,779 tok/s (+44%).
- oQ4e builds now run INT8 Activation Prefill on M5.
Faster Qwen3.8 Flash-Next
- FP16 builds get the fused kernels. Decode on M3 Ultra goes 38.9 -> 63.0 tok/s (+62%), and +23% on M2 Max.
- Decode after long prompts is 3-5% faster on M5 Max.
Better for agents
- Parallel tool results from Claude Code reach the model in call order, so Qwen no longer pairs a result with the wrong call.
- Agent sessions on a cluster reuse the prompt cache across tool turns, 743 -> 9,266 cached tokens by the fifth turn.
- Switch oMLX models inside Codex and Copilot with /model.
Also in 0.7.4.dev1
- Ming-3.0 TTS with a 2x faster decode loop.
- Ask AI can use an oMLX server on another Mac.
- Check for model updates in the macOS app.
- German across the web dashboard and the macOS app.
- Batched decode with 6 requests on 8-bit models is 16% faster.
A big thank you to everyone who sent PRs, reported bugs and tested builds for this release, and a warm welcome to the five new contributors!
Two Macs, one Thunderbolt 5 cable, tensor parallel DeepSeek V4.1 Flash. 88 tok/s decode vs 56 on a single M3 Ultra, prefill up 58%. Everyone asking "4x Spark or Mac?" just got a new answer. Mac clustering stopped being a meme. nitter.cf/ivanfioravanti/status/…
My upcoming mlx-serve PR adding support for DeepSeek-V4.1 Flash with tensor parallelism across two Macs over Thunderbolt 5 RDMA. 🔥
Tested on 2× M3 Ultra vs 1× using Lossless weights and llmprobe:
• DSpark decode 88 tok/s (vs 56), 116 on predictable text
• Plain decode 54 tok/s (vs 41)
• Prefill 1,608 tok/s (vs 1,018)
• 4 concurrent streams 94 tok/s total (vs 62)
Yes it's faster than DwarfStar! Evals in progress!
PR Draft here super @ddalcu github.com/ddalcu/mlx-serve/…
People keep pricing an M5 Ultra against a stack of DGX Sparks like the race is already over. It isn't. Most local inference stacks barely touch what Apple Silicon already has. You're comparing mature CUDA kernels to software that's still catching up, and software catches up.
Sebastian Christiansen retweeted
£24,000 raised in less than 24 hours. I still can't quite believe it. 🤯
I started TensorFold in February 2026 and archived it three times because I couldn't get the idea to work. I worked on MCDMA in between, then finally cracked it and released TensorFold with Qwen 3.8 27B.
I thought a few people might use it. Maybe I'd get the odd GitHub star.
When the 512GB M5 Ultra Mac Studio became available, I ordered one. Then my loan fell through because the bank wouldn't accept my X income as regular income.
I thought I'd have to cancel and admit defeat, but you guys didn't.
More than two dozen of you encouraged me to set up a fundraiser, so I did, and I hoped it might make the cost manageable.
You covered the entire cost of the Studio, with more donated directly to fund even more hardware. 🥹
To everyone who's donated, subscribed, shared the fundraiser, contributed code, tested builds or let me use their machines, thank you. 🫶🏼
The support has been beyond anything I could have imagined. ❤️
I can't wait to get the Studio here, push TensorFold further and share the results with you.
You've made something possible I couldn't have done alone. Thank you, from the bottom of my heart.
This community rocks!!!!!
Sebastian Christiansen retweeted
Hot take: @ashxhart is doing incredible work for the benefit of anyone running local AI on a Mac. The local AI community funded him a 512GB Mac Ultra in less than 24 hour! What a great group! 🚀🚀🚀
But it's totally embarrassing for @Apple that he has to run a donation campaign for $30,000 to make AI on Apple Silicon better for a company worth (checks stock page...) 4.855 TRILLION DOLLARS. C'mon @johnternus at least send him a t-shirt or a MacBook or something, or move his order up in the queue! 🤦♂️
£24,000 raised in less than 24 hours. I still can't quite believe it. 🤯
I started TensorFold in February 2026 and archived it three times because I couldn't get the idea to work. I worked on MCDMA in between, then finally cracked it and released TensorFold with Qwen 3.8 27B.
I thought a few people might use it. Maybe I'd get the odd GitHub star.
When the 512GB M5 Ultra Mac Studio became available, I ordered one. Then my loan fell through because the bank wouldn't accept my X income as regular income.
I thought I'd have to cancel and admit defeat, but you guys didn't.
More than two dozen of you encouraged me to set up a fundraiser, so I did, and I hoped it might make the cost manageable.
You covered the entire cost of the Studio, with more donated directly to fund even more hardware. 🥹
To everyone who's donated, subscribed, shared the fundraiser, contributed code, tested builds or let me use their machines, thank you. 🫶🏼
The support has been beyond anything I could have imagined. ❤️
I can't wait to get the Studio here, push TensorFold further and share the results with you.
You've made something possible I couldn't have done alone. Thank you, from the bottom of my heart.
This community rocks!!!!!
Sebastian Christiansen retweeted
Introducing Zero Wire: a public vulnerability intelligence feed on @CloudflareRadar, bringing the clarity of a news wire service to critical security developments. cfl.re/4iCvSbj
So proud of this community! 👏🎉
We did it! Local AI community is so powerful!
Less than 24 hours to help @ashxhart to get an M5 Ultra 512GB to push TensorFold to the next level!
WE ARE THE OPEN AI! 💪
I’m not a rich man by any means, but I believe in leading by example, so I just chipped in! I urge anyone able to do the same. 🤘
ko-fi.com/ashxhart
Sebastian Christiansen retweeted
Following my earlier post about the M5 Ultra 512GB Mac Studio, more than two dozen people asked me to set up crowdfunding for it.
I always try to pay my own way, so asking for help at this scale doesn't come naturally. But I couldn't make the numbers work this time, and my loan got rejected after placing the order.
I'd love to get one for TensorFold and to see how far we can push it.
If you'd like to help towards it, I've put the goal on Ko-fi below. Please only contribute if you're comfortable doing so.
ko-fi.com/ashxhart/goal?g=2
Thank you to everyone who's encouraged me to do this, and to everyone already supporting TensorFold. I'm incredibly grateful and will continue to push TF. ❤️
@TheAhmadOsman @0xSero @ivanfioravanti
Ash makes TensorFold for free but got turned down for a personal loan so can't buy a Mac M5 512GB to do more work on.
What do you guys think? You'd help to promote a crowd fund, wouldn't you?
And if you want to support the developer directly you can do so here
buymeacoffee.com/ashhart
“Still too accessible to JavaScript developers.” 😂😭
We have used Bend2 for the last few weeks. Here is an honest review:
Bend2 is what happens when an inbred type theory autist decides Haskell is for fucking casuals. No type inference. No type classes. No tactics. Want to prove something? Write the fucking proof yourself. Dependent types with absolutely none of the ergonomic bullshit. Also, affine types because apparently full linearity would be too much commitment. The ecosystem is nonexistent, the learning curve is a vertical cliff, and writing trivial programs feels like defending a PhD thesis.
9/10. Still too accessible to JavaScript developers.
scikit-learn API, running on Apple Silicon GPUs through MLX and Metal. mlx-learn claims up to 30x faster training and 300x faster inference with matching results. The Mac story was never only about LLMs. Classical ML just got its own local fast path.
A bank just told the dev behind TensorFold that subs and X income aren't "reliable income", so no loan for a 512GB M5 Ultra. A lot of local Apple Silicon inference runs on one person's credit score. We call it open source. It's really unfunded infrastructure.
Sebastian Christiansen retweeted
@johnternus @AppleSupport @Apple
TensorFold support for Apple devices is doing heavy lifting for Apple’s competitive credibility in AI. Sales are being driven directly because of Ash's support.
Does anyone know Apple dev relations that can organise getting HW into Ash's hands?