https://nitter.cf/t.co/YJ88R0IWCp | On-device foundation models. High intelligence, low memory, low power
San Francisco Bay Area
Joined September 2026
- Tweets92
- Following72
- Followers346
- Likes36
Pinned Tweet
๐ฌ ๐ ๐ฏ๐ฑ๐ ๐๐ ๐น๐ฎ๐ป๐ด๐๐ฎ๐ด๐ฒ ๐บ๐ผ๐ฑ๐ฒ๐น. ๐ฅ๐๐ป๐ป๐ถ๐ป๐ด ๐น๐ผ๐ฐ๐ฎ๐น๐น๐ ๐ผ๐ป ๐ฎ ๐ ๐ฎ๐ฐ.
This is ๐๐ฑ๐ด๐ฒ๐ฌ-๐ฏ๐ฑ๐ ๐ถ๐ป ๐ฎ๐ฐ๐๐ถ๐ผ๐ป in our macOS desktop app.
๐ป Edge0 keeps the quantized model weights on SSD and streams the expert weights it needs on demand, instead of loading the full 35B model into memory.
๐ In our macOS app benchmark, ๐๐ฑ๐ด๐ฒ๐ฌ-๐ฏ๐ฑ๐ ๐ฟ๐ฎ๐ป ๐ฎ๐ ๐ญ๐ฌโ๐ญ๐ฎ ๐๐ผ๐ธ/๐ ๐๐ถ๐๐ต ๐ฐ ๐๐ ๐ฟ๐ฒ๐๐ถ๐ฑ๐ฒ๐ป๐ ๐บ๐ฒ๐บ๐ผ๐ฟ๐ ๐ผ๐ป ๐ฎ๐ป ๐ ๐ฏ ๐ ๐ฎ๐ฐ๐๐ผ๐ผ๐ธ ๐๐ถ๐ฟ.
๐ Once the model is downloaded, inference runs ๐น๐ผ๐ฐ๐ฎ๐น๐น๐ ๐ฎ๐ป๐ฑ ๐ผ๐ณ๐ณ๐น๐ถ๐ป๐ฒ โ your prompts stay on your device, with no cloud API or per-token fees.
๐ฅ๏ธ And yes, thereโs an actual desktop app. ๐ก๐ผ ๐ฆ๐จ๐ซ๐ ๐๐ฟ๐ฒ๐๐๐น๐ถ๐ป๐ด ๐๐ถ๐๐ต ๐๐ฒ๐ป๐. Open the app, download a model, load it, and start chatting.
๐ข๐ฝ๐ฒ๐ป-๐๐ผ๐๐ฟ๐ฐ๐ฒ. ๐๐ฟ๐ฒ๐ฒ ๐๐ผ ๐๐ฟ๐.
๐ ๐ง๐ฟ๐ ๐ถ๐ ๐๐ผ๐๐ฟ๐๐ฒ๐น๐ณ
github.com/Edge0-AI/Edge0
๐ฌ๐๐ผ๐ถ๐ป ๐ผ๐๐ฟ ๐๐ถ๐๐ฐ๐ผ๐ฟ๐ฑ to get help with setup or share what youโre building:
discord.gg/Tg3W2FEuQ
Follow us for more Edge0 demos, benchmarks, and use cases.
Audio8-TTS-Realistic: realistic zero-shot voice cloning in 0.6B parameters, and is currently #1 on the Hugging Face TTS Arena.
Open weights, 23 languages, with fine-tuning scripts in the repo.
Hear it below ๐ง
Introducing Audio8-TTS-Realistic, @Edge0_AIโs new open-weight speech model: realistic, zero-shot voice cloning in just 0.6B parameters.
โข Human-like prosody and expressive delivery
โข Clones a voice from one short clip, in 23 languages
โข 601M params: small enough to fine-tune and serve affordably
Currently #1 on the Hugging Face TTS Arena, where people vote blind on which voice sounds more human.
Open weights are available now on @huggingface. Try it free.
Your quantized model fits in memory. It even loads successfully.
Then you send the first prompt, and the runtime crashes with an out-of-memory error.
Why does a model that fits on paper still run out of memory?
Because model weight size is not peak runtime memory. The runtime needs memory for much more than weights.
The useful question is not just โDoes my model fit?โ Itโs โWhen does memory usage peak?โ
When debugging an OOM, measure memory at different stages:
- Model loading: Are the weights expanded or copied?
- Prompt processing: How much workspace and attention memory is allocated?
- Generation: How much KV cache and persistent state is retained as the workload grows?
Also check context length, batch size, concurrency, and backend-specific temporary allocations.
And distinguish GPU allocations from system RAM, as well as reserved memory from actively used memory.
A successful model load only proves that loading succeeded. It does not establish that the runtime has enough memory for the workload that follows.
We built a model that almost certainly won't cause you to run out of memory: Edge0.
It's a 35B MoE model that can run on your phone (iOS and Android) and PC (both MacOS and Windows) that won't take up too much RAM.
In our Mac mini M4 Pro benchmark, Edge0-35B used 2.9 GiB of peak MLX allocator memory. Native runtimes are available for supported iOS, Android, macOS, and Windows devices.
Try Edge0 on GitHub: github.com/Edge0-AI/Edge0
Model info: huggingface.co/Edge0/Edge0-3โฆ
Join our Discord to get help with setup or share what youโre building: discord.gg/Tg3W2FEuQ
1/ your phone can run an ai model without sending every prompt to a server.
that's on-device ai. the model runs on your phone or laptop, and can work offline.
but your device now has to do the computing itself.
5/ ๐๐๐ผ๐ฟ๐ฎ๐ด๐ฒ ๐ถ๐ ๐๐น๐ผ๐๐ฒ๐ฟ ๐๐ต๐ฎ๐ป ๐ฟ๐ฎ๐บ, ๐๐ต๐ผ๐๐ด๐ต.
so a prerouter predicts which experts we'll need one token ahead. that lets us fetch weights while the model is still computing.
recover-lora helps restore quality lost to quantization and predicted routing.
6/ on an iphone 18 pro max, our ios build runs edge0-35b at 20 ๐๐ผ๐ธ๐ฒ๐ป๐/๐, with 1s to first token.
the repacked app uses ~20 gb of storage. battery, heat, and storage speed still matter. on-device isn't magic.
7/ ๐ฐ๐๐ฟ๐ถ๐ผ๐๐ ๐๐ผ ๐๐ฟ๐ ๐ถ๐?
code: github.com/Edge0-AI/Edge0
model: huggingface.co/Edge0/Edge0-3โฆ
setup help: edge0.ai/discord
Edge0-35B averages ๐๐.๐ ๐ฎ๐ฐ๐ฟ๐ผ๐๐ ๐ณ๐ถ๐๐ฒ ๐ข๐ฝ๐ฒ๐ป๐๐ผ๐บ๐ฝ๐ฎ๐๐ ๐ฏ๐ฒ๐ป๐ฐ๐ต๐บ๐ฎ๐ฟ๐ธ๐, compared with 83.2 for its fp16 base.
๐๐ผ๐ ๐ฑ๐ผ ๐๐ฒ ๐ฟ๐ฒ๐ฐ๐ผ๐๐ฒ๐ฟ ๐พ๐๐ฎ๐น๐ถ๐๐ ๐๐ถ๐๐ต๐ผ๐๐ ๐ฟ๐ฒ๐๐ฟ๐ฎ๐ถ๐ป๐ถ๐ป๐ด ๐๐ต๐ฒ ๐ณ๐๐น๐น ๐บ๐ผ๐ฑ๐ฒ๐น?
๐ง๐ต๐ฒ ๐ฎ๐ป๐๐๐ฒ๐ฟ: ๐ฅ๐ฒ๐ฐ๐ผ๐๐ฒ๐ฟ-๐๐ผ๐ฅ๐, ๐ฎ ๐๐บ๐ฎ๐น๐น ๐ฎ๐ฑ๐ฎ๐ฝ๐๐ฒ๐ฟ ๐๐ฟ๐ฎ๐ถ๐ป๐ฒ๐ฑ ๐๐ป๐ฑ๐ฒ๐ฟ ๐ฝ๐ฟ๐ฒ๐ฑ๐ถ๐ฐ๐๐ฒ๐ฑ ๐ฟ๐ผ๐๐๐ถ๐ป๐ด ๐ฎ๐ป๐ฑ ๐ธ๐ฒ๐ฝ๐ ๐๐ฒ๐ฝ๐ฎ๐ฟ๐ฎ๐๐ฒ ๐ณ๐ฟ๐ผ๐บ ๐๐ต๐ฒ ๐ฏ๐ฎ๐๐ฒ. ๐ฆ๐๐ฒ๐ฝ ๐ ๐งต
1/6
๐ง๐ต๐ฒ ๐ฐ๐ผ๐บ๐ฝ๐น๐ฒ๐๐ฒ ๐๐๐ ๐ฝ๐ถ๐ฝ๐ฒ๐น๐ถ๐ป๐ฒ ๐ฎ๐๐ฒ๐ฟ๐ฎ๐ด๐ฒ๐ ๐ฎ๐ฏ๐ผ๐๐ ๐ณ๐ผ๐๐ฟ ๐ฝ๐ผ๐ถ๐ป๐๐ ๐ฏ๐ฒ๐น๐ผ๐ ๐ณ๐ฝ๐๐. This comparison includes int4 weights, predicted routing, and the recovery adapter, evaluated with identical OpenCompass settings on a compute server.
Edge0 / fp16 โ AIME 2026: ๐๐.๐ / ๐๐.๐; HumanEval: ๐๐.๐ / ๐๐.๐; GPQA-Diamond: ๐๐.๐ / ๐๐.๐; MMLU-Pro: ๐๐.๐ / ๐๐.๐; IFBench: ๐๐.๐ / ๐๐.๐. Average: ๐๐.๐ / ๐๐.๐.
The largest gap is in long-chain reasoning: AIME is ๐.๐ ๐ฝ๐ผ๐ถ๐ป๐๐ ๐น๐ผ๐๐ฒ๐ฟ for the 35B model and ๐๐.๐ ๐ฝ๐ผ๐ถ๐ป๐๐ ๐น๐ผ๐๐ฒ๐ฟ for the 8B model.
5/6
๐ง๐ฟ๐ ๐๐ฑ๐ด๐ฒ๐ฌ ๐ผ๐ป ๐๐ถ๐๐๐๐ฏ: github.com/Edge0-AI/Edge0
๐ ๐ผ๐ฑ๐ฒ๐น ๐ถ๐ป๐ณ๐ผ: huggingface.co/Edge0/Edge0-3โฆ
๐๐ผ๐ถ๐ป ๐ผ๐๐ฟ ๐๐ถ๐๐ฐ๐ผ๐ฟ๐ฑ to get help with setup or share what youโre building: discord.gg/Tg3W2FEuQ
6/6