GreenBitAI (@GreenBitAI) header
GreenBitAI (@GreenBitAI) profile picture

We specialize in low-bit AI models and AI Agent software, delivering unparalleled efficiency and robust data privacy.

Germany
Joined March 2025
Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)Photo by GreenBitAI (@GreenBitAI)
Libra by GreenBitAI — a fully offline, local AI Agent OS. Powerful AI agents running 100% on your device. No cloud. No data leaving your hardware. See it live at #VivaTech2026 👇 @VivaTech #LocalAI #AIAgents #OnDeviceAI
1
250
A 113 GB model on a MacBook Pro with 48 GB of unified memory. In a GreenBitAI benchmark, gbx loaded parts of the model from SSD as needed and generated text at ~11 tokens/s. See how it works in our #electronica2026 recap. #GreenBitAI #LocalAI #EdgeAI #electronica2026
1
8
48
“Two fingers.” “Close hand.” “Rock sign.” 🤘 At electronica 2026 in Munich, this robotic hand responded to spoken commands using GreenBitAI’s voice model and inference engine. #electronica2026 #GreenBitAI #Robotics #RoboticHand #EdgeAI #VoiceControl
7
75
Behind the demos: GTA1000 voice AI. - ASR, language understanding & TTS fully on-chip - <700 ms response; data stays local - GreenBitAI 1–4 bit models; full pipeline in a few hundred MB - ~530 mW typical voice-interaction power - No internet or subscription fees web: gruenbit.cn
38
Day 1 at #electronica2026: multilingual TTS, a voice-controlled robotic hand and face-to-face exchanges at Giantec’s booth. GreenBitAI provides model customization and the inference engine for GTA1000’s on-device voice AI. Thanks to everyone who stopped by! 📅 Nov 10–13 | Hall C6 · Booth 421
1
8
119
How can large AI models run on devices with limited memory and power? Meet GreenBitAI at #electronica2026 and explore the GTA1000, featuring fully offline voice AI powered by 1–4 bit model compression. 📅 Nov 10–13 | Hall C6 · Booth 421 #EdgeAI #OnDeviceAI
1
1
8
65
Fish Audio’s Drama 3 TTS now runs locally on Apple silicon with gbx-lm 🎙️ • 2.6x real time on 64 GB+ Macs, still runs on a 48 GB MacBook • Zero-shot voice cloning, across languages • OpenAI-compatible streaming API huggingface.co/GreenBitAI/Dr… Built with Fish Audio
78
Ternary Bonsai 2 27B on gbx v0.8.2 ⚡ • 114 tok/s on M3 Ultra, 2.7× the official MLX runtime • 62 tok/s on DGX Spark, 2.1× llama.cpp • 27B in 5.9 GB of memory GreenBit Ternary Models, coming soon.
2
1
3
113
GreenBitAI retweeted
⚡ Inside DeepInfer 🔹 High-performance kernels & optimized inference engine 🔹 Distributed inference & speculative decoding 🔹 SWA Bounded Replay & structured outputs Inference code for future DeepSeek models and ongoing optimizations will land here too. 2/4
37
72
8
2,638
506,878
Why it's worth running locally: DeepSeek reports 77.0 on DeepSWE v1.1 (from 74.2) and 42.3 on Terminal-Bench 4.0 (from 31.2), with shorter outputs. An agentic coding model that now stays on your own machine.
1
53
DeepSeek's new V4.1-Flash-1017, running locally on a Mac. Our 4-bit build on one M3 Ultra Mac Studio: • ~22 tok/s decode • ~44 tok/s on code with DeepSeek's draft head (1.99×) • ~33 tok/s chat
1
4
3
107
Replying to @moelkholy95
Good math, and you're right that it's read-bound. Two corrections though. 32 GB is the process's peak footprint, not the expert cache. The cache itself is 15.7 GiB: 114 of 512 experts per layer (22%), sized automatically from the machine's memory. And ~11 tok/s is with the MTP draft head on, where one verify pass reads each expert once for several tokens. Plain paged decode at the same budget is 6–11 tok/s, depending on the prompt. Cache hit rate is 61–76% without MTP and 52–70% with it (the draft head takes ~11 slots per layer). Measured on a Studio at the same 25 GiB budget, cold reads. That leaves ~270 MB per token from SSD: about 70 ms waiting on disk and 60 ms of GPU work. On topic shifts: the cache is a per-layer LRU, so it follows routing within a few tokens. Our runs mix code, prose and chat on one server, and decode stays in a 9.6–12 tok/s band. We haven't measured a switch in the middle of an answer on its own. Good idea, we'll add it.
2
31
Full write-up with methodology: medium.com/@semamediadata/ru… Download: github.com/GreenBitAI/gbx-lm… Models: huggingface.co/collections/G… Which model would you run locally next?
26
How it works: • cost-aware adaptive speculative decoding • mixed-precision draft head • elastic expert memory that streams MoE experts from the SSD • fused kernels for MLA, linear attention, Hyper-Connections One shared layer: Qwen3.5/3.6/3.8, GLM-5.3, DeepSeek-V4.1-Flash.
1
49
vs Strata (SSD-streaming engine on an RTX 5070 PC, its published numbers), same model at ~4-bit, 16K prompt, cold SSD: • prompt: 591 vs 160 tok/s (3.7×) • decode: 12.6 vs 7.9 tok/s Different hardware, so not like-for-like. Full conditions in the write-up.
1
65
Not enough memory? Same 105 GB model, 25 GB budget, experts read cold from the SSD, vs omlx: • 7k prompt: 362 vs 123 tok/s (2.9×) • decode: 12.7–14.5 vs 8.3–9.7 tok/s • later chat turns: 13 s vs 27 s to first token mlx-lm and mlx-vlm can't load it at this budget.
1
62
On Qwen3.8-Flash-Next (4-bit, fully in memory) the lead grows: 53 vs 35 tok/s without speculative decoding, and +22–32% with it across code, prose and chat. Prompt processing is on par.
1
41
Plenty of memory? gbx decodes faster than the mainstream Mac engines. Qwen3.6-35B-A3B 8-bit, M3 Ultra, tok/s: gbx 120.5 omlx 93.3 Ollama 88.2 mlx-vlm 82.0 mlx-lm 76.3 With speculative decoding: gbx 176 vs omlx 156.
1
80