🚨 €135 FOR A COMPLETE LOCAL AI COMPUTER. NOT JUST THE GPU. THE ENTIRE MACHINE! 🤯
And the story is even crazier.
🖥️ HP Z220, Xeon E3-1270 v2, 32GB DDR3 — €50. Bought 5 years ago. Collecting dust in my garage ever since.
🔥 AMD RX 580 2048SP, 8GB VRAM — €49 delivered this week.
💾 256GB NVMe — €30 delivered this week.
🔧 PCIe-to-NVMe adapter — €6 on Amazon.
TOTAL: €135!
We installed Linux, built llama.cpp b12318 with Vulkan, and benchmarked five quantized AI models.
📊 REAL MEASURED RESULTS
Prefill / Decode (tokens/sec):
• Qwen2.5 3B Q4_K_M — 397 / 56.8
• Nemotron 3 Nano 4B Q4_K_M — 270 / 39.1
• Nemotron 3.5 Bolt E4B Q4_K_XL — 441 / 29.3
• Ternary Bonsai 2 9B Q2_0 — 172 / 26.9
• Ternary Bonsai 2 27B Q2_0 — 48 / 9.3
Yes, even a 27B model runs on this €135 machine! Part of it is offloaded to system RAM.
Test protocol: 512 prompt tokens, 128 generated tokens, maximum GPU offload, flash attention OFF. These are benchmark workloads, not maximum context-window measurements.
GPU peak: 120W, 46°C. No thermal throttling.
🤯 AND HERE’S THE SURPRISE:
We moved the same RX 580 from a much more powerful 12-core, 64GB quad-channel DDR3 workstation into this old 4-core, 32GB dual-channel machine.
Performance barely changed!
For these small models, the GPU matters far more than the expensive workstation around it.
No cloud. No subscriptions. Your hardware. Your data.
This is a real, affordable experimental AI laboratory. And possibly a useful starting point for practical local AI applications.
🎄 Here’s an idea: build one for your kids as a New Year’s gift. Their very own computer for experimenting with language models, learning AI, and building things!
And now imagine what models we might be running on just 8GB of VRAM with llama.cpp by New Year’s Eve.
THE LOCAL AI REVOLUTION MIGHT ALREADY BE COLLECTING DUST IN YOUR GARAGE. 🚀
New version is up! The update was focused on stability 👀
See more: github.com/Niko1221/Strata/r…
1
144
🚨 Strata v0.2.17 by @coldniko is out! Strongly recommend updating.
🔹 Quality fix: disabled an optimization that could degrade responses.
🔹 21 engine crash/wrong-output issues fixed, plus tool-calling fixes.
🔹 Better RAM usage: 47 vs 40 GiB for experts on 64 GiB systems.
🔗 github.com/Niko1221/Strata/r…
1
15
2,116
Replying to @since1795
Не сомневался и не переживал по этому поводу, все еще впереди!
nitter.cf/uctive/status/21201122…
Replying to @since1795
Для нас и для прогресса не имеет значения при какой оценке они привлекут 1 или 2 триллиона…
А для них тоже не имеет значения, сейчас оценка 1 триллион, через полгода привлекут по оценке 3 триллиона 🤣
Или обанкротятся, но наследие в виде людей, моделей и клиентов никуда не исчезнет…
1
59
4/4 🧠 WHAT DOES THIS ACTUALLY PROVE?
First, an important clarification: this is an EXPRESS benchmark, not an exhaustive scientific study.
We wanted to understand the scale of the performance differences under realistic hardware constraints, not spend weeks establishing rankings precise to the last percentage point.
Why?
Because local AI inference is evolving incredibly quickly.
Strata, llama.cpp and specialized forks can introduce major architectural improvements within days. A comprehensive benchmark may already be outdated by the time its analysis is finished.
Our approach is deliberately practical:
🔹 Use the latest available versions.
🔹 Apply meaningful optimizations.
🔹 Run identical or closely matched workloads.
🔹 Measure performance, context handling, concurrency and stability.
🔹 Publish commands, versions, logs, model hashes and known limitations.
🔹 Repeat when important new versions arrive.
What did we learn?
1️⃣ Expert-cache optimization matters enormously.
llama.cpp gained 30–50% in several workloads simply by changing how MoE experts were cached and scheduled.
2️⃣ Architectural differences remain significant.
Strata’s expert placement, prompt processing, speculation and conversation management deliver a large advantage on this particular machine and model.
3️⃣ Caching can change the entire picture.
llama.cpp repeated an identical ~39K-token prompt in just 0.168 seconds.
Its warm conversation follow-up took 1.53 seconds, close to Strata’s 1.43 seconds.
But a warm-cache result is not a cold-prefill benchmark. Prefix reuse is also vulnerable to cache eviction, and llama.cpp’s cached conversation state does not persist across restarts.
4️⃣ Performance rankings are workload-specific.
On long contexts of 128K tokens and above, the aggressively optimized llama.cpp configuration actually regressed in generation speed relative to its previous settings. There is no universally optimal cache size.
5️⃣ No single benchmark should be treated as a permanent verdict.
We are not claiming that llama.cpp is universally slow, or that Strata is universally superior.
On this system, with this 54.5GB MoE model and 24GB GPU, Strata is clearly the more effective engine for the workloads measured.
Yesterday we also compared Basalt, our RTX 50-series-specialized Strata fork. That is a separate comparison, with different trade-offs.
Our goal is not to crown a permanent winner. It’s to make powerful local AI practical, affordable and measurable — and to share what actually works.
📂 Complete reports, exact configurations and reproducibility details:
Original comparison:
github.com/ViktorIXAI/strata…
Optimized llama.cpp re-test:
github.com/ViktorIXAI/strata…
All measurements are from our own hardware. No vendor-supplied benchmark figures were substituted for our test results.
And yes — if either engine improves significantly next week, we’ll test again. 🚀
157
3/4 📊 RESULTS — Optimization helped llama.cpp. A lot. But the remaining gap is substantial.
First, credit where it’s due.
The original llama.cpp configuration had no optimized GPU MoE expert cache. After enabling and tuning it:
📈 32K prefill: ~170 → 237 tokens/s (+39%)
📈 32K generation: ~30 → 41 tokens/s (+37%)
📈 4K generation: ~29 → 43 tokens/s (+49%)
✅ Eight agents with ~39K-token prompts each became possible, rather than failing because their requests exceeded the available per-slot context.
This is a meaningful improvement from configuration changes alone. Our same-version control confirmed that the gain was mainly due to expert-cache tuning, not simply a newer binary.
Now compare with Strata v0.1.62:
⚡ Cold prefill:
• ~39K tokens: Strata 2,866 vs llama.cpp 237 tok/s — 12.1×
• ~70–75K tokens: Strata 3,160 vs llama.cpp 225 tok/s — ~14×
⏱️ Time to first token:
• ~39K prompt: 13.7s vs 165.9s
• Long ~150–170K prompt: 47.6s vs 790.3s
The longer prompts have different token counts, so the second comparison is indicative rather than perfectly normalized.
🧠 Single-stream generation:
• 32K: Strata ~249 client-reported tok/s
• llama.cpp ~41 tok/s
But there is a major accounting caveat: Strata’s client-reported generation includes effects from speculative decoding, and its internal engine rates are materially lower. These numbers should not be treated as a clean 6× comparison of raw decode kernels.
👥 Eight-agent workload:
• Strata: 126.5 aggregate generated tok/s
• llama.cpp: 7.2–8.3 aggregate tok/s
That’s approximately 15–18× more delivered output in the measured workload.
However, Strata’s result benefits from prompt reuse, while llama.cpp’s eight-agent measurements include unique and shared-prefix variants. A fully cache-normalized eight-agent comparison was not performed.
🔋 Thirty-minute sustained operation:
• Strata: 1,285 successful requests
• Optimized llama.cpp: 270 successful requests
• Both: zero errors
GPU energy per generated token:
• Strata: 1.20 joules
• llama.cpp: 4.94 joules
That’s roughly 4.1× better measured GPU energy efficiency for Strata.
CPU and RAM energy consumption were not included, so this is not a whole-system power comparison.
🎯 Accuracy check: 5/5 injected fact-retrieval markers for both engines.
The conclusion is straightforward: tuning dramatically improved llama.cpp, but Strata remained substantially faster for this specific oversized-MoE workload.
1
178
2/4 🧪 TEST SETUP — Did we actually compare current, optimized engines?
Yes. We verified the versions against GitHub releases.
🔹 Strata: engine v0.1.62, corresponding to the latest Strata v0.2.16 release (November 18, 2026).
🔹 llama.cpp: b12533, with the new MoE expert-cache functionality enabled and tuned.
Hardware:
💻 NVIDIA RTX 5090 Laptop — 24GB VRAM
🧠 64GB system RAM
⚙️ AMD Zen CPU, Linux, CUDA 13.0
📦 Qwen3.8-Flash-Next, IQ3_S, ~54.5GB GGUF
The model is more than twice the size of available VRAM. This is exactly the kind of workload where managing experts between system RAM and GPU memory matters enormously.
Strata configuration:
• 16,888 MiB GPU expert cache
• Optimized expert placement and CPU/GPU scheduling
• 196,608-token context
• INT8 KV cache
• Conversation parking and restoration
• MTP speculative decoding and lookup
• 15 CPU pool workers
• Chunked prefill
Optimized llama.cpp configuration:
• 11,968 MiB GPU MoE expert cache
• --moe-cache-cpu-share 0.98
• CUDA backend and Flash Attention
• Q8_0 KV cache
• 15 CPU threads
• 198,144-token single-slot context
• Native prefix/KV-cache reuse
• Speculative decoding disabled
We tested nine llama.cpp configurations before selecting the best practical candidate for the main test.
Both engines used RAM-backed model loading and expert offloading. The underlying model files used memory mapping, not an independently verified, identical direct-from-disk streaming pipeline.
⚠️ Important: these are optimized configurations, not identical internal algorithms. Strata’s speculative decoding, expert-placement mechanisms, and conversation management differ substantially from llama.cpp.
We are comparing practical deployment performance on this hardware, not claiming a perfectly isolated kernel-level benchmark.
1
240
🚀 Strata vs optimized llama.cpp — RTX 5090, 24GB VRAM.
Same 54.5GB AI model. Same hardware.
⚡ Prefill: up to ~14× faster
🧠 8 agents: ~17× higher throughput
🔋 ~4× better GPU energy efficiency
We tested the latest Strata against a tuned llama.cpp.
Results & caveats 👇
Replying to @LeTechLead
Absolutely, it was my oversight not to clarify this in my very first post.
Strata is a product on a completely different level, especially after the impressive improvements in its latest versions. I’m immensely grateful to Nico for his outstanding work.
Basalt, on the other hand, is a highly specialized fork of Strata, designed for a specific set of tasks. And I certainly respect the work being done on Basalt as well.
This is precisely the beauty of open-source software: a powerful, general-purpose project can become the foundation for many specialized solutions, each evolving in its own direction and serving different needs.
Thank you for pointing this out. I hope others will read your clarification and understand this important distinction.
38
Replying to @FearL0rd
No, unfortunately. Basalt is a fork of Strata specifically optimized for NVIDIA Blackwell GPUs (sm_120), primarily the RTX 50 series. Older architectures like Volta (V100) are not supported.
1
352
🚀 Basalt v0.5.0 is out!
This morning, we published our Strata vs. Basalt benchmarks. And they’re already outdated! 😅
What’s new:
⚡ 92 tok/s in conversations, 171 tok/s structured output
🔥 3,454 tok/s prefill
🧠 1M-token context, FP8 KV cache
🔧 NVFP4 experts, DFly2 speculative decoding
RTX 5090 + 5060 Ti.
All figures are from the release author’s benchmarks. We’ll also be using the new Basalt version and sharing our own test results.
github.com/jesdga95/basalt/r…
The cold-cache numbers are in the report: Strata ~3,200–3,300 tok/s, Basalt ~3,035 tok/s. Single-stream decode: Strata ~94–156 tok/s, Basalt ~139–172 tok/s.
Yes, some Basalt runs were cache hits. We explicitly documented that instead of presenting them as cold results.
Raw data, configurations, methodology and limitations are all public:
github.com/ViktorIXAI/strata…
Constructive technical criticism is always welcome.
1
19
Replying to @Eth_ridgway_ @coldniko
I added the independent benchmark results further down in the thread. The performance improvement I mentioned in the original post was taken from the author’s GitHub description. However, my own independent testing produced similarly impressive results.
11
Replying to @pardoxAI
They’re different metrics. Aggregate throughput depends heavily on workload, concurrency, and KV-cache reuse. Our results include cache-heavy workloads, not fully independent cold decoding.
Full methodology and raw data:
github.com/ViktorIXAI/strata…
1
92
Replying to @spnova05
Not an ASUS ROG Strix SCAR. It’s a Captiva Ultimate Gaming laptop.
Hardware configuration:
▪️ AMD Ryzen 9 9955HX — 16 cores / 32 threads, up to 5.4 GHz
▪️ NVIDIA RTX 5090 Laptop GPU — 24GB GDDR7
▪️ 64GB DDR5 RAM
▪️ 2TB NVMe SSD
▪️ PCIe 5.0 ×8 GPU connection
▪️ Linux, CUDA 13.0
All Strata and Basalt benchmarks were performed on this same machine, using the same 54.5GB IQ3_S MoE model.
57
OpenAI Dots & GrokBot alternative running fully locally!! It uses PolyStrata (Strata engine) to serve and swap the models it needs and, of course being fast 😎
Check out more here: github.com/simplexai-labs/li…
Replying to @Kutaadaw
Yes! And it’s a little healthier than beer. Although I love beer myself, too — no point in hiding it! 😄
1
1
11








