Elation (@ObsidianLib) header
Elation (@ObsidianLib) profile picture

Goo - a .NET declarative desktop UI framework: https://nitter.cf/t.co/gCNIM647xo SharpTUI - a lean .NET TUI Framework: https://nitter.cf/t.co/aPXnMhEFWh

Joined July 2024
Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)Photo by Elation (@ObsidianLib)
Goo speaks for itself. Cross-platform UI framework in .NET on Vulkan. OSS. Try it out :)
1
1
1
11
498
2
11
3
43
2,419
Being anti-AI is just an endless cycle of not understanding how this chart works and crying about it
1
2
118
Hashrate on my wrist #bnt #blocknet
4
3
1
10
910
🐉longhorn update🐉 #bnt #blocknet
2
6
258
awesome the first #blocknet block mined on the longhorn pool happened last night! Not many people on the pool yet, so it all went to one miner. If you have an AMD CPU/GPU, get in on the pool: longhornpool.net/
1
4
7
228
LONGHORN is LIVE. longhornpool.net A #BNT #blocknet miner built for AMD, plus a zero-fee pool. Pool 0% fee Miner 2% fee The lowest total fees for AMD miners.
1
5
8
633
blocknet miner specifically tuned for AMD, anyone? Long Horn 🐉😈 #bnt #blocknet
2
4
6
733
local models take a long time to test...might as well make something to keep you mesmerized as it works.
2
87
Gloop 🤝 Tern
6
154
Elation retweeted
Donating inference to your favorite OSS project will soon be beautiful and easy. Set up your repo, donate to another repo, browse issues, save profiles with your preferred harness, model, and effort. Give your inference a purpose 🌞
1
2
4
65
Bus use: stock llama.cpp reached ~145GB/s on Ornith. These engine tweaks reach ~171GB/s over a whole pass and 234GB/s in its best kernel. The memory floor for a pass is ~17.4ms, this implementation reaches 23.5ms so still some headroom left at the same quant.
2
26
Caveats: custom ggml + libllama under stock llama-server Speed moves ~2.5% run to run and more by prompt. Pooled routing changes the model, a preliminary and partial SWE-bench run showed about 11/19 resolved rate with my pi harness, 10m per task.
1
19
The takeaway so far: capacity is free but bandwidth is not. The ideal model is ~100GB of small experts behind a thin trunk, reading ~3GB per token. Ornith has the read pattern right, but does not fill the hardware's capacity.
16
Draft head: only the 65,536 most common tokens of 248k, stored at Q4_0. Used for drafts only, every token is still verified by the full Q8_0 model so the output does not change. It only moves draft acceptance rate.
1
16
Results so far, Q8_0 @ temp 1.0 32k ctx: Decode with MTP: 59.3 -> 99.7 t/s Decode without MTP: 53.1 t/s Verify pass: 33.8ms -> 23.5ms Prefill: ~1,100 t/s TTFT: 126ms short prompt, 1.5s at 1.6k tokens 37.9GB file, ~43GB loaded
1
27
Next lever is MTP and Ornith ships its own draft block. It drafts 3 tokens, then one main pass verifies 4 positions. The trunk is read once per pass, not once per token. So, a thin trunk makes verification cheap. ~2.7 tokens kept per pass.
1
24
Then the engine, a stock verify pass is ~2,400 GPU nodes and each small kernel costs about 5us. I fused them: MoE layer 42 nodes -> 1 op, linear attention layer -> 3 ops. Now 284 nodes and each weight row is read once for all 4 tokens.
1
11