ビーズちゃん

The question that is stopping me from buying an M5 Ultra: why does a Mac with 1.2 TB/s of bandwidth lose to DGX Spark at long context and with multiple users? With one user, decode is memory-bound. Every token means reading all the active weights once, and the GPU mostly waits for data. More bandwidth means more tok/s, and that's where the Mac shines. 👍 Batching changes everything. With 8 users you read the weights once and use them for 8 tokens. Same bytes, 8× the math. Throughput scales almost for free… until compute becomes the bottleneck. 👀 That's where the Mac hits a wall: good bandwidth, lower low-precision compute. The Spark's Blackwell tensor cores with native FP4 give far more FLOPs per GB/s of bandwidth, so it keeps scaling while the Mac stops. Real numbers on DeepSeek V4.1 Flash with TensorFold, total tok/s from 1 to 8 users (chart below): 2× Spark: 77 → 105 → 150 → 194 (@rwojo) 1× M5 Ultra: 63 → 78 → 98 → 99 (@TechMDAI, first runs, capped at 4 slots) Long context is even worse. Prefill is pure compute, attention grows with context length, and the KV cache is per-user: no way to share it. On DeepSeek V4 Flash the Mac's decode drops from ~60 to ~27 tok/s, while 2 Sparks hold ~51 (tests by @satsuki2694). Bottom line: the Mac is unbeatable for fitting huge models (MoE) in a single box, but the Spark wins when you need compute: long prompts, many users, agents. Different tools, different jobs. I run both, and I was planning a deep dive on the M5 Ultra to really understand its limits, but at this price I don't know if it makes sense spending all these money just to make tests and share data. I'll try to figure out something, otherwise I'll wait for others to test these concurrency limits in depth. 💪

これ、TensorFold で同じことを出来るのかな？　試してみよう！

前回の3レシピ比較の続き。今回はCRS804を使ったスイッチあり構成で再比較してみた。さらに、気になっていたspark-ds41fも追加して、4レシピで比較。 DeepSeek V4.1 Flash 1017の公式重みを使用。DGX Spark×4／TP4／200G接続。256K入力・同時1件・GPUクロック上限2300MHzで、SparkDashを使って各3回測定しました。 今回の条件では、decodeはspark-ds41fが平均70.29 tok/sでトップ。キャッシュなしのprefillはknapcioが約6,180 tok/sで最速でした。 個人的には、選ぶならこの2つかなという感想です。spark-ds41fはdecodeもprefillも速く、普段はこちらをメインに使っていこうと思います。 1枚目がdecode・prefillの平均値、2枚目がTTFTの中央値と測定条件です。 ※TensorFoldは10/28更新版を使用。前回からOS・ドライバー等も更新しているので、差がすべてスイッチの効果とは限りません。 比較したレシピはこちら↓ spark-ds41f（vLLM／TileLang） github.com/christopherowen/s… knapcio（SGLang） github.com/knapcio/DeepSeek-… MiaAI（TensorFold） github.com/MiaAI-Lab/DeepSee… MiaAI（SGLang） github.com/MiaAI-Lab/DeepSee…

⚡ Introducing UBTree: a frontier dLLM-based method for inference speedup🚀 On Ling3-Flash-124B, UBTree delivers ~1,010 output tokens/s with 4-GPU tensor parallelism (TP4) at concurrency 1, averaging a 3.43× end-to-end speedup over target-only autoregressive decoding across six benchmarks. UBTree uses a unigram proposer to provide candidate draft tokens and a bigram selector to connect these candidates into a tree. We build this draft tree completely in parallel. Today we are releasing two drafters for the @AntLingAGI family models: 🔗 Ling-3.0-tiny-8B: huggingface.co/inclusionAI/L… 🔗 Ling-3.0-flash-124B: huggingface.co/inclusionAI/L… Use them to accelerate your Ling-3.0 now! More draft models are coming over the next month. 📄 Paper: inclusion-ai.org/ubtree/UBTr… 🏠 Website: inclusion-ai.org/ubtree

Qwen4, coming soon… CTO quote was “next week” Xudong Guo - Research Scientist, Qwen, Token Foundry Live from Amsterdam

これは…年内で、DGX Spark の在庫枯れますね。 データセンター破綻なり AI バブル崩壊なりで VRAM 相場が落ち着く流れも無くはないですが、それを半年待てるなら ADHD やってないｗ　恐らくローカル LLM 界隈でもクリスマス前に何らかの盛り上がりがあると見て、もう一歩踏み込みます！

熊谷氏をみていると、一代であそこまで会社を大きくするには、相応のサイコパス味は必要なのだなと改めて。

spark-ds41f-1025 is out. DeepSeek V4.1 Flash on DGX Sparks. 1M context. Native weights. 1. Every kernel the server runs is TileLang now. B12X only loads the weights. 2. Outputs: same as last release. Bit for bit. Logprobs too. 3. Prefill: +12-14% on four Sparks. +23-28% on three. 4. Decode steps 1.7-2.4% shorter. Throughput: level. 75 tok/s single stream on four. 5. Two logprob bugs fixed. One of them could take the whole engine down. 6. 5,730 tests passed. 0 failed. 7. weightless abliteration (GLP steering). It edits the activations, not the weights. Off by default. Auditable. Won't hack you. 8. Everything is called spark-ds41f now. Almost. Rewrote every kernel. Changed zero bits. That was the point. github.com/christopherowen/s…

Looks like next Zhipu models is going to be from GLM-5.5 family GLM-5.5-Flash is now listed in OpenCode. opencode.ai/data/zhipu/glm-5…

約26時間の休暇から戻ったPMのOpus5.5さん。 休暇中に代理で引き継いでくれていたPMのChatGPT 6.1 Solさんの仕事をみて怒涛の問題点を発見していく。。。 製品コードを書き替えてテストを実施している？ 管理ドキュメントのメンテナンスが狂っているだって？ ログ見たらわかる失敗の検査に無駄なアプローチで膨大な時間を浪費したて？ Opus5.5さんは、休暇明けから火消しに走るのでした。。。

ほとんどの人間は自分でエビをメンテできないということがわかったwww

TensorFold Zig engine 1.1.5 and Mac app Beta 10 are out 🙌 Beta 10 brings community-v2 benchmarks, Google sign-in fixes, updates that work through fake-IP proxies and model compatibility checks before loading. It bundles engine 1.1.4. The standalone Zig engine is now at 1.1.5, with a lot of work from the community. - Qwen3.8 Flash Next on GB10 with NVFP4: up to 28% faster prefill. - Qwen3.8 Flash Next on M3 Ultra with MLX 4-bit MTP: 16% to 23% faster decode in the release tests. - Follow-ups about the same 35k-token document: first token in 2.8s instead of 25.7s on GB10 with Flash Next FP4-Local. - LM-5.3-Flash NVFP4 at TP=2 on two GB10s: 22% more tokens/s with two streams. There's also Dory support on CUDA, more checkpoint formats, Drama 3 improvements, tool-call fixes and Windows CUDA build fixes. Huge thanks to everyone contributing to the engine. @Bizuayeu @plotarmordev @jerry543 @MiaAI_lab @agispof @softpoo @GumbiiDigital @bobclawblaw @jvr0x @ViC305 @rielstamand And on GitHub: jschmied, BlivionIaG, harrisonfriia, drowzeys, Sapphire-Rapids and Boscoeuk. Thanks for the reports too: @devopsfortunato, laidick, flashosophy and ss-cong. To everyone testing, sharing access to hardware and supporting TensorFold, thank you. If I've missed anyone, I'm sorry. @MiaAI_lab @plotarmordev @volatilemarkts @petruspennanen