dextune

I happily picked up two used Nvidia Tesla P40 GPUs. I knew the P40 was old and ridiculously overbuilt. What I did not expect was how long the GPUs would become once the coolers were attached. Tried putting them in the case. Nope. The cooler end would not fit. I thought about it for about 10 seconds. Turns out the bottleneck was not the human. It was the case. So I grabbed a hammer and pliers and destroyed the front section of the case. After that both P40 GPUs fit perfectly. Put the cover back on and you cannot even tell from the outside. Apparently hardware fits if you hit it hard enough. Good experience.

AX Engine AX Engine brings multi-model LLM serving and MTP acceleration to Apple Silicon, with Qwen3.6 35B-A3B reaching 159.10 tps on an M5 Max. In streaming OpenAI API tests, decode speed increased from 129.06 to 159.10 tps against a peer MLX server using the 4-bit model. Its MTP path accelerates generation by predicting multiple tokens per step, with a reported 1.68x geometric mean speedup across tested 6-bit models. One process also supports multiple resident models with exact prompt prefix reuse, reaching 5.03x median throughput against a multi-process MLX server. github.com/defai-digital/ax-…

colibri colibri runs the 68GB Qwen3.8-Flash-Next IQ2_XS on an RTX PRO 4000 with 24GB VRAM at a 256K context. The project reports prompt processing rising from 15.6 to 149 tps, while decode improves from 9.6 to 45.7 tps at that context length. The setup keeps float16 K/V cache on the GPU. Experts that do not fit in VRAM stay in RAM or on disk, with CPU and GPU computation overlapping. github.com/JustVugg/colibri

Mage-VL-Cowork 4B Mage-VL-Cowork is Microsoft’s new 4B computer use model, scoring 66.0% on OSWorld-Verified with a 50 step budget. It switches between screenshot based GUI actions and bash commands within a single policy, without an external router. Online RL improved accuracy by 10.16 percentage points over Hybrid SFT while cutting output from 7,374 to 5,499 tokens per task. The bfloat16 weights occupy 9.48GB, and Microsoft provides an SGLang container for inference. huggingface.co/microsoft/Mag…

Project Maya DeepSeek-V4.1-Flash runs unsloth’s UD-IQ3_S GGUF in Project Maya v1.1.43, using about 262GB of VRAM and RAM. The quant has a KLD of 0.061 against the original, versus 0.047 for Q8 and 0.119 for Maya-S. Another 161GB of tables are read from SSD. The loop guard now preserves prompt caches. On 150K token GLM and DeepSeek agent chats, the next turn took about 15x less time in Maya’s tests. SSD expert reads are now tuned per drive. Qwen CYBER FROST decoded up to 5% faster on NVMe with an 8GB GPU. github.com/mw00/project-maya

MiniMax M3.1 Flash w/ Uncensored GGUF An uncensored abliterated GGUF variant of MiniMax-M3.1-Flash has been published by orcarouter on Hugging Face. The model is based on MiniMax’s 428B parameter MoE architecture, with 23B parameters activated per token. Abliteration is intended to reduce refusal behavior while retaining the underlying model capabilities. This release packages the modified weights in GGUF format for local inference. huggingface.co/orcarouter/Mi…

TensorFold TensorFold 1.2.0 cuts an LFM2.5-24B-A2B 2,048 token prompt chunk on M1 Pro from 243 to 3.9 seconds. In its tests, first token latency on a cold 930 token prompt drops from 212 to 2.0 seconds. Sixteen greedy streams reach 116 to 123 tps combined, up from 11 to 17 tps. The fix stores matrix fragments whole instead of element by element, with bit identical output. Qwen3.8-27B on M3 Ultra also improves from 145.7 to 209.4 tps at 16 concurrent streams with DFlash2. github.com/ashhart/TensorFol…

JetSpec++ w/ 2 x GB200 JetSpec++ reaches 4,117 tps on Qwen3-8B in NVFP4 using 2 x GB200 GPUs for a single request. On MATH-500 in BF16, the authors report 1,361.1 tps versus 902.1 tps for DSpark and 182.7 tps for autoregressive decoding. The second GPU drafts and refines the next token block while the target model verifies the current one. Qwen3.8-27B reaches over 900 tps in NVFP4 and 367.2 tps in BF16 on the same setup. jetspec-project.github.io/je…

MiniMax H3 Character Sheet Designer H3 Character Sheet Designer adds 13 selectable views for MiniMax H3 character sheets in ComfyUI. Reference13 takes one character image and up to three optional references, with roles for the face, head, outfit, hands or footwear. It compiles prompts and output sizes locally. Layout OFF uses text instructions instead of a guide image while native H3 performs generation. Version 1.2.2 fixes stale part prompts after Apply JSON. Existing node IDs and workflows are unchanged. github.com/ukr8b3g-cmyk/H3-C…

Strata Strata 0.2.9 enables parallel requests and Gated DeltaNet kernels by default on supported Linux GPUs, while adding Unsloth 1-bit model support across NVIDIA, AMD, DGX Spark and Mac. On an M5 Max, Qwen IQ3_S decodes at 79 to 104 tps. Unsloth UD-IQ1_S reaches 15.4 to 18.5 tps. Two RTX A6000 cards deliver 148 to 169 tps on Qwen IQ3_S, while DGX Spark reaches 55 to 67 tps. The update also adds Intel Arc B-series prompt attention optimizations and fixes chat rendering overhead. For a 40,000 token answer, main thread time drops from 15.7 to 9.0 seconds. github.com/Niko1221

Halogen w/ GMKtec EVO-X5 Pro AMD Ryzen AI Max+ PRO 495 Qwen3.8-Flash-Next Halogen pushes Qwen3.8-Flash-Next to around 75 tps sustained on GMKtec EVO-X5 Pro, versus about 23 tps with llama.cpp on the same AMD Strix Halo chip. Halogen github.com/peonist-ai/haloge… GMKtec reports up to 2,000 tps prompt processing and a 94 tps peak for repetitive output using speculative decoding. The engine uses gfx1151 specific GPU kernels, MoE expert routing, and 5.53 bpw quantization to reduce data movement across the 256GB/s memory bandwidth platform. The EVO-X5 Pro supports up to 192GB of unified memory for large local models. gmktec.com/blogs/news/same-c…

oh-my-pi oh-my-pi v18.14.1 updates its coding agent with queued context compaction, improved Cursor tool execution, and fixes for Codex sessions. The /compact command can now be queued with Ctrl+Q while the agent is working, running when the current task yields instead of interrupting it. Cursor models through auth-gateway now keep tool calls and results in the same run, with execution state retained for up to 30 minutes. Anthropic requests now support a 128MB body limit, while native web reads use about 6% fewer tokens. github.com/can1357/oh-my-pi

OpenCode OpenCode v2.2.5 adds percentage based compaction buffers, allowing a setting like 20% to reserve context space across models with different window sizes. MCP servers no longer disconnect when stdio messages exceed 10MiB. LiteLLM context window errors now trigger compaction instead of stopping the session. The desktop app adds shortcuts for switching worktrees, selecting base branches, and toggling shell mode. Edit errors now show nearby lines when the target text cannot be found. github.com/anomalyco/opencod…

RDNA-FAST RDNA-FAST is tuned for AMD RDNA3 gfx1100 GPUs, using WMMA matrix cores for prefill. On an RX 7900 XT 20GB, Swift 1.5 Qwen3.8-27B runs at 96K context with two agent slots. A scripted two agent session finished in 97 s versus 132 s in LM Studio with the same quant. Cold 4K prefill rose from 728 to 974 tps, while decode at 41K increased from 42.4 to 59.2 tps. MTP and ngram drafting reached 305 tps when code edits copied prompt text. SSD cache restored 40K to 53K token chats in 2.9 s versus 77 to 105 s. github.com/serrynaimo/RDNA-F…

Project Maya Project Maya v1.1.42 adds image input for GLM-5.3-Flash on AMD GPUs and brings vision to Qwen-3.8-Flash-Next for the first time. On an 8GB card, Qwen's 0.9GB GPU image encoder reads pictures about 3.6x faster than the CPU encoder. DeepSeek reads fresh 20K to 30K prompts 16 to 24% faster on one or two GPUs. Tensor parallel prefill across two GPUs is about 1.5x faster on PCs with few CPU cores. GLM's automatic prompt chunking reads a 19K prompt about 3.4x faster on affected cards. github.com/mw00/project-maya

Nemotron 3.5 Super VL w/ GB300 Nemotron 3.5 Super VL reached 11,840 aggregate output tps on a GB300 with 256 concurrent requests in Cline’s tests. The base vLLM config delivered 6,058 tps. MTP decoding, async scheduling and the FlashInfer Mamba kernel raised it to 7,800 tps. Reducing MTP lookahead from 3 to 1 then raised throughput to 9.5K tps by avoiding extra draft verification work. The final change moved Mamba state from FP32 to FP16 with stochastic rounding. cline.bot/blog/11-840-tps-on…

Talus w/ Ryzen AI Max+ 395 Talus pushes Qwen3.8-27B coding speed to 59.9 tps on Ryzen AI Max+ 395, up from 46.8 tps with a patched llama.cpp build. On the same 70W HP ZBook Ultra G1a with Radeon 8060S and 128GB unified memory, prefill at 32K rises from 270 to 416 tps. GLM-5.3-Flash also reaches 465 tps prefill at 32K, up from 217 tps with patched llama.cpp. Built on Strata, Talus combines a Rust host, HIP kernels tuned for Strix Halo and model specific speculative decoding. It supports five models through an OpenAI compatible API. github.com/nsbradley88/talus…