CTO @conste11ation | generating utility
Joined November 2015
- Tweets735
- Following195
- Followers2K
- Likes303
I finally got my CMP 170HX LLM server running, and this thing is an absolute beast: 128GB of VRAM across two cards, running Qwen3.8 27B at 8x concurrency.
Single-stream decode is already above 150 tok/s with DFlash, with aggregate throughput approaching 200 tok/s. I’ve barely optimized the workload.
At API prices, it could easily burn through $100 a day. Working on some interesting experiments with agent swarms to soak up all that bandwidth...
Alex Brandes ² retweeted
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead.
You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement.
I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible.
But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier.
Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want.
Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well.
So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it.
If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
This is the truth and I think the people that are deep in it also see that we’ve only barely scratched the surface.
I’d rather be in the trenches than sitting on the sidelines.
You can do this with tmux on its own, or with wrappers. I find this works really well with Orca which gives you the option to see tabs of all the sessions or run the in the background if you want.
Agents coordinating agents becomes a necessity at a certain point. Subagents in a single session is not the same thing.
I’m seeing lots of tweet replies asking basically “but how do I get two models to talk to each other? Without me manually copy/pasting results between apps?”
And I’ve heard this same question repeatedly from friends who use AIs mainly through GUI apps.
Yes, there are various apps starting to be released for just this problem. But it’s worth knowing that in a pinch you can already do all this yourself, on the command line, just by using tmux.
Here’s how.
On the command line start a named tmux session with the command “tmux new -s chat”.
Create a second window within that session. (The default keystroke for this is doing “ctrl-b c”. You can switch between windows by doing “ctrl-b n” for the next window.)
Launch the codex CLI in one window, and the claude CLI in the other.
Then you can literally say to the AI in window 1, “please read the analysis of the AI in window 2 of the tmux session ‘chat’. Wdyt?” And vice versa. Now you have cross-harness review.
Or you can instruct one AI to delegate and manage the other AI over multiple turns, or monitor it over time, or whatever. This works because each AI can use the tmux CLI to read and write to the other’s window.
This does not on its own solve coordination problems like deciding who is in charge, or who talks first, or who waits for whom, but it establishes the basic communication primitives of “read the other AI’s output” and “write to the other AI”. You can build more complex workflows on top of that mostly by prompting.
The fundamental reason this all works is that the Unix-era command line environment is more interoperable than the modern GUI environment, so a decades-old tool like tmux is still one of the best ways to compose modern AIs. Obviously the command line tools are a PITA in other ways. But it’s great that they are so flexible, they already exist, and they work right now, so you don’t need to wait for the vendors to get their apps to play nicely with each other (which might never happen) or for someone else to catch up and write a new app just to connect things.
Astra is looking really impressive across pretty much every benchmark.
Kind of wild that turning up the reasoning actually makes it cheaper. It takes fewer steps to finish the task, so the total cost goes down.
GPT-6 Astra represents a step-function change in model capability for interactive reasoning problems. It scores 66% on ARC-AGI-3 using our standard harness, and nearly 100% with a continuous conversation harness and custom compaction, at a cost of roughly $360 per game.
In fact, the continuous harness version significantly outperforms our human baseline in action efficiency across almost all levels. When we examined the reasoning chains to understand how the model operates, we found it performing highly efficient, on-the-fly symbolic world modeling for each game and level. It goes as far as developing its own shorthand DSL to represent in-game situations -- essentially a game-specific algebraic notation.
Overall, Astra exhibits symbolic modeling behaviors we had previously only seen with sophisticated harnesses -- so harness capabilities are increasingly shifting into the model itself.
We see Astra as a major breakthrough in model intelligence.
Read our post on Astra and what these results mean: arcprize.org/blog/astra
A lot of people are sleeping on how big a deal it is to generate video faster than it can be played back.
Fast H3 can generate a video before you finish watching it, opening up a whole new category of real-time video applications.
This is the beginning of video becoming an interactive interface, not just content.
🎬Video generation faster than playback!
🚀MiniMax H3 on vLLM-Omni + FastVideo's FastH3: a complete 10.1s MP4 - video AND synchronized audio - rendered in 8.7s!⚡️
Thanks to @MiniMax_AI for the great Minimax H3 release, the FastVideo team @haoailab for open-sourcing FastH3 and helping on the serving integration, and @NVIDIAAI for the continued sponsorship and joint optimization efforts!
The defects of the past are the raw material of the present.
Let’s get weird!
I think the CMP 170HX just became one of the weirdest local-LLM value plays. 👀
I got Qwen3.8-27B W8A16 running on a single 170HX 64GB with vLLM 0.27.1 + DFlash2:
⚡ ~120 tok/s decode
🚀 ~1.6–1.9K tok/s prefill
🧠 262K context with BF16 KV
📦 342K-token KV capacity
🎯 DFlash2 averaging 3.66 tokens/draft
💾 60.5 / 63.5 GiB VRAM used
27B model is doing ~120 tokens/sec on one GPU.
Software optimization is getting ridiculous.
Those numbers are from your measured W8A16 + DFlash2 run: ~120 tok/s decode, ~1,580–1,940 tok/s prefill, 342,729-token KV capacity, and 60.5/63.5 GiB VRAM usage.
#170hx #LocalAI #localllm
This blog post is not about Fable 5.1 but it could be.
Nonetheless I wrote about how I use coding agents to improve and maintain @openpokerai
constellationgate.ai/blog/op…
Wow. I can’t see any way this doesn’t become the future of video content.
Video created on the fly as you watch it, able to be directed and influenced by the audience reaction is a new paradigm that has just arrived.
Pulled the trigger on a pair of these CMP 170HX cards last night.
They’re old crypto mining GPUs with 64GB of VRAM physically on the board, but Nvidia software-limited them to 8GB. Hack around the limit and suddenly you’ve got a very capable 64GB AI card.
Felt like an appropriate way to get deeper into local AI.
🤯 How about a 64GB GA100 for $1,500?
🔥Nvidia sold this GPU with 8GB of memory. Local AI hackers unlocked 64GB of HBM2e AND its crippled Tensor Cores.
Local AI hackers have now unlocked ...
🔥 8GB → 64GB HBM2e
🔥 crippled Tensor/SM compute
🔥 PCIe Gen1 → Gen2
Prices of used CMP 170HX are already shooting 🆙
The NVIDIA CMP 170HX, a 2021 crypto-mining card built from GA100 silicon, was sold by NVIDIA, but they nerfed it! 🔫
🔒 8GB HBM2e
🔒 throttled Tensor/SM compute
🔒 restricted PCIe
🎯The open-source CMP Unlocker can now turn the 8GB version into:
🔥 64GB HBM2e 👀
🔥 full SM compute throughput
🔥 PCIe Gen2
Here is what the author benchmarked...
🧮 Tensor throughput
6.3 → 193 TFLOPS
🦙 Llama 2 7B Q4_0 / llama.cpp pp512
599.6 → 3,468 tok/s
That's a 5.8× increase in prompt processing.
They dug deeper and found what appears to be a deliberate compute throttle ...
🐌 Locked → 256 cycles per MMA instruction
🚀 Unlocked → 24 cycles
🎉The card suddenly gets roughly95% of A100 per-SM Tensor throughput in their test.
And remember ... this is now a 64GB HBM2e Local AI GPU. 👀
Here are the caveats ...
⚠️ Linux + patched NVIDIA open driver
⚠️ Secure Boot off
⚠️ PCIe still only Gen2 x4 without hardware mods
⚠️ unlocked memory has no ECC
⚠️ community hack — definitely enthusiast territory
🔗 Reddit: r/LocalLLM/comments/1vulabl/
🔗 GitHub: /amoghmunikote/cmpunlocker
Woohoo! Handles and peer to peer sending on Arca.
I got @alex.
Meet Arca Pay: a simpler way to send digital dollars between Arca users.
Claim your @handle, share it, and get paid without copying a wallet address.
Sending is free. Confirm the recipient and amount, then tap Send.
Money should feel this easy.
Arca Pay is available on iOS and Android.
Update Arca Wallet to the latest version through the App Store or Google Play to start using it.
Yesterday we got two great open-weight models: Qwen3.8-Flash and GLM-5.3-Flash (formerly Ox Alpha).
Look at where they land on these benchmarks.
If you’re into local AI, you may be able to run them yourself. If you’re using the cloud, you can run them for ridiculously cheap through @_GateAI.
Proprietary AI is getting speedrun in real time.
Qwen 3.8 Flash Next (125B A6B MoE) on the Artificial Analysis Agentic Index:
- Beats Kimi 3 (Max)
- Sitting just 1 point behind Claude Fable 5
- Hardware: Single RTX 3090 / 4090 (24GB VRAM) + 100GB RAM
Matching top tier API performance on a single consumer GPU setup is insane.
Open weights are moving absurdly fast. Are you still paying top dollar for closed APIs or self hosting?
Many people don’t realize but pretty much anyone can run Whisper or Parakeet open source speach-to-text models at home on less than 8GB RAM.
Skip the subscription - all the value is in the open source models. Have your agent set this up for you for free in 5 min.
We're making Whisper models free for all users.
Starting today, you no longer need a Superwhisper Pro subscription to use them.
We’ve also reset Pro usage for everyone.
If you’d already reached your limit, you now have 3,000 words to try the latest Pro features.
Local, private, and fast voice-to-text. Free for everyone.
As it turns out, Ox Alpha is GLM 5.3 Flash!
Try it out on @_GateAI
