VibeCodingBench: We benchmarked 15 AI coding models on what developers actually do
Current benchmarks have an ecological validity crisis. Models score 70%+ on SWE-bench but struggle in production. Why? They optimize for bug fixes in
Python repos—not the auth flows, API integrations, and CRUD dashboards that occupy 80% of real dev work.
So I built VibeCodingBench(vibecoding.llmbench.xyz): 180 tasks across SaaS features, glue code, AI integration, frontend, API integrations, and code evolution.
Multi-dimensional scoring: Functional (40%) + Visual (20%) + Quality (20%) - Cost/Speed penalties. Security gate: Any OWASP Top 10 vuln = automatic 0.
Results (Jan 2026):
🥇 @claudeai Opus 4.5 — 89.2% | $12.31 | 44s
🥈 @claudeai Haiku 4.5 — 89.0% | $3.03 | 22s
🥉 @grok 4 Fast — 88.8% | $0.21 | 70s
4️⃣ @OpenAI GPT-5.2 — 88.8% | $5.01 | 28s
5️⃣ Qwen3 Max — 88.6% | $5.42 | 45s
The real story? Cost varies 60x between similar performers. Grok 4 Fast matches GPT-5.2 at 1/25th the cost. Claude Haiku 4.5 delivers near-Opus quality for $3 total.
Pass rate ≠ final score. Qwen3 Max hits 100% pass rate but lands at 88.6% after quality/cost penalties. Our multi-dimensional approach reveals what
pass-rate-only benchmarks hide.
All 15 models passed security. The top 10 cluster within 2 points. Frontier models have converged—the differentiator is now cost-efficiency.
📊 Live dashboard: vibecoding.llmbench.xyz/
📂 GitHub repo: github.com/alt-research/vibe…
📄 Thesis: github.com/alt-research/vibe…
The ultimate test isn't fixing a bug in scikit-learn. It's shipping a feature your users need—safely, efficiently—before the sprint ends.
Open source. Contributions welcome.
trenchtrapperonsol retweeted
D² is coming to NYC May 20-21
@Designing_DeFi is an academic conference with an emphasis on incentives, economic mechanisms, and protocols
Want to speak? Submit your research by Valentine's Day!
Details ↓
designingdefi.xyz
trenchtrapperonsol retweeted
Biggest swap ever on @THORChain? 🔥
Insane amount. The fact that this can be executed decentralized, permissionless and with minimum slippage is beyond impressive. Love to see this.
🌊 New streaming swap
[aq0y]: 232.2 BTC ($21.7M) → ⚡ → ETH
Track Tx: track.ninerealms.com/EA7F389….
Runescan: runescan.io/tx/EA7F389F32DB5…
trenchtrapperonsol retweeted
Dinamarca, que en 2001 creyó tanto en el Artículo 5 que mandó 18.000 soldados a Helmand, una de las zonas más peligrosas de Afganistán y perdió más vidas percápita que el resto de OTAN, es irrelevante para EEUU.
Otro recordatorio para los que aún quieren apaciguar a Trump.
trenchtrapperonsol retweeted
Who would have guessed shutting down electricity generation is a bad idea
Who are these people running these countries?
Zero accountability, maximum destruction
And if they don’t like your tweets that accurately criticize them, they will just silence you, mock you as some extreme right wing figure
Develish
trenchtrapperonsol retweeted
GM Farcaster ep331 @farcaster_xyz and @clanker_world with Slingshot's @igoryuzo nitter.cf/i/broadcasts/1lDxLBLmY…
trenchtrapperonsol retweeted
trenchtrapperonsol retweeted
Starknet infrastructure progress in 2026.
> More throughput (2x, then another 2x)
> Faster L1 finality via Stwo proof verification
> More predictable fees via better blockspace efficiency
> More decentralization
Starknet optimization never stops.
PERSONAL PRIVATE PROGRAMMABLE
I’ve been thinking more about the intersection of Claude Code and Obsidian. There is an upcoming tech stack here that I’m calling personal private programmable.
Here’s a sketch of the idea.
First, if you squint ahead a few months, we will likely have open weight AI models that work as well as Claude Code does today, and that can handle the context that Google Gemini does.
We also already have many different kinds of file types that can be exported out of apps, from markdown files (like Obsidian) to git repos to mbox files (from email) to all the miscellaneous file types you store online or on disk. The open weight AI models can do a lot if they can see all these different data types at once.
Finally, we have hundreds of millions of local crypto wallets, and scaled crypto name systems like ENS (Ethereum Name System) and SNS (Solana Name System). These give us tools for working with encrypted data, keys, and identity.
So if you put all that together, the personal data becomes far more programmable (with local AI) and yet also more private (because it’s being computed on locally). And if you have a local crypto wallet, you can also encrypt some or all of your personal data and transmit it to another ENS/SNS name, such that you can collaborate with them in an end-to-end encrypted way.
There are many networking details to be worked out regarding secure synchronization of packets between different ENS/SNS names across different machines, and you might need some kind of private cloud like Gitlab to really make it work depending on the application (because pure p2p is hard).
Nevertheless, the Claude/Obsidian trend points at a powerful emerging concept where it’s now *better* to redecentralize and keep all your data local. Because local data is easier to encrypt with crypto and compute on with local AI.
Thus: the personal becomes private and programmable.
trenchtrapperonsol retweeted
It’s hard to believe that all we had to do to change the military was to stop backing down.
We went from a disastrous withdrawal from Afghanistan to snatching rogue heads of state in their sleep.
Ending nuclear threats with stunning precision.
Reducing drug deaths in America by orders of magnitude.
And through it all, the ranks swell with recruits. Why?
Because America loves winning, and responds accordingly.
And the only battle casualties belong to the enemy.
Our standards are unforgiving and the results are unforgettable.
Credit where it’s due to the Secretary of War @PeteHegseth , and the force that remembers what it can do when they’re allowed to do it.
Our current reality is unrecognizable from over a year ago. And for that, I am thankful. And I know I’m not alone in thinking that.
A stroke couldn't stop her. Political chaos won't either. The 72-year-old songwriter talks protest music, resilience, and why age is just a number
spin.com/2026/01/lucinda-wil…
trenchtrapperonsol retweeted
#BTC Dominance Weekly attempting to set this up again finally
Every time it has been in position over the past 4 months it has sold off before the weekly close:
trenchtrapperonsol retweeted
aped some coins $ETH and $BTC
not sure how long im going to hold tbh, looking at some altcoin charts they somehow look even worse than november
structure has completely broken down for eg look at coins like $UNI, it just hit a new 4-year low even their rev-sharing update which is meant to be major major news
at this point i would rather cashout more and ape more stocks [nitter.cf/0xSleuth_/status/19864…]
onchain is still alive (with very short time-spans) but crypto alts have just been completely embarrassed and i dont see any reason as for how/why this could change
i know this is stating the obvious but im looking at charts of altcoins and they all look unfathomably ruined
especially if you look at some of the older tokens
if you were a holder of any altcoin (outside of a very small few) you have been ruined this cycle
crypto participants as a collective have agreed that alts are worthless and purely exist to sell at a profit to the next idiot
the believer culture that led alts to sky high valuations in 2021 has been obliterated, this is detrimental and something which cant really be reversed
each time someone on CT mentions this (something about taking profits early or that majority of coins go to 0) this trend accelerates, there are less believers and it becomes more common knowledge. It’s a self fulfilling cycle.
the only way i see this changing is a huge injection of liquidity from somewhere and then people for some reason choosing to gamble on crypto besides the various other methods like prediction markets, 0dte options, meme stocks etc. even then with the huge amount of coins retail would likely pivot to coins they are familiar with (dino coins)
i still think crypto is the place to be if you are a trader, but as a holder it has been extremely unforgiving
trenchtrapperonsol retweeted
MELANIA is now playing in AMC Theatres. Get tickets now: bit.ly/4c0Z8GY