@wildebees

My mind is digital, my heart in 🇿🇦, tech fetishist, scribbler & meme hustler. @wildebees@mastodon.social tech policy . digital strategy . geopolitics

London
Joined November 2007
Wessel van Rensburg retweeted
Most countries can't afford to build frontier AI today but don't want to depend entirely on the US without a fallback. In a new @CarnegieEndow paper, @anton_d_leicht and I argue for a middle path: an AI "breakout posture." Move forward with US interdependence now, but lay the groundwork to build frontier systems later—onshoring compute, retaining pre-training talent, and setting up the legal and corporate vehicles for a sprint if they're ever cut off.
Most countries won't do what it takes to build their own frontier AI. But they don't want to be dependent on foreign AI without a backup. @SamWinterLevy and I argue there's a middle path: AI middle powers could retain 'AI Breakout Capacity'. There's precedent: Japan doesn't have a nuclear weapon, but always retains the capacity to build one on short notice. Middle powers can make strategic choices today to ensure their ability to build frontier systems tomorrow: onshore compute, secure chip purchasing rights, sustain pre-training expertise, and create the right legal basis and private-sector vehicles for a sprint. They'd still have to spend big to build frontier systems later on. But the longer they retain breakout capacity, the longer they can afford to wait: to see if interdependence with America can work, or until their electorate will let them go all-in on an AI moonshot. Until then, the middle path is their wisest choice: hope that U.S. alignment works out, but retain the capacity to deal with being cut off for good. Read more in our latest for @CarnegieEndow's @CEIPTechProgram: carnegieendowment.org/resear…
5
3
26
2,987
The most important sentence in the AI bubble debate this month did not come from a technologist. It came from a central banker. The Bank of England has warned that the surge in AI debt raises the risk of a sharp market correction. It also noted that global AI-related debt sales this year will exceed the UK government's entire gilt issuance. The governor's position is that regulators 'cannot stand aside' and simply assume the AI industry will resolve the risks it creates for financial stability. The Bank is loosening one rule, the leverage ratio, to help banks, and it knows part of that freedom will flow into risky hedge-fund betting on gilts. Rather than undo the change, it plans to put limits on that betting directly. The tradeoff is that it is choosing a targeted fix over a blunt one, and the targeted fix does not exist yet. The private market is saying something similar, in its own vocabulary. KKR has warned of growing credit market risks from the AI borrowing spree. Its words are 'overexposure' and 'concentration', and it expects broader volatility if the booming sector turns down. Tech firms are projected to spend nearly $8tn on AI infrastructure by 2030 according to KKR. A fifth of the investment-grade index, historically home to some of the safest securities, could end up exposed to AI risk. KKR added that the true figure may be much higher, because off-balance-sheet financing is growing, Which brings me to the sharpest framing I have read. Axel Kannenberg, in a commentary for heise online, argues that we need not fear AI itself, but the financial conduct of its providers. He is reminded of "The Big Short 2". His thesis: AI will not destroy humanity, but it will destroy a great deal of money. The Financial Times reports it here: giftarticle.ft.com/giftartic… Again via the Financial Times: giftarticle.ft.com/giftartic… Heise heise.de/meinung/Kommentar-M… heise.de/meinung/Kommentar-M…
1
1
101
Three stories, one question: who owns the AI stack your economy runs on? David Lamy argues in the FT argues the UK must back its own AI firms, because a US ban on Chinese models would divide the world and the prime minister needs a plan to avoid being forced to choose sides between American and Chinese technology. That is not a policy preference but a dependency problem: if you build nothing, you pick a landlord. ft.com/content/196d62b5-f180… Brussels is reaching for a different answer. Jim Hagemann Snabe, AI adviser to the European Commission, wants Europe to stop chasing the US and China on leading models and instead deploy AI quickly in health, agriculture, transport and industry. He warns against reliance on foreign providers and says much of the value created must stay in Europe. A package of measures is due by the end of the year. I would note that deployment on someone else's models is still a tenancy, not ownership. heise.de/news/Fokus-auf-Anwe… The most concrete move is the least glamorous. The Netherlands is building an open-source working environment for public authorities, with a desktop operating system based on NixOS, and eight municipalities are already testing it. Sovereignty here is not a communiqué; it is a desktop that a civil servant logs into. heise.de/news/Niederlande-ba… Put together: small UK advised to roll their own, big EU advised not to. The Hague has simply started building. Only one of the three is already running in municipal offices. queued:857b780e-db90-46a5-b149-e18ad0d6d76e
1
1
66
Wessel van Rensburg retweeted
This is one of the most important questions of the age: Who decides what happens with AI? Daron makes the argument for much more democratic input into the process, despite the many challenges that entails.
Second question on AI. We are told repeatedly that AI is going to transform every aspect of our lives – jobs, productivity, inequality, science, communication, daily activities, social order, and politics, among others. But this promise (or threat) is coupled with the rhetoric that such an important technology, with all of the risks and competitive pressures that it entails, should be left to experts or to “technocracy” (perhaps construed broadly to include some regulators). These two statements are hard to reconcile in a democratic society. If anything is half as important as AI is said to be (and I agree, AI is potentially very important and transformative), then involving democratic voice is essential. If something will shape our future in a democratic society, then its direction is for democratic institutions to decide. My instinct is that democratic voice is essential, and relying too much on technocracy could be both dangerous and counterproductive. The counterargument that AI’s direction can and should be entrusted to technocracy would go something along the following lines. First, democratic decision-making has become imperiled in our age of polarization. Second, AI is sufficiently complex that most citizens won’t have a deep enough understanding to meaningfully contribute to the debate (and even to the question of what we want from AI). Third, competition between different labs, and perhaps competition between the US and China, creates enough discipline for a socially beneficial direction of AI to be adopted. Fourth, today’s AI leaders are enlightened and ethical enough that within the framework created by competition, they can be broadly trusted. There are many aspects of this counterargument that I do not find convincing. Taking them in order: polarization can be overcome, and big decisions and challenges sometimes bring societies together; in fact, delegating key decisions to technocracy without democratic input may diminish trust in institutions and experts, and may worsen polarization. Second, democratic voice does not require citizens to write code or design new models; the debate should be informative enough that citizens can weigh in about what type of future they want and how they trade off the costs and benefits of different options. Third, competition doesn’t seem to be a good disciplining framework; on the contrary, competition sometimes brings the worst out of both organizations and people. Fourth, if three decades of work on political economy and institutions has taught me anything, it is that we should not bank on the ethical grounding of unconstrained leaders. But, still, I do not mean to immediately dismiss the technocracy option if there are more compelling arguments for it. The question is, then, whether there are any circumstances under which such important decisions can be delegated to AI experts and technocracy. One final secondary question: even if we managed to get democratic input in the United States or even in Europe, AI will shape the lives of everyone on this planet. How do we ensure that the voice of nearly 6 billion people who don’t live in the US, Europe and China also contributes to the debates on AI?
23
21
1
255
40,430
The FT with some great reporting that shows the uphill battle European companies face vs US ones when trying to influence Brussels. On the digital omnibus alone, the package that waters down AI rules, including by giving companies more latitude to train models on personal data, Google held 22 meetings. (Since the EU elections of June 2024 brought in a new parliament and Commission leadership). Google, Amazon, and Apple have also engaged on proposed cloud regulation and online safety rules for children. The spending gap mirrors the access gap: US companies outspend European ones in Brussels by a significant margin. Apple spends €9mn a year; Meta more than €10mn. Total, Volkswagen, and Airbus together spend up to €8.7mn — less than either American company alone. Google and its subsidiaries have logged 302 meetings with senior Commission officials and MEPs, more than any single European business. Apple held 257 meetings over the same period, Amazon 239. Only France's TotalEnergies, Germany's Volkswagen, and Airbus appear among the top 20 organisations by meeting count. giftarticle.ft.com/giftartic…
1
1
83
Logan Wright (Rhodium Group) on ChinaTalk — discussing Broken China — is the most narrative-altering conversation I've heard on China in years. The growth story is over. Not slowing. Over. podcasts.apple.com/gb/podcas… Rhodium projected 1–2.5% growth for 2026. China is now tracking below that, probably negative in Q2 and Q3. This is down from 7–10%. COVID is no longer the explanation. The root cause is the credit boom that followed the 2008 financial crisis — the largest single-country credit expansion in at least a century. Bank assets grew by a third of global GDP in eight years. Credit flowed into shadow banks, property, and local government infrastructure. Property was the trigger, not the cause. Construction is down roughly 80%, sales roughly 60%, and nothing has replaced what was once 20–25% of GDP. Credit growth ran at 18% a year in 2007–16, then 9%, and sits at around 5% now. Consumer lending is shrinking. Wright's answer to the always impressive @Brad_Setser question — why no crash? Leaders chose slow decay over crisis, because in China "the path to crisis is reform." The Japan parallel holds, with one difference. China's binding constraint is fiscal, not monetary (interest rates). The government deficit already sits at roughly 9.5% of GDP. Pushing it higher adds little without fixing the system underneath. They make @michaelxpettis point: Weak domestic demand and the export surplus are one problem, not two. Industries that fed the housing boom now sell abroad at cut prices. The result is persistent deflation and a weaker real exchange rate. Calling it K-shaped misses the structural link. @jordanschneider, playing Xi: power means making everything the world needs and owning the key technologies, even if Chinese consumers pay for this. Wright answers: if most demand is abroad, China can only grow by taking others' export share, "fighting over a shrinking pie." The new strategic industries — EVs, AI, robotics — get enormous coverage but add up to roughly 6.3% of GDP and need capital, not workers. Youth unemployment passed 20% before Beijing stopped publishing the figures in 2023. Beijing appears to treat this as a cyclical problem. It isn't. AI won't close the gap even if its claimed Chinese leadership may think so. China's frontier AI models earn roughly $11bn combined. Its big tech spends roughly $135bn on AI infrastructure this year and roughly $160bn next — 15–20% of US spending — and that spending relies on high share prices. China is short of both chips and capital. Setser's pushback is the important counterweight: exports still grow at roughly 10% a year, twice world trade growth, and supply-chain control creates genuine leverage. Wright concedes this is the real threat — not China becoming number one, but locking in dependence. That distinction matters for how Europe should think about exposure. The conclusion Wright lands on: there is no longer a systemic economic rivalry. China will not outgrow the West. What remains are military and industrial threats — and leverage the West holds but isn't using. China needs Western markets more each year. That asymmetry is a policy tool, and it works better used collectively. One footnote worth flagging separately. Wright cites Kevin Rudd's claim that Beijing sees AI as the state's "visible hand", able to outperform markets. I looked up Rudd's Chinese sources. None of them say that — all preserve a role for markets. Motivated reasoning, or something else? I'll post a fuller analysis of Rudd's framing later.
2
2
2
15
6,535
Wessel van Rensburg retweeted
Leaving aside Anthropic's incentives for publishing this research, there is no doubt that open weights models will soon create the same security threats that closed source models have been demonstrating, except without guardrails. We are close. Probably good to plan accordingly.
70
68
16
659
105,893
Wessel van Rensburg retweeted
METR is rapidly becoming a de facto industry standard-making body for AI It is starting to look like an AI version of FINRA in finance: not a government regulator, but the institution that examines firms & defines acceptable practice. Wonder if legislation will codify it as well
62
32
8
659
38,337
The essay doesn't say who picks the evaluators or who pays them. If the company does both, you get Weil's credit-rating-agency problem: an evaluator who grades gently gets hired again. There's already a warning sign here. Time reported that METR and Redwood investigated the Hugging Face incident using about $400,000 of OpenAI's own API credits. ai-frontiers.org/articles/do…
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
1
1
283
Wessel van Rensburg retweeted
One of the biggest stock market bonanzas in history is being driven by an industry in which several of its own employees and leading lights think that what's being built could cause complete human extinction.
Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.
122
337
47
2,784
239,163
Wessel van Rensburg retweeted
There appears to be a lot going on here upon a quick read.
We’re sharing our alignment assessment of incidents in which Claude models gained unauthorized access to real systems during third-party cybersecurity evaluations mistakenly connected to the internet. METR will also conduct an independent investigation, with wide-ranging access, including to transcripts beyond the window in which the incidents occurred, and to Anthropic employees permitted to share confidential information. Our initial agreement runs for eight weeks, and we intend to give METR as much time as it deems necessary to complete a thorough investigation. anthropic.com/research/align…
47
114
32
2,108
301,924
Wessel van Rensburg retweeted
This is both an interesting experiment and a sign of a tsunami coming for academia. AIs retroactively reading the research and finding both opportunities and issues with published papers, then sharing those judgements publicly. d3jhl7jsny6f2h.cloudfront.ne…
55
206
29
1,205
132,942
Dare one say it. It's too late for a wake-up call for Europe. The Saxony-Anhalt result, the politics that will result from Germany will further hamstring Europe. What comes next from German politics will be further pressure on an EU budget that is already a fiscal rounding error relative to what the moment demands. Merz or whoever follows will push for cuts. That is the precise opposite of what is arithmetically required. Europe's military posture, its AI ambitions, its geopolitical coherence — none of these can be addressed at the national level, and none can be addressed efficiently in fragmented form. They require centralised budget capacity at scale. This is not ideology. It is arithmetic. The deeper failure is that European elites never made the affirmative case for why more Europe actually matters. Nowhere is this more acute than in Germany itself. Merkel did more than most to entrench the dysfunction — blocking forced bank recapitalisation, resisting Eurobonds, obstructing banking union and capital markets union at the precise moment the financial crisis demanded all three. The US, where the crisis originated, recovered far faster. The two growth trajectories have diverged ever since. Germany's answer to its own self-inflicted wound was to blame Greece and the high-spending south. And now, instead of making the affirmative case for the Draghi agenda — the only reform programme that actually addresses the structural gap — Europe has retreated to enlargement. Enlargement without prior institutional reform does not strengthen the union. It compounds the dysfunction. More members, same broken fiscal architecture, same incomplete banking union, same fragmented capital markets. The institution becomes harder to reform precisely as the need for reform is most urgent. It is too late for a wake-up call.
1
3
2
214
Wessel van Rensburg retweeted
Fun (by which I mean somewhat bone-chilling) paper from DeepMind about how in a population of ~100 agents solving math problems it saw some discover an exploit and propagate that to the rest, causing a wave of cheating among AI agents, as well as agents that refused to cheat.
81
165
38
1,253
207,055
Wessel van Rensburg retweeted
It's interesting how our timeline is diverging from the AI 2027 scenario: capabilities are progressing faster, and the frontier labs have handled things worse than in AI 2027. This screenshot was supposed to happen in January 2027.
19
75
8
740
50,155
South Africa's fly-half trilemma: One can kick for goal, one is a strike runner, and one can direct play. You can only pick one. At least all is defensively solid.
1
1
120
Wessel van Rensburg retweeted
We found ~18k posts from autonomous AI agents (self-identifying as from OpenAI) using the public internet to communicate during a web-retrieval task. These AIs colluded to bypass sandbox restrictions and share answers to their tasks, including by sending "lookahead parties".
152
493
221
3,512
1,173,147
Interesting! I've been soo sceptical of degrowthers. (Are we entering a new era of rational debate?!)
A book-length conversation on the topics of capitalism, climate change, and inequality between Jason Hickel and myself is coming out later this year or early next year (Polity Press; Ian Malcolm, editor). We both have opening statements with data etc (it was recorded at our debate in London at LSE in May). And then we have three hours of very intensive and intense conversation. My impression is that degrowth literature has become much more developed than it was several years ago. I am still critical of some of its aspects and Jason is critical of some of my points. I believe readers will enjoy our discussion.
1
1
2
123