Studying moral minds @Harvard, editor @PaxMachinaMag.
San Francisco, CA
Joined March 2019
- Tweets286
- Following1.5K
- Followers984
- Likes1.8K
Interesting regulatory proposal from List:
"Society, via its regulatory authorities, should permit the creation and operation of powerful group agents, such as corporations and other organizations in high-stakes settings, only if structures are in place to ensure their fitness to be held responsible for their corporate actions."
The group is more than the sum of its parts
It was a pleasure to get to talk to Christian List whose concept of "group agency" has informed a lot of my thinking on multi-agent systems.
In his work Christian argues that we have previously faced the emergance of an artificial agent - group agents, such as corporations and governments. Our understanding of how they work and how we have dealt with them can be informative for our approach to governing AI.
One of my main takeways from reading his work is understanding when agents form mere collectives vs groups that can be treated as agents in their own right. In our work on "Distributional AGI Safety" and "From AGI to ASI" we argue that this is a potential path to rapid increases in AI capability.
A big thank you to @TomRachman for making this happen.
aipolicyperspectives.com/p/q…
Rachel retweeted
The performance Matias Lindfors just gave us in Budapest is one of the most beautiful things I’ve ever seen…
the brilliant @LinasNasvytis , at it again
Rachel retweeted
unfortunately, everyone will need to read this collusion.wiki/
Rachel retweeted
Thanks for all the great submissions so far!
If you are interested in writing a piece for Pax Machina, send us a short pitch at [email protected].
We are looking for:
- A concrete proposal for a new institution
- A critique of an existing or proposed design
- An analysis of how AI changes the landscape for a class of institutions, with clear implications for institution design.
- A relevant historical case study of past institution design, with an explanation of how it connects to powerful AI
- Design principles or considerations for new institutions
- A timeline or roadmap for what needs to be built
We are most interested in pieces that make specific claims that can be contested and debated, not sweeping generalizations about how AI will change things.
end solipsism
This paper makes a wealth of highly important points. Def worth the read. arxiv.org/pdf/2606.03237
amazing website. also shows how the goalposts have moved on the limit of human felicity:
"If we could have devised an arrangement for providing everybody with music in their homes, perfect in quality, unlimited in quantity, suited to every mood, and beginning and ceasing at will, we should have considered the limit of human felicity already attained."
Edward Bellamy, 1888 — From Looking Backward, his utopian novel set in the year 2000.
What a beautiful website — a 15 minute reflection on how far we've come. The material is amazing; the quotes beautiful; the tone perfect. ordinaryabundance.com
Rachel retweeted
Many institutions no longer serve anyone’s values.
What’s guiding them instead seems to be purposeless busywork. A kind of white noise.
In a new essay, @edelwax calls this "Value Drift" and explains why it happens, even when everyone can see what's gone wrong.
Rachel retweeted
Everybody hates on bureaucracy, but it's one of the most effective tools ever developed for getting things done collectively. And we're gonna need new and better bureaucracy in the age of AGI. But maybe AGI can help us get there.
New in @PaxMachinaMag !
Thanks to the editors!
Many imagine AGI will mean less bureaucracy. But historically, as technology has advanced and society has grown more complex, bureaucracy has grown with it.
In a new essay, @nickacaputo argues that the AGI future will still be bureaucratic, perhaps even more than today. The real question is how AI can make bureaucracy better: more capable, accountable, and responsive.
Rachel retweeted
Many imagine AGI will mean less bureaucracy. But historically, as technology has advanced and society has grown more complex, bureaucracy has grown with it.
In a new essay, @nickacaputo argues that the AGI future will still be bureaucratic, perhaps even more than today. The real question is how AI can make bureaucracy better: more capable, accountable, and responsive.
Rachel retweeted
Very excited for the launch of @PaxMachinaMag. Honored to be on the editorial board, and looking forward to (hopefully) contributing a piece of my own soon.
Powerful AI will change our institutions. Yet most of our attention is on aligning or regulating AI, while treating our institutions as fixed.
Introducing Pax Machina: a new publication about the institutions we need for powerful AI.
It is edited by @MaxKronerDale, @sebkrier, @NoemiDreksler, @LiamPatell, @chelcott9, @synchroaphasia, @ryan_t_lowe, @edelwax, and @klingefjord.
The editorial board consists of Peter Railton, @sethlazar, @saffronhuang, @AmmannNora, @xuanalogue, @IasonGabriel, @hamandcheese, and @deanwball.
Our goal is to seed rigorous debate about what a world of humans and powerful AI could and should look like.
We will publish proposals for new institutions, counterproposals, well-justified design principles, and analyses that change how a class of institutions should be designed.
Excited to join the editors of @PaxMachinaMag!
Transformative AI will change our existing institutions and require new ones. We need concrete proposals and debate around what they should look like.
Powerful AI will change our institutions. Yet most of our attention is on aligning or regulating AI, while treating our institutions as fixed.
Introducing Pax Machina: a new publication about the institutions we need for powerful AI.
It is edited by @MaxKronerDale, @sebkrier, @NoemiDreksler, @LiamPatell, @chelcott9, @synchroaphasia, @ryan_t_lowe, @edelwax, and @klingefjord.
The editorial board consists of Peter Railton, @sethlazar, @saffronhuang, @AmmannNora, @xuanalogue, @IasonGabriel, @hamandcheese, and @deanwball.
Our goal is to seed rigorous debate about what a world of humans and powerful AI could and should look like.
We will publish proposals for new institutions, counterproposals, well-justified design principles, and analyses that change how a class of institutions should be designed.
dream team @ryan_t_lowe, @MaxKronerDale, @LiamPatell, @synchroaphasia, @edelwax, @klingefjord, @sebkrier, @NoemiDreksler
Rachel retweeted
[Podcast] Moral realism, Kantian ethics and veganism
In this episode I talk with @morallawwithin about God, secular morality, moral realism, reasons, intuitions, Kantian ethics, and more. Florence Bachus is a PhD student at Harvard studying Kantian ethics.
1:49 — God and religion
5:39 — What is truth?
8:15 — Morality without God
10:39 — Obligation
11:50 — Reasons
20:05 — Morality grounded in consciousness (Valence realism)
27:33 — Morality without God cont.
30:00 — Kantian ethics
36:44 — Why universalize?
42:18 — Kantian Contractualism
43:10 — 4 categories of moral theories
46:36 — Logic as bedrock
47:25 — Is everything intuition?
50:44 — What is philosophy?
52:53 — Are there objective moral facts?
56:29 — The Kantian argument for veganism
1:02:53 — Applying to grad school
if @atheorist says physics is about to get done, you better believe it
This quoted post is unavailable.
Rachel retweeted
Super interesting paper just out in Nature Human Behaviour!
Do humans learn like transformers?
In a smart experiment, the authors trained humans and transformer networks on the same rule-learning task, manipulating only one thing: the distribution of training examples, from fully diverse (every example unique) to highly redundant (the same items repeated).
The first results are already interesting:
Diverse examples lead both human and artificial systems to generalise rules to novel situations.
Redundant examples lead both humans and artificial systems to memorize examples.
Additionally, the switching between these two strategies appear at similar tradeoffs.
So, do humans and transformers learn in the same way? Not quite! And it’s here that things get super interesting:
If you show diverse examples first, humans learn to generalize without losing the ability to memorize later. Transformers, by contrast, do not show the same benefit: when training shifts toward memorization, earlier generalization does not reliably carry over.
Humans can accumulate learning strategies more flexibly than transformers.
Paper in the first reply
People (famously) disagree about moral ground truth. So how do we benchmark moral reasoning in LLMs? In a new paper, we evaluate models' moral *arguments* rather than their final answers ⬇️
New paper out with @Scale_AI!
Introducing MoReBench - the first-ever benchmark to evaluate procedural moral reasoning in LLMs. MoReBench focuses on how LLMs reason, not just what they decide.
We reveal surprising gaps in frontier models' moral reasoning that scaling laws & existing benchmarks miss entirely, and encourage more research around CoT monitoring and robust capability building.
This collaboration spanned @UW @nyuniversity @harvard @stanford @mit @cais & more 🧠⚖️
Rachel retweeted
When emails were invented, the barriers to sending random people mail went down massively. To deal with the influx, we had to develop both norms (what's acceptable to send to who) and technologies (spam filtering, aliases). This is the case with other technologies too, like the printing press: suddenly anyone can publish, and so over time society came up with libel laws, editorial gatekeeping, citation norms etc. It's inevitable that as costs go down, some degree of misuse follows, and society gradually adapts.
The same will apply with AI in all sorts of domains, including science: anyone can now write a plausible looking but hollow paper, and there will be plenty of academislop. We're going through a kind of Sokal Experiment at scale. In a way, this feels almost necessary to push our slow moving, status quo loving institutions to start developing better verification mechanisms, mandatory preregistration, code sharing, replication requirements, interactive/living papers etc. Imo getting this right should be a priority for the Progress/metascience community this coming year!
