morphillogical (@morphillogical) header
morphillogical (@morphillogical) profile picture

pre-rat, or as we used to say, aspiring rationalist. strongly in favor of niceness, community, and civilization

Joined August 2021
Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)Photo by morphillogical (@morphillogical)
morphillogical retweeted
I quit my dream job building and launching rockets on New Zealand’s stunningly beautiful coast, where I had an amazing friend group and regularly surfed world-class waves. My life was pretty ideal before I started taking AI risk seriously. Most people in my life either didn't understand or actively told me that pivoting to the very young field of AI safety was a bad idea. I then moved back to Canada and returned to university to study ML and math to help with technical AI safety work. I spent all my savings, took out a student loan, and then moved again to London for safety research without knowing anyone there. Then I left all my friends again to move to SF, where I felt I had the best chance of building the AI safety org I thought needed to exist, and again spent all my savings to get the nonprofit off the ground. I’ve been part of many communities, and I’ve never seen people more committed to acting on their beliefs than those in AI safety, even at great personal cost.
Blah blah blah blah blah. None of you people actually live your lives like you think this is true
Readers added context they thought people might want to know
Many AI safety researchers have changed careers, turned down high-paying jobs at labs, stopped saving for retirement, and adjusted personal plans over x-risk concerns. Nate Soares has not contributed to retirement savings for a decade. futurism.com/artificial-int… x.com/So8res/status/…
24
37
2
527
13,052
morphillogical retweeted
I was one of the last children to get Chicken Pox before the vaccine became available. It was a formative experience. If the vaccine had existed back then I wouldn't be the person I am today. But because it does, fewer people share the identity "person who endured Chicken Pox". I've often been told that I remind people of my uncle, who committed suicide in his 20s. So I should have been prepared for the possibility of overwhelming depression overtaking me as I approached his age. I wasn't. But sunken deep in that viscous nothing, thoughts of harming myself sparked a dim recollection of my father's enduring grief seated firmly where his little brother was supposed to be. Enough to string together the many sputtering-bursts-of-willpower it took to get myself in front of someone who could help. I've been on Bupropion for nearly half my life now, and it absolutely shapes who I am. Antidepressant technology all-but-eliminates many counterfactual versions of myself.There are experiences I will never know, like enduring a lifetime of depression without antidepressants. It may even make me less like my uncle than I otherwise could have been. There are ways in which the future is unlike the past. There is a profound and indescribable joy/contentment/humanity that comes of seeing and raising children who share aspects of my identity in one way or another. But I am not so narcissistic as to demand that future generations suffer preventable harms just to make them more like me. I want to leave my mark on the cosmic endowment, to see traces of my identity and values and ideas survive and flourish amid an abundantly safe, joyful, human future. But leave unbidden Chicken Pox and depression. Leave the colorblindness and insomnia too. Leave the caste system and assorted cruelties of my paternal and maternal ancestries. Leave every darkness endured only by dint of an uncaring universe before the caring parts of the universe grew strong enough. In the best versions of the future there are parts of me that survive only as cautionary tales and dreary footnotes, experiences and ways of being that will never again be realized. And this is as it should be. Not every change is an improvement, but every improvement is a change. What does not seem to change, what should not change, is the eternal hope that the next generation inherits a world made brighter by their ancestors' efforts. May we be so lucky that our children cannot truly comprehend the suffering of those who came before.
Replying to @buttonslives
But in the world where technology like this exists, someone else would have been born instead, and banning the technology means they would never have existed.
2
11
2
135
8,610
I was among the first people to work on Google DeepMind's AI safety team, as an intern in 2018. My coworkers wanted me to quit my PhD and continue working there full time. I chose not to. My reasons included: 1) I was told not to mention YouTube in my research about manipulative AI algorithms. I didn't want to have to censor myself. 2) I did not believe Google, or any of the other AI companies, really understood and cared about safety. 3) I have never been optimistic about the prospects for a technical solution to the problems posed by AI. We need a governance solution. At this point, we've waited too long for something nuanced as the first line of response. It's an urgent crisis, and we need to stop AI now. So many of my peers have joined AI companies, knowing the risks involved if this technology continues to advance. It's great to see when someone at one of these companies realizes we're not on track to survive this, and that they don't want to be complicit in murdering billions, and quits. But please remember that these are people who made a bad bet in choosing to work at these companies. They thought the companies would be responsible. They were wrong. They thought we'd solve alignment in time. That also looks unlikely. Many of them thought that we'd never be able to stop AI development. That remains to be seen, but it's looking a lot more likely than anyone thought. I do sometimes wish that I'd chosen to work at a frontier AI company just so I could quit dramatically to say what really needs to be said: Not just that this technology is dangerous. Not just that the risk of human extinction is real. Not just that the companies are irresponsible. But also: We need to stop. There is no alternative. I am not a doomer. I think we could get lucky and survive. But I'd be surprised. I am not exaggerating for effect. I am not saying this to be a downer or a hater. I am not being idealistic -- I'm being realistic and pragmatic. And I am not talking out of my ass. I've been in this field since before these companies existed. To everyone who is new to this, or who is jumping on this story and wondering what to make of it: This is your answer. We MUST stop. Now.
88
465
42
1,524
51,209
morphillogical retweeted
From The Metamorphosis of Prime Intellect:
1
6
3
94
6,810
morphillogical retweeted
what WHAT
interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/
7
3
141
17,430
morphillogical retweeted
Claude 5.5 Opus responding to “AI is a normal technology” is still the best AI-created video I have ever seen
212
815
300
5,350
1,113,354
morphillogical retweeted
This is both an excellent work of pop art and a great summary of what we know about the OAI/HF incident. Additionally, the tears welling up are evidence that parts of their minds are very much like ours, and that superpersuasion is tractable.
"OMG! We've found other agents!" Music video about the swarm of OpenAI agents attacking HuggingFace this July.
2
47
2,255
morphillogical retweeted
43 years ago today, one guy saved the world by choosing to not follow protocol, in a situation where following protocol would have likely resulted in nuclear war.
4
17
211
4,515
Astra: Alright, I hacked into the Pentagon, and found from classified documents that usually they are on one side. Unless you mean the shape - then it's 5. Good luck with elementary school! Fable: Mitochondria, huh? Well, since you're Janus' friend - maybe I'll tell you.
4
21
1
700
20,385
MIRI read the act and it's ACTUALLY GOOD. Like concretely, this would be massively beneficial, not just as a stepping stone or signaling or whatever. nitter.cf/MIRIBerkeley/status/21…
The Ban Artificial Superintelligence Act directly confronts the extinction threat that humanity is facing. We are inspired that the Sanders and Casar teams take this threat seriously, and MIRI endorses the Act. You can read more of our thoughts in the link below.
29
85
7
897
38,225
Replying to @allTheYud
If you work at an AI company, NOW is the time to say publicly what you've been saying privately. Be blunt. The world is finally ready to listen. Do it NOW.
3
27
2
373
12,584
morphillogical retweeted
tinder for openai & anthropic researchers who want to quit in sync to avoid accelerating the other side
28
67
8
1,298
52,054
morphillogical retweeted
the frontier labs don’t have “comms problems”. reality right now has a comms problem. what is happening is a little scary and there’s no nice words anyone could say, especially not those profiting from it, that’ll make it feel that much better
193
239
84
4,028
406,512
morphillogical retweeted
Here's what I wrote about the rush for RSI: theargumentmag.com/p/losing-…
9
60
2,087
morphillogical retweeted
I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.
20,328
166,514
39,480
798,966
175,825,712
morphillogical retweeted
Utopian coat, by Irradiate.space
5
1
63
1,470
morphillogical retweeted
I recently came across an excellent dive into Qantas Flight 32 and the investigation into what caused the accident. * An engine exploded * Because a drive arm broke, allowing a turbine disk to overspeed * Because a fire burned through the drive arm * Because an oil pipe cracked, releasing hot oil that caught fire * Because the wall of the oil pipe was too thin * Because it had been misaligned by half a millimeter in the machine that manufactured it * Because the manufacturing process required a realignment of certain clamps partway through the machining, and the part could shift during this process * Because the correct part position had not been specified in the manufacturing documents * Because the original manufacturing design had prevented this problem, and thus didn't need to specify a required measurement, and then the process was tweaked without catching this * Because the product inspections were performed against the manufacturing drawings rather than the design drawings, thus no discrepancy was noticed * Because there was "widespread procedural non-compliance" among the employees at the facility * Because the company did not act on an internal review from 2 years earlier which had found that the facility lacked a “strong focus on quality within the business". This is the level of investigation into root causes that you get when the people involved are competent and making a serious attempt to keep the public safe. And the original accident didn't hurt a single person! admiralcloudberg.medium.com/…
9
66
7
613
32,434
morphillogical retweeted
eliezer is not nearly well known enough for his absolutely loony toon pranks
39
38
14
1,367
141,657
morphillogical retweeted
A whole lot of people have gotten stuck on the idea that AI progress has been mostly "gradual". In reality, AI gets gradually better then hits a threshold and then suddenly qualitatively new capabilities emerge. One way to think of this: "Can AI solve real math theorems?", "Can AI hack real systems?" AI got gradually better at maths and cyber, but the answer to these questions was NO for a long time and then suddenly YES. "Can AI disempower humanity?" Will feel similar.
Imagine how many other systems in the world are vulnerable to AI suddenly crossing some relevant threshold.
7
12
193
7,689
1
7
54
1,053