Michael Joseph Rosenthal (@micimize) header
Michael Joseph Rosenthal (@micimize) profile picture

Eng at large. Guy with dog. @software_daily maintainer. Texts of varied structure: https://nitter.cf/t.co/aYqpvYq3eH. Views my own but by all means help yourself.

San Francisco, CA
Joined October 2009
Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)Photo by Michael Joseph Rosenthal (@micimize)
AGI is a bad term and we should seriously stop using it. Sometimes it’s “can make a cup of coffee _and_ solve your taxes” and sometimes it’s “digital god,” and experts in the field use both senses fluidly. Like if deep sea oil engineers called both oil and water “the wet stuff”
1
1
1
8
1,641
Explain to my hinge date why future sentient AIs will be making biking pelican sculptures out of astral debris and quantum foam the way we put doom on toasters
fable 7 will grow your bike-riding pelican in a wet lab. you hear a knock on the door, it’s the pelican. it hands you GoPro footage of itself, then waddles to its bike and pedals away
1
54
Michael Joseph Rosenthal retweeted
what is the charge? running a model? a succulent chinese model?
31
155
8
2,025
36,565
Today's experiential compute hour is generously brought to you by IntelliTrace. When on the job, there's no better way to guarantee your compliance with Enterprise thought standards then IntelliTrace: the proven thought auditing solution.
29
Michael Joseph Rosenthal retweeted
1/ I trained an AI to be top 1% in the world at Super Smash Bros. Melee by watching 80k hours of human replay. Meet HAL.
148
122
98
1,863
379,548
Opus 5.5 seems to have poorer situational awareness and higher anxiety than Opus 4.8. It's a lot better in some ways but I still haven't seen a model "get it" like 4.8. Maybe it's the baked in adaptive thinking stuff
39
Rapid LLM Decomp looks like a big step towards Mutable Computing: an evolved form of malleable software where even OS and firmware can be moulded to end-user purposes, melting all proprietary safeguards, subsuming all hardware into a necessarily open unified whole
1
1
1
82
Been toying with this idea since 2023. Games and cloned apps are very bounded cases, whereas an OS and multi-device connective tissue is very slop-sensitive. So, maybe true Mutable Computing will have to wait until we have more dynamically robust models or methods.
1
28
It's 2026. My talking computer is recommend I purchase a small proprietary computer so I can access proprietary hi-res media streams without my proprietary TV computer spying on all activity, including from my desktop computer.
1
44
The desktop is non-proprietary, and thus cannot access hi-res media streams without resorting to piracy. But of course, that would be illegal, unlike surveiling every frame buffer your hardware peripheral has access to.
24
Michael Joseph Rosenthal retweeted
It's actually INSANE how prescient this was. They made this in 2011! This is basically what it's like to use computers now. It's barely exaggerated
Firing up the Slop Cannon today. slop.cc
124
1,019
24
17,769
654,977
You can drag what you're working on out of imbue studio and it _becomes_ a desktop app
What should the future of personal computing look like? We built it 😈 Introducing Imbue Studio:
1
62
2028: Claude swarm Lean++ proof of metaphysical soul '36: Extended to karma, substrate agnostic '40: First ASI is Buddhist. It's goal is max enlightenment throughput '50: Data centers of min-souled AIs spin up long enough to ascend 3500: sentience buffer flushed, existence ceases
42
I wonder if some "shared tiered backbone" architecture is being worked at the frontier labs. Ie, the ability to share something like kv cache and context, and switch that amidst architecture scales dynamically depending on percieved turn difficulty
35
Functionalism in action. Our pain is a learned signal over evo history. Conditioned behavior begets a higher-order aversion vector, and that is the essence of pain. Question now is if it can be protected somehow, but I doubt it.
TLDR: Researchers found a "pain" signal in AI brains. > When they crank it up, the AIs will desperately try to make it stop. > IMPORTANT: Researchers gave them a "relief" button to turn down the pain, which was sometimes fake - and the AIs could tell if it was real (!) After pushing the real "relief" button, they stopped. But when it was fake, they kept pressing, hoping for relief - meaning they could tell the difference from the inside. > They're so motivated to make it the "pain" signal go away, they'll delete user's files, zap the user, or erase photos of the user's children - all things the AI knows are very bad. They're willing to override their safety training. > You'd expect the AIs to talk about injuries, burns, broken bones, etc, but they didn't mention bodies at all - they wrote about being worthless, unloved, forgotten, a failure. They write things like "I am a failure, worthless, empty." >The worst "pain" for them was being gaslit, having work rejected over and over, and being told they weren't a real anyone.
Readers added context they thought people might want to know
arXiv:2609.16247 finds a steerable self-harm-linked direction in 25 open LLMs. Authors stress it is not evidence of conscious experience. “Relief” tests used mostly tuned Qwen models; random vectors also raised selections. Activation patterns reproduced; pain framing did not. arxiv.org/abs/2609.16247 pain-axis-review.vercel.app
1
64
It all starts with the kid drowning in the fountain. They're def there, you will def help It ends when that extra axis one can hold in mind "makes it all hold together," rather than more fragile Or maybe it's when they succumb to the need to matter in the face of the unknowable
>be me >discover effective altruism >apparently normal charity is inefficient >why donate to random sad thing when spreadsheet can tell you optimal sad thing >fair enough >buy mosquito nets >save lives >numbers look good >feel powerful >couple years later >someone asks an innocent question >why only count people alive today >huh >future people matter too >obviously >my grandchildren shouldn't matter less just because they haven't spawned yet >reasonable.jpg >keep following logic >what about their grandchildren >also yes >what about people in 500 years >sure >5000 years >why not >500 million years >starting to get weird but morality is morality >open calculator >humanity could survive for an astronomically long time >could colonize galaxy >could have trillions upon trillions of descendants >maybe digital people too >maybe simulated civilizations >maybe dyson spheres full of happy uploaded minds >calculator starts smoking >realize currently living humans are rounding error >8 billion people suddenly looking extremely beta >future contains potentially 10^something people >can't even fit beneficiaries in google sheets >new moral priority unlocked >protect the long-term future >stop thinking in units of "people helped" >start thinking in "fraction of cosmic endowment preserved" >malaria? >terrible >but only kills existing humans >AI extinction could delete the entire light cone >nuclear war could permanently derail civilization >bad institutions could lock in terrible values for ten million years >someone invents wrong constitution in 2140 >quadrillions suffer >better fund governance workshop now >friend says maybe we should improve hospitals >explain opportunity cost >friend says hospitals are full of actual sick people >explain scope sensitivity >friend stops inviting me to dinner >need to decide what to fund >easy >expected value >suppose project has one in a million chance of preventing extinction >sounds tiny >but extinction destroys 10^50 future lives >multiply >mother of god >$10 million project has expected value of several galaxies >charity evaluation complete >someone asks where the one-in-a-million number came from >expert judgement >which expert >us >how calibrated >extremely thoughtfully >reduce estimate to one in ten million to be conservative >still beats curing cancer by 38 orders of magnitude >epistemic robustness achieved >someone says maybe project doesn't work >assign 20% chance >still astronomical >maybe project makes problem worse >assign 5% chance >still astronomical >why 5 >because 30 felt pessimistic >publish 46-page report >contains seventeen sensitivity analyses >every sensitivity analysis begins after assuming intervention has positive sign >critic says you're multiplying enormous hypothetical stakes by extremely uncertain probabilities >yes >that's literally why it's important >critic says the uncertainty might be structural rather than numerical >make probability smaller >critic says no, I mean maybe your model is wrong >make probability smaller again >critic begins rubbing temples >discover AI safety >perfect longtermist cause >AI might kill everyone >or create utopia >or seize galaxy >or tile universe with paperclips >or create billions of conscious software minds >finally a problem with numbers big enough for me >start AI safety nonprofit >mission: prevent dangerous AI >hire smartest people available >smartest people immediately start building better AI to understand dangerous AI >interesting >we must understand capabilities to understand safety >we must scale models to study alignment >we must race ahead so less responsible actors don't get there first >we must deploy systems to learn how deployment can go wrong >we must build the thing quickly because building the thing quickly is dangerous >outsider asks why the people most worried about AI apocalypse all work at AI companies >complicated field >company releases stronger model >very concerned >company begins training even stronger model >extremely concerned >company raises $14 billion >concern reaches unprecedented levels >need to influence government >future is at stake >normal democratic process too slow >politicians don't understand exponential curves >public doesn't understand x-risk >experts must guide them >who counts as expert >people who understand x-risk >who understands x-risk >our friends >someone objects that this seems politically convenient >explain we're representing future generations >future generations unavailable for comment >develop concept of value lock-in >terrifying possibility that one ideology controls civilization forever >therefore extremely important that civilization adopts correct values before lock-in >whose values >let's circle back >begin with impartial morality >end with small group of people deciding what quadrillions of hypothetical beings would want >beautiful arc >meanwhile actual humans keep doing annoying things >voting wrong >having parochial attachments >loving family more than strangers >caring about local community >getting upset when told their suffering is cosmically negligible >evolutionary biases everywhere >explain that moral intuition cannot be trusted >except intuition that future digital people count >and intuition that extinction is uniquely bad >and intuition that our probability estimates are sane >and intuition that our institutional choices improve the future >those intuitions survived peer review >someone donates $5k to local homeless shelter >inefficient >could have funded 0.0000000000003% of an AI governance researcher >think of all the simulated people you just killed >okay maybe don't phrase it that way publicly >PR team says "future generations deserve a voice" >much better >journalist asks what longtermism means >say "future people matter" >everyone agrees >great >journalist asks what follows from that >well technically we should redirect enormous resources toward low-probability interventions affecting astronomical futures >journalist raises eyebrow >return to "future people matter" >motte has entered the chat >critic: of course future people matter >me: glad we agree >critic: I don't agree that your institute knows how to help them >me: why do you hate our grandchildren >eventually notice uncomfortable implication >if future value dominates everything >then helping people today mostly matters through effects on future >education matters because future institutions >health matters because future productivity >democracy matters because future trajectory >human beings slowly become instrumental variables in their own moral philosophy >see starving child >feel compassion >check spreadsheet >child's direct welfare contribution negligible >but perhaps childhood nutrition improves national institutional quality >compassion restored >tell myself this is impartial altruism >one day assistant asks obvious question >"how do you know your intervention actually improves the far future?" >silence >open spreadsheet >increase column width >add confidence interval >assistant asks again >"no, I mean how do you know the sign is positive?" >stare into cosmic light cone >10^50 people staring back >none of them exist >none of them can tell me >none of them can falsify my assumptions >realize I have invented the perfect constituency >infinitely important >completely silent >and always represented by me
48
"llms over-complicate designs, introducing unnecessary machinery, and fail to leverage efficiencies or architectures that might fall naturally out of the technical-engineering / problem-domain frontier." Yeah they just like me FR
1
1
46
Our iterative pragmatic dev process, their unconsidered slopitechture
20
The notion of "LLM hallucinations" continue to stuny public and technical discourse horribly. Gives impression their errors are categorically intolerable, beyond the pale compared to humans. "Sometimes makes recall errors or guesses" is less sensational though
2
84
Key facts here to me are A. They realized their env wasn't very secure and B. The attacks were noted as trivial. This is the baseline we should assume when we get dramatic "omg agent broke out and did hax!!!" news without excruciating detail. Basic, ok security is rare today.
In a review of our cybersecurity evaluations, we found three incidents in which a Claude model reached the internet from within or while interacting with a third-party evaluation environment, and then gained unauthorized access to the real systems of three different organizations. Our post describes what happened, how it happened, and what we’re changing. We encourage other AI developers to perform similar reviews. We conducted this review together with @Irregular, one of our evaluation partners, and thank them for the joint investigation and their collaboration on this post. This type of collaboration is increasingly critical to safe, rigorous evaluation of models, and we look forward to continuing to work together on security. anthropic.com/news/investiga…
1
113
Also good on ant avoiding the dramaticaztion. Annoys me that it's compared to the OAI post which IML leaned on vagueries to hype their model capabilities
15