Ben Goertzel

1) HOW BENEFICIAL AGI MAY ACTUALLY EMERGE, in 20 non-simple but feasible steps --

To be fair to @sama tho, I think what he said was that it was a waste to have so much RAM sitting un-used much of the time on peoples' idle home machines. He does have a point, but the solution doesn't have to be for all the RAM to sit in Sam's hyperscaler server farms. The more democratic and broadly beneficial solution is tools like ASI:chain and NuNet which allow idle processor and RAM on everybody's laptop to enter into decentralized compute networks carrying out processes of their choice. So folks are not asked to give up gaming in order to feed ChatGPT, they are offered an opportunity to leverage their hardware to contribute to processes they feel have value and make a few extra bucks or tokens along the way. The either/or conflict RAM use that is posed by Sam's framing is implicit in the centralized organization of the modern commercial AI field -- but like many other current either/or conflicts posed regarding AGI, it can easily be transcended if one takes a broader approach to AGI design, implementation, deployment and education... @nunet_global @SingularityNET @BGILabs

nytimes.com/2026/10/30/opini… OMG the common formulation of the "bitter lesson" is quite stupid I mean yes Alpha Zero is "brute force" ish in a way, but is also a genuine algorithmic and architectural innovation -- it's not like any random CS undergrad with a big computer could have trivially coded a standard RL algo to beat Go... It's like saying a brain just needs a lot of cells to be smart --- well, no ... it probably does need a lot of cells if it uses a standard biological sort of dynamics, but it also needs the right architecture and a lot of other well-honed particulars.   The fact that hand-engineering a brain from a small number of cells works poorly, does not imply that having a large number of cells is the whole secret to biological brains' intelligence... There is a difference btw a necessary condition and a sufficient condition, folks... (and also a difference btw a necessary condition and a "necessary to make things not extremely laborious" condition...) It is unfortunate that public dialogue on AI is stuck at this sort of elementary level, but unfortunately the style of discourse of famous AI big-shots does not always help the situation...

Nice article on how Amazon's human warehouse workers and warehouse robots are interacting in complementary ways, businessinsider.com/amazon-r… .... HOWEVER... the general tone is that: Whatever robots can't do in the warehouse right now, may be indefinitely far off.... It's true that with humans rather than AGIs driving robotics development, it will take years (rather than months or decades) for the remaining portions of warehouse work to get robotized... Once there is an AGI breakthrough, one suspects that the number of years needed for this final robotization to happen will be compressed to a quite small integer...

Will Big Tech +Doomers Succeed at Regulatory Capture of AGI? -- How the AI Safety Debate Could Deliver the short-term Future to Big Tech rather than the long-term future to Humanity TL;DR — it increasingly seems to me that the AI doomers and the Big Tech companies are now either conspiring, or else at minimum collaborating and coordinating, to bring about a quite dangerous regulatory capture of the whole AI industry. bengoertzel.substack.com/p/w…

Yes, absolutely. Anyone who thinks human academic paper reviews are stronger than AI reviews has not carefully and objectively compared recent examp;les of both. There should totally be expert humans in the loop at this stage, but it is not just OK but better at this stage to let well constructed AI agents do the bulk of the reviewing work... AIs do have a disturbing tendency to revert to the mean and over-criticize fringe/eccentric work. HOWEVER human referees have this tendency too -- and they are much harder to prompt out of it...

Super-Crude Simulative Explorations of the Geometry of Superintelligence Space -- Some evocative if very simplistic toy-simulation results exploring dynamics of goal-sets under self-modification and intelligence increase -- Just trying to get a verrrry rough quantitative feel of the basin of attraction size/ shape for supermind attractors like "selfish reward optimizer", "wirehead", "beneficial compassionate agent", etc. -- There would also be ways to run more realistic/meaningful such simulations even without having actual supreintelligences on hand! bengoertzel.substack.com/p/s…

What Sam is not saying yet is: There is a solution here which is to use coding agents to replace all the bad code (i.e. essentially all code) with code that is formally verified to have safety properties. This is a huge and difficult undertaking yet the same non-AGI-yet-superpowerful AI tech that is doing all the hacking is capable to enable this. If government wants to do something valuable to smooth the path to the Singularity, one helpful step would be to allocate a trillion dollars (oh maybe just try $500B for starters?) to open-source communities to roll out software for automatically replacing badly buggy code (again almost all code) w appropriately formally verifiable code... Gov't can find trillions of $$ to bail out "too big to fail" companies in crisis -- it did it for the banks in 2008 and will do it for Big Tech if the circular financing causes a tech-industry financial collapse in late 2027/2028 ... it can find massive $$ for wars... but finding hundreds of billions of $$ to end world hunger or give the world a solid software foundation is somehow utterly infeasible... We are not even to the stage of human-level AGI yet but we are already meeting fascinatingly and disturbingly unprecedented situations as modern AI tech meets legacy society and legacy IT systems. It's just going to get crazier, before ultimately -- if all goes well -- getting gloriously abundant for all... We need some "out of the box" thinking across all levels of society to get through this transitional phase with a modicum of grace amidst all the chaos...

new blog post by me, "The Harsh Geometry of Superintelligence-- The Selfish Supermind Attractor Generally Has a Wider Basin than the BGI Attractor" bengoertzel.substack.com/p/t… So ... in a series of prequel posts I have described several kinds of possible AGI mind, each with its own basic drives and each tending -- as it increases its intelligence and capability, to stay roughly the kind it starts out as.. Examples are: * the power-seeking maximizer of Steve Omohundro’s 2008 "Basic AI Drives" paper * the BGI minds that my colleagues and I are trying to build, ...and half a dozen others. Knowing what kinds of superintelligent minds are attractors under intelligence/capability increase is only part of the puzzle, though. The next question is: What are the shapes of the basins of these attractors? I.e, the zillion-dollar (actually, well, money-obsoleting…) question is: Which kind of attractor will a given early-stage AGI system fall into and how hard will it be to knock out again? In dynamical systems terms, this depends on how wide and how deep the basin around each one is. So this blog post analyzes these attractor basins a bit, with an aim toward milking lessons for building AGI systems and for understanding what an AGI system should do when something strange happens to it that may potentially zoink it out of its historical attractor basin. The disturbing yet unsurprising punchline here is: While Omohundro’s original “Basic AI Drives” paper overstated the omnipresence of the “selfish” attractor in the space of superminds … nonetheless it DOES seem that the selfish attractor has a big, wide and deep basin that makes it especially easy for AGI systems to fall into it … if they are not designed just right…. This doesn’t mean you need some sort of extremely carefully calibrated mathematical design to avoid convergence to selfishness – but it does mean that if you don’t bother to build things like compassion and goal-continuity into your AI system’s initial goal framework, you have no reason to expect your system to converge elsewhere than to a selfish Omohundro-esque attractor as it self-modifies and its intelligence and capability increase.

Bill Gates on AGI and its future is a bit conservative but basically rational and clear-thinking.... He also notes that one potential solution to AGI-ethics issues is a different/better foundational AI architecture which as he says "a few are working on" [raising my hand here ;p ...] ... theverge.com/podcast/1015408… However he thinks the tumultuous transition between here and ASI will likely take 20 yrs or so ... I suspect it will go a bunch faster... The headline is rubbish ofc, he does not think AI will make us live like the Amish ... he thinks that a certain percent of the human population will have an Amish-like attitude of "we don't have to use these new technologies" toward AGI/ASI.... I personally think this will be a rapidly decreasing percentage as the incredible value of AGI/ASI to improve life becomes clear... but it's a valid point for sure... this is what Hugo DeGaris called the Terrans as opposed to the Cosmists...

The Scope of Potential “Basic Drives” for Superintelligences is Likely Quite Broad bengoertzel.substack.com/p/t… This is the third in a series of posts trying to tell a more general and accurate story of superintelligence motivational systems, taking up where Steve Omohundro's classic 2008 "Basic AI Drives" blog post left off. The first two posts argued that whether an AGI, as its intelligence explodes, converges to a selfish ASI or a beneficial ASI, depends in pretty clear ways on how the AGI is initially architected. This post goes beyond these two examples (selfish vs. beneficial) and outlines a general recipe for a “basic AI-drive set” in its simplest form. Describe an AI system by two things: what it’s trying to improve, and what it won’t let be changed. What it tries to improve needn’t be a single number that it maximizes. It could be several objectives that it tries to push up together, or something as open-ended as an intelligence trying to increase its own individuation and self-transcendence. What it won’t let be changed could be nothing at all, or everything, or something in between: these are my goals and I’ll never change them; these are my safety checks and I’ll never give them up; etc. Now suppose the system can modify itself freely, and is free to use everything around it as a resource. As it gets smarter it will push its objectives up, make use of more and more of the options it can find, and hold fixed whatever it holds fixed. What you get in the end is a system that has optimized itself, digesting the resources available to it, into an extremely good machine for improving its objectives, one that never allows its fixed points to be moved. A fully selfish AGI fits this description. Omohundro’s maximizer tries to improve one thing, its reward according to its single goal, and it never changes that goal: successor after successor, as it rebuilds itself, the goal is kept. A BGI as I’ve described it tries to improve several things, one of which is care for others and respect for their choices, and what it won’t give ground on includes some basic ethical constraints on the path from where it is to where it’s going, constraints on the means as well as the ends. From that starting point come the drives the second post found: understanding itself, improving itself without high odds of doing damage, empowering others, and checking itself against independent others as it goes. Once you look at it this way there are plainly other possibilities besides the simplistic "beneficial vs. selfish" dichotomy, infinitely many in fact, and a few of them are recognizable enough.... Which is what this third post in the series is about...

Fun chat with my architect/maverick-physicist friend Brett Holverstott on the nature of free will, profanescience.substack.com/…