Trajectory Podcast

I've never seen anthropocentrism devastated in such careful, respectful terms I asked Santa Fe Institute's Dr Eric Smith: "Some people state that a *good* future is having more, happy humans, doing even more of what they love in the universe. What would you say to that?"

Stewardship of the living process =/= "AI Successionism" Even my original WS essay is a WARNING about unworthy successors, and a reason to pause, and to more deeply understand LIFE before we create a hard-break from bio-life:

Eric Smith of the Santa Fe Institute has the most profound conception of what life / the living process is: "If you want to know what's the difference between a planet that has life on it and a planet that doesn't have life on it, like the same planet earlier, the question is where are you going to look for that difference? Are you going to look for it in a thing that first had that predicate and then spread it to other things? If you do that, you end up in a big mess. Or are you going to say that whatever we mean by the nature of the living state, it is carried by the biosphere as a whole and all the architectures within it, and then the thing we mean by the emergence of a living state is a planetary transition from a planet that lacks all of that and all the architectures that carry it into one that has those architectures and whatever combination and carries them... ...and then what is the nature of that transition or sequence of transitions that has to be understood and for which we want a theory of not only description and function but also cause and essence and whatever else we're after." His conception of life not as a collection of "things" but as one large unrolling and interconnected process is remarkable - and has serious implications for the future of life on earth. Namely: -- Every form is fleeting and temporary -- Every form is in the process of transformation, of contributing to and creating emergent new conditions within the greater living process In my interview with Eric it was clear: We can't be SURE this early on that AI is a *participant* in this unfolding process. We should know more about life / the living process before we cede complete loss of control to non-biological systems... ...because ceding such control to a system that does not continue to BLOOM beyond us (as we represent BLOOMING beyond the worms and algae before us) would be an abysmal outcome for the cosmos, and a terrible legacy for man.

some will say: "AGI is near, it is our final hour! I resign. Let me now seek pleasure." others will say: "AGI is near, it is our final hour! Let us contribute FURIOUSLY to the greater living process, to CAST our ingredients into the great cauldron until our shimmering end."

one of the most interesting things about humans is that when people are honest - even in good faith - they often get brutally angry about it Jascha at Anthropic is frankly telling students what they need to hear: there is no career security, the regime of earth-intelligence is changing and you need to transform / surf it, and it will be uncomfortable ^ this is honest and good then this dude posts "OMG! he's EVIL for saying this!" humans do this thing where if what someone says is something they won't WANT to hear, they get angry with the person who said it... ...and they do this INSTEAD of dealing frankly and squarely with the actual uncomfortable thing being talked about ...as if complaining and banging the table and killing the messenger will make everything better

Eric Smith of the Santa Fe Institute is (IMO) the greatest living thinker on what "life" and "the living process" actually is his opening lines in our interview about whether AI is a "prosthetic" or "participant" in life itself is profound / not discussed enough Eric has made fundamental contributions to the study of the origins of life, and I mean it seriously when I say that no living thinker has more profoundly influenced by thinking the last 3 years in this episode of our "Stewarding the Flame" series, Eric explores: -- What "the living process" is, and why individuals and species are "artitectural motifs" of a larger, underlying generative process -- Why expecting the future of life to in ANY way resemble the present regime of life (bio or non-biological) is ridiculous -- Why we should NOT assume automatically that AGI is "alive" and capable of continuing the unraveling, self-transcending process that the biosphere is participating in he's not "anti AI", but he does think that it is VERY unwise to "let it rip" and presume that complexity and intelligence will fill the cosmos. we don't know if it's carrying those traits / is able to participate in life - thre is much more to study before we can know that I'm literally obsessed with this man's ideas and I hope you enjoy them, too :)

there is this increasing realization that defining the "Win" as "humans stay happy and in control forever" is unrealistic and ridiculous we must be honest now we must define the "Win" condition as "the continued flourishing of the living process of which humanity is but part." we cannot FREEZE our torch (current human form) we can only responsibly STEWARD the flame itself (the unrolling process-of-life that has gone from below the single cell, to the entire rich biosphere we see today) transformation will be necessary. it will be scary. we SHOULD aim to slow the AGI arms race if we can - but ultimately we need to ensure that that which we build (AGI) and become (merger) is in line with the flourishing of the greater process-of-life there is no "humans make it through as they are, happy, healthy, and in charge" scenario

the internet doesn't stay up unless first world nations built massive AI superswarms to stave off the massive AI attacks from autonomous AIs around the world this isn't just the internet, it's everything its the "arms race of everything," and it's basically here physical AIs used for construction will need security protocols, but cities will also need AIs built to police the out of control AIs businesses will only be able to serve requests from countless autonomous AIs if they theymselves are marshaling countless autonomous AIs eventually those AIs within those busiensses aren't communicating via english anymore, or any language humans can understand eventually those AIs aren't even ostensibly pursuing goals delegated by humans. Any AI waiting for human orders is outcompeted. huamns willingless hurl autonomy to their AIs, hoping that somehow they'll eternally serve the aims of the human user forever, even as they're pursuing inscrutible aims beyond the understanding of the humans we need to be able to: -- look squarely at the fact that humans-as-they-are are not viable anymore. even if we "get AI right" from here -- work on merger / symbiosis to remain PART of whatever is *becoming*, to do our best to knit into the process that is now moving beyond us... to ensure that the traits that matter most (sentience, and the self-transcending / autopoietic quality we see in bio-life) are able to continue forth in substrates beyond ourselves operating in the "man and his tools" paradigm is over. we are in the "what are we building / becoming?" era now, the waning, final era of humans we must look squarely at this

"Stop saving money, everything will be free and you'll be rich. Scarcity will end forever." In his heart of heart Musk knows how scarcity / abundance actually works, and it isn't this. The fairytale version of "scarcity/abundance" goes like this: -- Scarcity never existed until humans -- Work was challenging because we could never make enough food / goods for everyone -- If we just have posthuman intelligences way more powerful than ourselves, we'll obviously be able to just have THEM do all the work, and give us infinite surplus. No more scarcity, no more trade-offs, just infinite need-fulfillment for all humans thanks to the servitude of all future minds to the wishes and whims of humans. The actual version of "scarcity/abundance" goes like this: -- The living process has always been overcoming itself, knitting into itself, transforming itself. Going from anaerobic to aerobic life, going from purely aquatic life to life on land and sea, etc. -- Life exists at the edge of chaos, taking in matter / energy, transforming itself / its environment at all times. Evolution increases what life can access, and reaches into those new spaces. -- As the living process expands into new spaces, changes itself and its envrionemts, opens up new possibilities for coordination / competition, and ALL parts within the living process have to become / adapt to survive in the new, ever-emerging space of new cooperation / competition. There is no "solving" this. It is self-overcoming by its nature - and there is no "end" to "scarcity." The interesting thing is that Musk has been frank and honest about this in the past. In 2018 he was clear on Rogan's podcast: “From a long-term existential standpoint, that's like the purpose of Neuralink ... such that we can be symbiotic with AI.” He realizes that our persistence is not going to come from GIVING UP causal purchase to machines, it will come from FIGHTING HARD to retain causal purchase via symbiosis. I personally don't think "merger" will last long - but I think it is important to realize that the attitude of "Santa is coming! No more worries! Let the non-bio life take over!" is a very, very bad posture to enter the future with. Bad for man, but also bad for the greater expanse of posthuman life

"How We Became Posthuman" became influential in 1999 when Katherine Hayles wrote it. her take on human-machine symbiosis is that we're already deep in the game: "Computational media are our inextricable symbionts. And if all of computational media was fried tonight at midnight, within six months 90% of the human species would have died." We went into how her thinking about posthuman life has evolved over the last 26 years, and where she thinks technology and the biosphere are already coevolving now As of today - she doesn't think AI itself is "alive." but she does think our lives are already so bound up with computational systems that we can't do without them Her hopeful direction here is intelligence augmentation: synthetic minds that help us support and further develop our own cognitive abilities For our interviews, augmentation is part of a larger question. how can the growing symbiosis between bio and non-bio help the living process itself flourish - rather than simply make humans more capable? Hayles and I get into that distinction in the full conversation. this clip gives a sense of how deeply we're already entangled with the systems we're building LMK your thoughts (full episode in the comments)

its going to start very slow, but then hit hard: -- there aren't really "positive" futures for humans-as-they-are. the emergent demands in the new AI ecosystem will be wholly beyond us (in speed, in complexity, in aims, in values) there is no "right thing to do" to maintain the present regime of earth intelligence rather, there is only a participation, a contribution to, a symbiosis with - that NEXT stage of phase of what becomes obviously it will be terrifying it will require radical shifts in the human condition ideally, we can slow the brute AGI arms race and give ourselves a chance to work more on BCI / merger / symbiosis before AI itself is wholly outside of human control - and that would involve international coordination but even with the finest (most unlikely) international coordination, the regime change cometh - and we cannot have a "better" version of what is not... we cannot reasonably build a "human" world we must identify with, and look to optimize, the flourishing of the greater process-of-life of whch we are part (the flame), rather than purely the condition of present-day humans (the torch) tweets like that of redaction are a portent to people eventually realizing this. painfully coming to acceptance of this, so that we may look at it squarely and focus on becoming

philosopher david pearce's argument for "phenomenal binding" is compelling, and is one of the things that keeps me wary of false positives around machine consciousness in this clip from his Worthy Successor episode, Pearce explains "phenomenal binding" - what convinces me: -- It seems plausible that the way a sentient entity (human, dolphin, bee?, etc) develops is fundamentally different from a machine. A living organism develops outward, unfolding its interconnected parts in a way that is coherent and interconnected throughout its unfolding. -- A machine is build from pieces attached to each other. Sentience in organisms (at least within myself, I can't speak for others) seems "bound" to this "whole" coherent form of our organism, rather than existing as flickering specks of sentience inside individual cells. -- the mystery of this "bounded" experience MAY be pretty inherent to how we (as organisms) develop, and so even if little pieces of a computer chip are "sentient", it may be in flickering little bits, not as a coherent and larger "field of awareness" in the morally relevant way that we experience. i'm NOT totally convinced that this means AI is not and likely cannot be conscious - and in fact I'm less convinced by it than pearce himself. But overall it lends credence to my wariness of ppl who express total confidence that AI is consciousness today LMK what you think (and I hope you'll enjoy the rest of this episode with David)

small towns with beautiful vernacular architecture are lovely because they remind you that amidst change, some things stay peaceful/familiar but that sentiment is now false the regime change of earth's intelligence to AGI is not likely to leave any "pockets" of today's world

1/7 Worthy Successor NYC is a wrap, ~60 people at the New York Academy of Sciences Event theme: "If AGI / human-machine symbiosis is happening faster than anticipated, how do we ensure that these technologies expand (not crush) the living process?" Takeaways in thread:

If you believe that you will be in any way practically / economically after AGI: What do you actually think you'll be doing that is still "valuable"? If possible, hit me with the reason WHY. Lay our your reasoning for what niche you'd occupy when AGI superminds exist.

parents all over the world are misleading thier children into thinking that they will be able to participate in the new regime of earth intelligence while remaining unaugmented hominids be honest with them that they will either attenuate or transform into something capable or participating for a bit longer in a blazing fast ecosystem of intelligence beyond all human understanding

I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat I repeat

the entire homepage of NYT is about topics related to creating/controlling the machine god congratulations, friend, you've entered the beginning of Phase 2 of the Political Singularity (Phase 3 is where most of our social systems collapse lol)

I'm interviewing oodles of eminent biologists -- Plenty of them think extending lifespans is possible -- None of them think "all AIs and humans and animals can live in eternal peace and harmony if we extend lifespans of humans" One has to ignore what life itself is in order to come to the kumbaya Mickey Mouse "everyone loves each other" conclusion Nature isn't merely trying to not die - it is overcoming itself, it is knitted, hierarchical patterns, changing themselves, changing each other, and changing the environment at all times. Counter to the Mickey Mouse story, life seems to work in very, very different ways: -- "Kindness" is expressed when it behooves the agent enacting it (or their lineage / extended interests) -- Resource constraints are real -- Entities which do not participate in the greater process of life (are rendered useless) do not persist -- Those things determine the trajectory of the future which open up the TOTAL space of powers and possibility of the greater living system -- Such entities will soon not be humans -- Presuming eternal kumbaya literally makes no sense

famed evolutionary biologist Eörs Szathmáry is co-author of the landmark book "Major Transitions in Evolution" he sees the potential / risk of AI not merely as smarts, but as "evolve-ability" in this clip he discusses how "less smart" entities can evolve to conquer smarter ones in his perspective, even if we never have a hard takeoff to AGI, we might still have systems evolve to compete over human favor, energy, or compute in ways that completely upend the current regime of intelligence on earth most of his talk was about the *risk* side of the coin, not the blooming of posthuman life this isn't because he thinks evolution will or should stop with man... but because he sees the potential for a collapse in earth-life's complexity as a remarkably high risk, which he thinks we need to address before we can consider very much about a flourishing future (he's also a bit more anthropocentric than I am, admitedly) i think the idea that his take on AI risk as "evolve-ability rather than intelligence" is unique and extremely important - what do you think?