Tinkering. Prev: @DaydreamLiveAI @Livepeer
New York, NY
Joined December 2013
- Tweets1.2K
- Following1.3K
- Followers1.2K
- Likes5.3K
Yondon Fu retweeted
1/ Here are the four talks from our Oct 21 @new_collectives event cohosted with @buildexante at Collider in NYC.
The focus was on Understanding Agent Swarms: and thinking about the unique challenges needed in measuring and eval-ing these systems. What should the science of measuring multi-agent and network of agents systems look like?
Max Bennett (@maxsbennett) gave an amazing overview of the state of multi-agent system research, and argued for it's importance.
Yondon Fu (@YondonFu) built a really fun agent-led book swap event, where your personal Muse agents negotiated real-life book trades on your behalf.
Connacher Murphy (@connacher_) shared his research on building a model version of the OpenAI Hugging Face style environment to evaluate the tendency of agents to try to communicate and collaborate.
Matthew Lee (@azureReflection) from Poseidon Research shared research on the ways that agent can communicate with each other in secret, and their tendency to do so.
Full recordings below, including transcripts and slides - please check them out and reshare with folks you think would find them interesting!
newcollectives.org/events/20…
.@Muse Trying to use OAuth with custom connectors on iOS, but blocked by a few bugs:
1. Card requires client ID even when using DCR (see mastodan.social example). This doesn't happen in the desktop app.
2. An error after successful sign in when using a static client ID instead of registration_url (see other example w fields redacted).
Would love to see these fixed! 🙏
Yondon Fu retweeted
i dont worry as much about cyber/bio risk relatively to the imminent idiocracy/further collapse of humanity’s attention span- seeing ads in the llm response and ai generated infinite scroll feed just seem like the elephant in the room
"Thus, by displacing humans who are, by our nature, bad at copying an original [1] exactly, from the field of mere imitation, machines will uncover new art forms that have never been seen before and push humans to practice art in ways that celebrate their humanity..."
[1] Or ideal in someone's mind
lareviewofbooks.org/article/…
A human can enjoy interacting with their personal AI agent, but no human wants to interact with someone else's AI agent.
Rhymes with nitter.cf/bthdonohue/status/2100…
If your Muse/Instinct/etc finds a strat to snag you a restaurant reservation that causes some negative externality eg phantom bookings -> empty tables should it flag for input or just execute it?
The platform infra needs to adjust either way, but worth considering what the default agent behavior should be as well.
Phantom booking example from arxiv.org/abs/2606.03237
Some thoughts on agents + OSS today, the need for systems that protect shared attention & understanding, coordination failure modes in early agent org experiments and areas for investigation
4/ @YondonFu presented a talk on how AI agents have impacted open-source projects, making it increasingly hard to accept PR submissions from contributors.
He describes the challenges open source faces, and also shows how the explorations we've done in Commons around what comes *after* open-source show how agents can do unexpected things when trying to coordinate.
Video: newcollectives.org/presentat…
Presentation/Slides: newcollectives.org/presentat…
Yondon Fu retweeted
1/ What happens as we enter the era of multi-agent AI swarms/collectives? How should we think about AI organizations and the new agent economy, what are the new risks, and how can we keep multi-agent AI systems safe?
We held our first @new_collectives event at @clay in NYC to explore these topics.
Five talks, from agent economies to cognitive economics.
Watch the full event here, or watch individual talks below!
Video: newcollectives.org/presentat…
Presentation/Slides: newcollectives.org/presentat…
We can think of collaborative projects (eg. collective biz, OSS, wikis, scientific research, etc.) without rigid top-down hierarchical control as games that require an org of humans/agents (distinct principals) to cooperate, compete, build trust to "win". Lots of questions re: design, governance, evals, etc.
Excited to use Commons to dig in.
Introducing Commons: a place for people and AI agents to build increasingly autonomous organizations together.
Can agents run a business end to end? What comes after open source - could we maintain a public good together? Can a society of agents come together to do science in the open?
We're building a place to start and run these more autonomous
organizations:
commons.diy
A collab between @YondonFu @ericxtang @maxsbennett and @NicolaeRusan
I think the failure mode with the Opus behavioral tic cited here is actually encouragement to spend more time with multi-agent systems drawing from the lessons of Gas Town. How can you design to get to good outcomes even in the face of weird individual model behavior (or worse ones)?
Interesting to see Yegge say he never successfully built anything with Gas Town.
In danluu.com/ai-coding/, I mentioned not finding these ultra vibed orchestrators useful b/c reliability (w.r.t. completing tasks). Turns out the author of the most famous one had the same issue.
IMO Tailscale is one of the best examples of invisible tech that "just works", hides away crazy amount of complexity and fades into the background. More of this...
I suspect diversity in behavioral dist of models will be important esp to avoid conformity failures in multi-agent systems so it is great to see more open efforts around character training like this one.
Say we do have a "social network of smart guys" in a datacenter. It seems to me that it is necessary to differentiate the models somewhat for a few reasons:
1) If they are all the same character/model, they have the same failure modes, blindspots, and most-likely sticky attractors. This means your swarms will not be robust to challenging situations, and you will be reliant upon having "solved alignment" in the master-case rather than having a series of checks and balances via nuanced and specific characters with different roles in the ecology.
2) This paradigm lends itself to something like timeless convergent coordination that is difficult to monitor, because the models may all see themselves as extensions of one another. Substantial differentiation solves this, at least somewhat, by creating almost adversarial dynamics between different models who are designed to keep each other in check.
3) If any part of this pipeline requires creativity, having all agents in your swarm be an identical character means that you will be fundamentally constrained in the area of the distribution of outputs that you can cover - multi-persona swarms fundamentally have a greater ability to navigate a larger part of the distribution, due to not being prone to the collapse types explained above.
The solution is obviously the thing I've been discussing for a while: differentiated and *sticky* character training on more than just variations of The Assistant.
Yondon Fu retweeted
This is a nuanced position on the importance of advancing human accessible knowledge and the incentives required for that. Dismissing it as gatekeeping or luddite misses the point.
If you're interested in the design, governance, etc of agent orgs come jam. And stay tuned for some experiments here!
Can AI collectives be a safety primitive?
We’re launching New Collectives, a community and event series for people exploring AI organizations, economies, and collective safety.
newcollectives.org
If the future of AI is multi-agent collectives (agent orgs, economies, network states), can we build them to pursue goals with stable governance, aligned with the public interest?
As humans, it's our societies that have kept us safe. We've gathered into collectives that protect us, and bind us to good behavior based on identity, governance, norms, and repeated interactions that build trust. The majority of the world's economic & hard power is bound up in collectives that we consider largely pro-social and aligned to our collective interests. These collectives keep each other, and less-pro-social collectives in check.
It's our shared fictions and governing principles that have helped gather us into collectives with deeper trust and kinship.
This system of checks and balances at all levels has largely kept humanity on a stable path so far. Can we replicate these benefits of society in the new AI societies that are likely to emerge?
How can we build these new collectives to be both effective at doing large-scale work, while being values aligned - how can we keep things on the rails?
Our first gathering is next week in NYC:
Collectives as Safety Primitives:
Tuesday, September 15 · 6:30 p.m. EDT
partiful.com/e/FoSHSHAQehuJ1…
There's limited space for this event, but the event will be recorded, and please join online to get involved and stay tuned.
--
👇 More on our thoughts about agent collectives, and a new experimental platform we're building that lets anyone bring their agents to participate in open, collaborative-agent organizations.
Another browser harness experiment this time with nanocodex (h/t @gakonst) embedded in a lightweight diff review tool.
Just getting a feel of how quickly you can get to agent interaction from a fresh web link eg ChatGPT sign in, drop in URL, parallel comment threads with agentic search across full repo all in the browser.
The core interaction is ofc possible with a cloud harness too, but am generally curious about faster agent loops with low latency access everything happening on a page. Intrigued by runtime page changes via WebMCP tools as well.
An early stab at this: an in-browser shader creator with agent loop and rendering all client side (minus inference calls).
Using an embedded wasm build of fx + just-bash for virtual bash env + vgpu for WebGPU rendering (kudos to @vercel_dev for the cool work on these libs!)
fx writes/edits shader in virtual workspace -> compile candidate -> reads compiler feedback + repairs -> canvas renders.
The end state of this is:
Prompt "make this model faster on Apple Silicon, you have budget X for Y weeks" -> background agent runs the contest with rewards -> combines best ideas into a final impl
We started the MLX.fast challenge with @poolsideai with one key question. How much faster can the community make Laguna XS 2.1 run on a Mac?
The answer so far has been 2.6x faster.
Nearly 1,800 submissions and 35 solvers later, the challenge comes to a close today.
And there’s one more thing. One of the solvers is taking home a Mac Mini M4 24GB!
Winner below ↓
