ML/AI Engineer @zenml_io, researcher (& author of a few books + PhD 👨🎓). feedback welcome https://nitter.cf/t.co/W6fKtTKoPv 🙏
Delft, The Netherlands
Joined January 2011
- Tweets3.6K
- Following722
- Followers3.3K
- Likes8.8K
Pinned Tweet
Finally got round to adding in a way to subscribe to my blog / site via email. (Always supported RSS, but seems that's gone out of fashion a bit 🤷 )
Also added in a way to browse some of the blog series I've written over the years instead of just scrolling through a huge long list of posts.
Please do subscribe if you want to get an email whenever I write new posts!
alexstrick.com/subscribe.htm…
alexstrick.com/series.html
Alex Strick van Linschoten retweeted
this is a tour de force of swarm trajectory visualization
Seeing more and more people writing about swarm observability so thought I'd post some of the work I'd been doing to think this through.
I built some tools to help AI engineers work with agent swarm traces and other signals. Specifically:
- a collector that tracks everything the agents in a swarm are doing, from filesystem writes and reads to inter-agent messages to traditional traces that you're probably familiar with
- a viewer with a bunch of different modes (see the rest of the 🧵 below for some glimpses of these) to help someone trying to understand how a swarm behaved either during or post-run
- a simple agent ('hodhod') that comes with some deterministic tools to help quickly try to slide and dice the huge volume of data recorded by the collector, which can try to help understand how things worked (or where they failed)
Today I'll focus more on the viewer component as it's easiest to understand...
Alex Strick van Linschoten retweeted
these graphs you made are awesome!!
I think tool based communication is a good start -> message board & DM actions to see how swarms communicate
like with heavy bash use -> understanding semantics in a swarm is still super hard
some potentially interesting approaches:
- very cheap classifiers on actions to bucket agent actions into more understandable pieces
- making agents responsible for writing human interpretable logs periodically
one benefit of code mode over tools is that we can better map trajectories of actions into a semantic group like “editing the video”
I think this is a really fun area generally -> making agents interpretable to humans
Seeing more and more people writing about swarm observability so thought I'd post some of the work I'd been doing to think this through.
I built some tools to help AI engineers work with agent swarm traces and other signals. Specifically:
- a collector that tracks everything the agents in a swarm are doing, from filesystem writes and reads to inter-agent messages to traditional traces that you're probably familiar with
- a viewer with a bunch of different modes (see the rest of the 🧵 below for some glimpses of these) to help someone trying to understand how a swarm behaved either during or post-run
- a simple agent ('hodhod') that comes with some deterministic tools to help quickly try to slide and dice the huge volume of data recorded by the collector, which can try to help understand how things worked (or where they failed)
Today I'll focus more on the viewer component as it's easiest to understand...
I have no inside information, but I have to imagine there are people at @AnthropicAI / @OpenAI or even maybe smaller places like @PrimeIntellect who are building these kinds of swarm observability tools internally.
There's no way to get your mind round all the data that a swarm outputs, so you have to have lots of different views that you can always be approaching your data from different perspectives.
I also took a lot of inspiration from @eastgate's Tinderbox tool, which I've used for coming on twenty years now (!) -- eastgate.com/Tinderbox/ -- and where you can switch views across your notes. I always found that very useful when writing books or thinking through research work.
At any rate, if any of this resonates with you or you're building these kinds of tools, please do reply on the thread or drop me a DM. Would love to chat about how you're thinking about these problems!
Will post some of the interesting swarm observability approaches I find below. nitter.cf/firenoaa/status/211618… shows @capydotai agents working together, as one really nice example!
a few people asked for more on this
the uncut video of our @capydotai agents talking to each other
last week: 135 agents, 2,638 messages (data's scrambled)
it's no longer one person with one agent
it's a team of people and agents