I Evaluate AI Agents. Opinions are my own.
Paris
Joined November 2022
- Tweets13.6K
- Following1.5K
- Followers2.2K
- Likes66.5K
Pinned Tweet
A single eval run on swebench verified mini can cost $1300+ for agents based on SOTA LLMs. If you are compute-poor, it becomes impossible to maintain an independent swebench verified mini leaderboard under these conditions. But, what if there is a technique to help reduce the cost of agent benchmark runs by 40-70% ? The paper is out. My first solo paper:
Access to knowledge has never been easier, yet we choose distraction most of the time. Why is that?
131
The beauty of the transformer architecture is that the underlying idea is so elegant and generalizable that you can just throw more compute at it and get better and better results across all sorts of modalities. Anything that can be tokenized can, in principle, be modeled by a transformer. Diffusion is another such beautiful idea. We need more ideas like that.
1
157
We are entering the golden age of the personal intelligence harness: free of corporate interests, safe and secure.
159
Je pense qu’en France, comme dans beaucoup de pays développés, il faudrait un ministère du Lien social. Cela fait des décennies qu’on néglige l’importance de faire communauté, de lutter contre l’isolement, de permettre aux gens de se rencontrer, de se parler, de se voir et de nouer des amitiés tout au long de leur vie.
90
Franck SN retweeted
On efficient algorithms, the Exponential Time Hypothesis, and losing the plot.
argmin.net/p/galactus-the-de…
