Zurich, Switzerland
Joined January 2021
- Tweets5.7K
- Following715
- Followers85.5K
- Likes13.9K
Over the weekend, I ran an experiment: Can LLM agents, individually or in teams, manage a weekly cross-asset portfolio better than simple systematic rules?
1/8
There are several caveats to this experiment. For example, I only used a small model and not giving agents the full breadth of information at a time also puts them at a comparative disadvantage, etc.
Many ways to tweak and iterate on this experiment.
7/8
I dont think the ECB *should* hike rates more this year.
Financial conditions are a lot tighter than usual measures suggest, and sovereign yields have contributed to this. This tightening will continue to play out, and today's infl environment is very different to post-Covid
Why do people still say they expect "higher for longer" interest rates?
I get that this was en vogue 2 years ago because it was reasonably "novel" then but we've broadly been in that environment in the US for a while now?
Ultimately, and in the limit, I think AI agents will behave and act in financial markets similar to what we see through Game Theory Optimal in Poker.
Agree with this. The whole model-only benchmarking is a waste of time.
You need to evaluate model + harness on real-world tasks.
Model Benchmarks Are Worthless
How can a model score 24% and 59% on the same 113 coding tasks?
It happens when you vary the harness that wraps it.
I’ve increasingly ignored most of the recent model leaderboard results. Partly from exhaustion and partly because improvements in a ranking don’t show up in the actual work.
So I asked the 8090 Labs team to measure what the leaderboards skip: the model and the harness together, on real bug fixes and features from open-source projects.
8090 Labs ran 7 models through 5 harnesses.
Here are a few results I didn't expect:
- Anthropic's Claude Fable 5.1 scored 41% in Claude Code and 61% in OpenAI's Codex. It made 28 tool calls at the median in one and 66 in the other.
- The cheapest runs weren't the cheapest work. GLM in Codex cost $0.22 a run and $0.93 per success. In Pi it cost $0.25 a run and $0.43 per success.
- GLM passed a task on both attempts 51 times in one harness and 16 times in another.
- GPT-5.6 Sol and Grok 4.6 barely cared which harness they were in.
The bottom line is that if you're picking a model/harness combo off a model-only leaderboard, you're making a critical decision based on only half the information.
8090.ai/blog/same-model-diff…
