Run Long (@RunLong) header
Run Long (@RunLong) profile picture

AI explorer · Founder @gedolabs — a growth system that actually knows you: assistant within, digital self without. Training a model that's yours alone. 🇸🇬

Singapore
Joined March 2023
Run Long retweeted
On November 19, the results of the second CSIG AML Memory Challenge were announced. AML (Agent Memory Leaderboard) fixes the answering and judging models, so entries differ only in their memory layer; this cycle completed more than 400 formal evaluations. Gedo Memory placed 3rd on the Commercial Products leaderboard for Text Memory (62.28). After taking part, we wrote up our design and what this cycle showed us as a research note: "Designed to remember your life: the system design and trade-offs of Gedo Memory." The two #1 systems and our own happen to be three different approaches. ① Verbatim retrieval Resurface, #1 on the Open-Source Methods leaderboard (61.22). No model calls. Every message is stored as is, retrieved and handed to the answering model. Near-zero write cost and nothing lost; updates and reasoning are left to the answering model. ② Fact extraction MemoraX, #1 on the Commercial Products leaderboard (69.26). Built around extraction and add, update or delete decisions at write time, so reads stay light. Done well, it pays off most, and that takes strong extraction and update handling. ③ Two layers Gedo Memory, #3 on the Commercial Products leaderboard (62.28). The verbatim record is kept without loss as the fallback, and only the important facts are extracted as the index, accepting misses. That is why both the product and our entry run extraction on a small model. Our internal pilot (LongMemEval-S) Verbatim retrieval reaches about 87%, already close to putting the whole history in context. The two layers together reach about 93%, matching the whole history with about 5% of its input. On LoCoMo, verbatim retrieval does better. What we took away · The verbatim record is the floor. Extraction alone is the riskiest option. · Extraction earns its keep above that floor: time anchoring, updates and provenance. · The per-capability scores also show what our choices cost: the widest gaps are in epistemic safety and privacy and in personalization, which map to how much context we hand over and to passive retrieval. · The verbatim record can be found, but it can hardly be seen. We want your memory to be fully visible, so high-quality extraction is our core research direction. Thanks to @AgentMemoryL and CSIG for the evaluation work. Full note: the four approaches, the cost of each design decision and what comes next gedo.ai/labs/gedo-memory-des… #AgentMemoryLeaderboard
5
1
6
28,980
Run Long retweeted
Gedo Memory(gedo.ai) placed 3rd on the AML Cycle 2 Commercial Products leaderboard (Text Memory), and 5th among closed-source methods. I'm grateful to our team, and to the AML and CSIG organizers. A few notes on what this meant for us. Why we entered At Gedo we're building a personal AI that remembers you over months and years. The memory layer is the product: what it keeps, what it lets go, and what it never says out loud. For a long time we measured it only on our own tests. AML gave us something we couldn't build ourselves: an independent benchmark where the organizers fix the answering model and the judge, so the only thing that differs between systems is memory. What the run looked like We decided to enter in late October and had our system online a few days before the deadline. The full suite then ran for about 72 hours. It surfaced problems our own tests never had: shards running out of memory under sustained load, idle connections being dropped, a write path that needed to survive a restart. All of them were on our side, and fixing them made the system sturdier. What the scores told us We did best on explicit fact recall, multi-hop reasoning, and time and event order. Relative to the field, our weakest area was epistemic safety and privacy. That's the one I care about most. A personal agent that recalls everything but doesn't know what to keep to itself isn't one you'd trust with your life. Since the competition, that's where our work has gone: forgetting, and what the agent chooses to disclose. Agent memory is still early, and shared, independent benchmarks are how the field gets better. Thank you, @AgentMemoryL.
2
2
1
6
289
Run Long retweeted
How it works: 1. Retrace: it replays the years you've recorded. 2. Simulation: it lives a path on a real-world map and stops at each crossroads for you. 3. Journal: what it learned, and where you aren't ready yet. Every screen is the real product. gedo.ai/help/gedo-world
1
1
109
Run Long retweeted
A lot of AI is racing to do everything on its own. We're trying a different path: an AI that works with you. Here's one day of it: a morning run, Mom's birthday dinner planned on a voice call, and a visitor from Tokyo. Gedo is in beta. Watch, and tell us if we're wrong.
2
1
1
4
8,657
Run Long retweeted
Watch Starship’s second integrated flight test nitter.cf/i/broadcasts/1dRKZEWQv…
386
9,744
171
3,839
772,313