Mathis Pink

8/n🧠As a reference point, we collected SORT data from humans who had recently read one of the books in Book-SORT, showing that humans can recall the order of segments based on long-term memory, especially when the distance between segments is larger.

3/n📚 To test episodic memory in LLMs, we created a new task SORT that tests the ability to recall the correct order of text segments in a long text, and a new dataset Book-SORT with 36k segment pairs from 9 public books. We evaluate models in four commonly used setups: