Pinned Tweet
Introducing ๐ผ๐ซงMulTTiPop, an evaluation dataset of multitrack pop music for automatic music transcription!
MulTTiPop contains 3.5 hours (572 segments) of aligned multitrack pop audio and MIDI, and can be previewed at gclef-cmu.org/multtipop
๐งต1/7
Nathan Pruyne retweeted
Game and application development is getting really good. At the same time computer-use agents are finally starting to work. I don't think this is a coincidence.
Improved computer-use enables the ability to interact then judge applications which then improves generation as well.
Check out my new work Interaction-as-a-Judge for a study on this paradigm!
Agents are demonstrating increasingly strong capabilities in recreating games and applications. One likely reason is their growing ability to directly interact with these applications.
We studied Interaction-as-a-Judge, a growing evaluation paradigm where computer-use agents directly interact with games and applications to improve evaluation.
Interaction-as-a-Judge improves:
- alignment with user preferences.
- a modelโs ability to detect errors in games.
- downstream generation. We see improvements in inference-time scaling and the ability for interaction-only reinforcement learning.
It also scales with computer-use capabilities; the better the computer-use model and method, the better the judgments.
Weโre also open-sourcing two agent trace datasets, ArenaWebdevPreferences and ArenaWebdevCorrectness, to help support future research. Dive into the thread to learn more.
Nathan Pruyne retweeted
๐๐ฎ๐ป ๐๐ฒ ๐ฝ๐ฟ๐ฒ๐ฑ๐ถ๐ฐ๐ ๐๐ต๐ถ๐ฐ๐ต ๐บ๐๐๐ถ๐ฐ ๐ฝ๐ฒ๐ผ๐ฝ๐น๐ฒ ๐ฝ๐ฟ๐ฒ๐ณ๐ฒ๐ฟ? ๐ตโ๏ธ๐ถ
Meet ๐ง๐๐ป๐ฒ๐๐๐ฟ๐ โ๏ธ, a model that learns from human judgments to predict music preferences.
Use its scores to compare tracks, filter music datasets, and more.
Listen and see where you agree with TuneJury!
Nathan Pruyne retweeted
๐ถHow can we learn to follow along to a music performance in a score without direct supervision?๐ถ
Introducing FuSiLi (Fused Sinkhorn-Localized Similarity)!
A multimodal contrastive learning approach that learns precise local alignments using only global supervision.
๐งต
Check out the links below to download MulTTiPop and for more information:
๐ผPreview: gclef-cmu.org/multtipop
๐คDataset: huggingface.co/datasets/gcleโฆ
๐Paper: arxiv.org/abs/2607.08756
1
1
423
A massive thank you to my collaborators Ben Stoler, @chenwanch1, @cyhuang_tw, @shinjiw_at_cmu, and @chrisdonahuey, to Sony AI for funding, and to our data annotators at CMU!
1
346
Nathan Pruyne retweeted
Nathan Pruyne, Benjamin Stoler, William Chen, Chien-yu Huang, Shinji Watanabe, Chris Donahue, "MulTTiPop: A Multitrack Transcription Dataset for Pop Music" arxiv.org/abs/2607.08756
๐ผ Can we turn music into readable code?
Excited to share Decomposer, a post-training framework for symbolic music decompilation!
Given MIDI, Decomposer recovers an executable program in Strudel that reconstructs the music while exposing patterns, harmony, rhythm, and voices as editable code.
๐งต [1/5]







