Nathan Pruyne (@pruynathan) profile picture

๐ŸŽผ G-CLef Lab @ CMU PhD student ๐Ÿง‘โ€๐Ÿ’ป๐ŸŽถ

Joined June 2026
Photo by Nathan Pruyne (@pruynathan)Photo by Nathan Pruyne (@pruynathan)Photo by Nathan Pruyne (@pruynathan)Photo by Nathan Pruyne (@pruynathan)Photo by Nathan Pruyne (@pruynathan)
Introducing ๐ŸŽผ๐ŸซงMulTTiPop, an evaluation dataset of multitrack pop music for automatic music transcription! MulTTiPop contains 3.5 hours (572 segments) of aligned multitrack pop audio and MIDI, and can be previewed at gclef-cmu.org/multtipop ๐Ÿงต1/7
3
27
2
228
16,469
Nathan Pruyne retweeted
Game and application development is getting really good. At the same time computer-use agents are finally starting to work. I don't think this is a coincidence. Improved computer-use enables the ability to interact then judge applications which then improves generation as well. Check out my new work Interaction-as-a-Judge for a study on this paradigm!
Agents are demonstrating increasingly strong capabilities in recreating games and applications. One likely reason is their growing ability to directly interact with these applications. We studied Interaction-as-a-Judge, a growing evaluation paradigm where computer-use agents directly interact with games and applications to improve evaluation. Interaction-as-a-Judge improves: - alignment with user preferences. - a modelโ€™s ability to detect errors in games. - downstream generation. We see improvements in inference-time scaling and the ability for interaction-only reinforcement learning. It also scales with computer-use capabilities; the better the computer-use model and method, the better the judgments. Weโ€™re also open-sourcing two agent trace datasets, ArenaWebdevPreferences and ArenaWebdevCorrectness, to help support future research. Dive into the thread to learn more.
5
17
2
46
6,845
Nathan Pruyne retweeted
๐—–๐—ฎ๐—ป ๐˜„๐—ฒ ๐—ฝ๐—ฟ๐—ฒ๐—ฑ๐—ถ๐—ฐ๐˜ ๐˜„๐—ต๐—ถ๐—ฐ๐—ต ๐—บ๐˜‚๐˜€๐—ถ๐—ฐ ๐—ฝ๐—ฒ๐—ผ๐—ฝ๐—น๐—ฒ ๐—ฝ๐—ฟ๐—ฒ๐—ณ๐—ฒ๐—ฟ? ๐ŸŽตโš–๏ธ๐ŸŽถ Meet ๐—ง๐˜‚๐—ป๐—ฒ๐—๐˜‚๐—ฟ๐˜† โš–๏ธ, a model that learns from human judgments to predict music preferences. Use its scores to compare tracks, filter music datasets, and more. Listen and see where you agree with TuneJury!
3
8
2
48
79,605
Nathan Pruyne retweeted
๐ŸŽถHow can we learn to follow along to a music performance in a score without direct supervision?๐ŸŽถ Introducing FuSiLi (Fused Sinkhorn-Localized Similarity)! A multimodal contrastive learning approach that learns precise local alignments using only global supervision. ๐Ÿงต
1
11
4
86
11,118
Check out the links below to download MulTTiPop and for more information: ๐ŸŽผPreview: gclef-cmu.org/multtipop ๐Ÿค—Dataset: huggingface.co/datasets/gcleโ€ฆ ๐Ÿ“ƒPaper: arxiv.org/abs/2607.08756
1
1
423
A massive thank you to my collaborators Ben Stoler, @chenwanch1, @cyhuang_tw, @shinjiw_at_cmu, and @chrisdonahuey, to Sony AI for funding, and to our data annotators at CMU!
1
346
Nathan Pruyne retweeted
๐ŸŽผ Can we turn music into readable code? Excited to share Decomposer, a post-training framework for symbolic music decompilation! Given MIDI, Decomposer recovers an executable program in Strudel that reconstructs the music while exposing patterns, harmony, rhythm, and voices as editable code. ๐Ÿงต [1/5]
11
48
8
276
45,626