RJ Skerry-Ryan

A lot is being made of the fact that Dessa was trained only on human conversations, without any text or pretrained models. First, dGSLM (arxiv.org/abs/2203.16502) already did this in 2022 with just 2k hours of Fisher data. Vendors now provide thousands of hours of high-quality 2-channel dialog off-the-shelf. Second, why do you even want to do this? Humans spent hundreds of thousands of years before inventing writing systems, and then optimized it over 5k years to store knowledge very efficiently. dGSLM had to train only on dialogs because at the time, we didn't know have unified models that could train jointly on text and speech, but this problem was solved by autoregressive transformers. You can cut your training ~10x by letting the model learn from text first, then spoken dialog, like Moshi (and basically every other modern S2S model). Different != smart. Making very natural chit-chat bots is not where the frontier lies. Demos are cool, but the real challenge is intelligence, agentic behavior, and delegation. I would have been blown away if the naturalness was accompanied by GPT-Live style function calling, but alas!