A very interesting area of research!
The future of voice interfaces requires a large amount of visual understanding as well
Both will come hand in hand
Introducing Orion, a foundation model that predicts where you look. Orion finds twice as many UX bugs as frontier models, cuts computer-use latency by 31%, and even won a drone race.