Information Systems Researcher | Building tools for community | Adobe and Tachyon alum 🦦🇺🇸
Ann Arbor, MI
Joined April 2008
- Tweets10.1K
- Following4K
- Followers1.8K
- Likes47.1K
Jonathan Dunlap retweeted
Wow, I did not expect that DINOv3's global [CLS] token linearly represents the continuous geometric latents of dSprites (size & X/Y position) 🤯
It only took me 3.5 years to finally run this experiment 😂
I'm looking to do more of this MechInterp work, dissecting foundation models like biological artifacts and building theory. If you want to collaborate (especially students looking for a fun project) reach out! 🔬🤖
Jonathan Dunlap retweeted
Is it possible to make 3D Gaussian Splatting even faster?
Short answer YES!
Enjoy a 10.5x optimization on forward/backward cuda kernels over the published 3DGS ones!
grgkopanas.github.io/3dgs_cu…
Jonathan Dunlap retweeted
There is a pattern underneath some of the most interesting computer graphics techniques. Culling, level of detail, light clusters, particle systems, Gaussian splatting: they look like different problems, but the same pattern shows up in every one of them.
My new video shows that pattern:
count → prefix sum → write
the three shapes it takes (compaction, expansion, binning), and how they chain into a GPU-driven frame.
Watch the full video on youtube
Jonathan Dunlap retweeted
Here's a quick look at some of the editing features in Pixel Art Lighter.
The tool will be fully free and open source!
Jonathan Dunlap retweeted
"Mathematical Theory of the Transformer" is an extensive, 825-page mathematical monograph exploring the theoretical foundations of Transformer architectures.
It covers a remarkable range of topics, including embeddings and positional encoding, self-attention and multi-head attention, matrix decompositions, linear algebra, spectral analysis, information geometry, training dynamics, redundancy, optimization, and incremental Transformer architectures.
This is an advanced, mathematically rigorous read for anyone interested in understanding the mathematical structures underlying Transformer-based AI systems, including modern generative AI.
doi.org/10.5281/zenodo.20199…
Jonathan Dunlap retweeted
My dear friends, I’d like to show you my latest toy project
Introducing Handwired: turn your regular JavaScript code into a neural net!
- no training
- exact outputs vs the JS program
- fully differentiable
All runnable in a browser, without GPU
Jonathan Dunlap retweeted
Complex shock and rarefaction interactions in a formally verified, ultra-high-resolution multi-GPU simulation for compressible hydrodynamics.
One shot from a single prompt. 2 minutes to write the solver and prove correctness. 5 minutes to run the simulation on 8 GPUs.
Jonathan Dunlap retweeted
BudgetPix
Compute-Adaptive Tokenization for Pixel-Space Image Diffusion
paper: huggingface.co/papers/2610.1…
Jonathan Dunlap retweeted
This is a beautiful analysis of the order in which diffusion models resolve uncertainty about information at different abstraction levels in different representation spaces, with significant implications for latent diffusion.
panjiashu.github.io/feature-…
Jonathan Dunlap retweeted
Vision Transformers Learn Word Meaning from Images Alone
TL;DR: DINOv3 learns word semantics from images alone. Its features capture semantic similarity and country–capital relations, and align written words with their visual counterparts—without explicit language supervision.
arxiv.org/abs/2610.13252
Jonathan Dunlap retweeted
Wrote an interactive blog post on the theory of autonomous flows 🌊 and transport maps 🗺️ for language modeling!
sophtang.github.io/discrete-…
Explore how removing time dependence from the generative flow yields straighter trajectories on the simplex, stable equilibria, and an endpoint map that yields a clean sequence in one step by construction —and can be trained end-to-end directly from data without a teacher flow.
Had a lot of fun with animations, so hope you enjoy!
Jonathan Dunlap retweeted
Few-Step Generation via Data-Space Iteration
Paper: arxiv.org/abs/2610.12102
We rethink the iterative process behind few-step generation. Instead of iterating in flow space, we propose data-space iteration as a new paradigm for better few-step generation.
Details in the thread!
Jonathan Dunlap retweeted
We're entering the age of "tasteslop" - content that can be slopped out with little effort, but exhibits good taste.
Spec ad for AmEx - Claude built a mosaic animation engine with 500,000 tiles.
No video models. Javascript is the language of the universe.
Jonathan Dunlap retweeted
Decision models in llama.cpp are now available
The `/v1/systemone` endpoint is available in the latest llama builds. Use it to do Jev-style inference locally, efficiently and privately. Multiple open models are supported with more to come.
huggingface.co/blog/ggml-org…
Jonathan Dunlap retweeted
big AI news
Google just demonstrated a recursive self improvement loop for AI discovery
Google/DeepMind researchers introduced Dream-RSI, a system where an AI agent improves how it explores problems by replaying its past discovery attempts, testing thousands of alternative strategies cheaply, then deploying the better strategy in the next round.
Across algorithm design, mathematical optimization, and GPU kernel engineering, it matched or improved discovery quality while cutting search costs dramatically, in one setting reducing agent calls by up to 162x. 👀
Importantly, it improves the exploration policy, not the underlying model weights.
Jonathan Dunlap retweeted
After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?
I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev
• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions
AFAICT the shortest path to AI-based economic revolution
Jonathan Dunlap retweeted
Ironed out the last performance issues with my full 2.5m line codebase explorer. 120hz awesomeness. Can only upload 60fps video tho. Much nicer uncompressed
