the lab of Michael DeWeese at UC Berkeley | neuroscience, neural nets, and nonequilibrium stat mech | https://nitter.cf/t.co/YfRrMlakoN | run by students
Joined November 2020
- Tweets178
- Following162
- Followers776
- Likes302
Pinned Tweet
Very excited to share our new paper out in PRL today (as an Editors’ Suggestion)! We unify thermodynamic geometry 🔥 and optimal transport 🥖 providing a coherent picture for optimal minimal-work protocols for arbitrary driving strengths 💪 and speeds ⏱️ ! journals.aps.org/prl/abstrac…
We are pleased to see our paper with @benkuznetsspeck freshly published on PRE! We derive a novel time-asymmetric fluctuation theorem that provides the theoretical foundation for stochastic normalizing flows, useful for efficient free-energy estimation! journals.aps.org/pre/abstrac…
Interested in representation learning or self-supervised learning (SSL)? Check out this work led by lab member Jamie Simon, which gives the first compelling scientific picture of the training process of modern SSL methods like SimCLR! Appearing at ICML '23.
Self-supervised learning lets neural nets learn from unlabeled data. Yet despite its success w/ models like @Midjourney, how image representations are learned has been poorly understood.
In a @ICMLConf paper, we share the first scientific picture of the SSL training process 👇
DeWeese Lab retweeted
Happy to see Jamie's research about the stepwise nature of SSL on the BAIR blog: bairblog.github.io/2023/07/1…
If you're interested in learning more, stop by and say hi to us at ICML next week!
Lab member @dhruvakarkada just revamped the NTK wikipedia entry to make it more accessible to technical non-experts. cool gif inside -- check it out!
en.wikipedia.org/wiki/Neural…
Check out @genintelligent, a new startup aiming to develop fundamental understanding of AI, and their Avalon environment for the development and testing of generally capable RL agents!
Today, AI systems can create stunning art & beat humans at chess & Go. But they can't do things a 3-year-old can do. Why?
We're launching @genintelligent & open sourcing our research environment, Avalon, so as a community we can answer these questions: generallyintelligent.com/lau…
DeWeese Lab retweeted
In 2020, @michael_nielsen & I began a 2-month project to write: "how would we fund science?"
2 years & 40,000 words later, it's become: "how can the culture & institutions of science actually change, and ultimately become self-improving?"
Our answer: scienceplusplus.org/metascie…
Seen the recent papers demonstrating that neural nets' learned functions are very close to kernel regression (KR) with the net's *post-training empirical NTK*? What properties should we expect in such data-adapted kernels? Our new arXiv note suggests some:
arxiv.org/abs/2209.01691
galvanizing thinking about *adaptive* kernel methods. If we understood how a neural net's NTK really adapted to data, this would go a *long* way towards clarifying nets. This note is a first little step towards clarifying how we might think about such objects.
just updated deweeselab.com/! check it out for a view of new research and new lab members.
DeWeese Lab retweeted
How can we design neural networks in a principled way? New work from @DeWeeseLab @berkeley_ai explores a theoretically-motivated paradigm for doing just that for dense networks! Read about it here: bair.berkeley.edu/blog/2022/…
In common learning rate schedules for deep learning, lr almost always *decays* (i.e. decreases with time). Loss plateaus if you don't do this.
Is it known why? Help us understand what's known!