Perceptual grounding and explainable AI (#xAI) lab at @InfAtEd @EdinburghUni Mastodon at: https://nitter.cf/t.co/O7iRXGnZ1T
Joined September 2022
- Tweets35
- Following8
- Followers55
- Likes20
Pinned Tweet
We are advertising a postdoc position to work on #generative #models, #structure #induction, and MI #estimation with Michael Gutmann as part of
#GenAI hub!
elxw.fa.em3.oraclecloud.com/…
Get in touch! (#ML #AI)
👉 homepages.inf.ed.ac.uk/snara…
👉 michaelgutmann.github.io
ExLab retweeted
Transformers struggle with length generalization and long context. What can we do about it?
Our new #TMLR paper with @rolandalong , @paul_smolensky and @JianfengGao0217 shows how to handle the issue. Using a new attention mechanism called TRA.
Curious? Read the 🧵 for more 🤓
Poster is at 11 today East Exhibition hall E-1810!
Are you compositionally curious 🤓
Want to know how to learn embeddings using🌲?
In our new #ICML2025 paper, we present Banyan:
A recursive net that you can train super efficiently for any language or domain, and get embeddings competitive with much much larger LLMs 1/🧵
We will be advertising for a postdoc position soon, to work on #generative #models #structure #induction and #uncertainty with Michael Gutmann as part of the GenAI Hub!
Keep an eye out, and get in touch! ( #ML #AI #ICML2025 )
👉 homepages.inf.ed.ac.uk/snara…
ExLab retweeted
Are you compositionally curious 🤓
Want to know how to learn embeddings using🌲?
In our new #ICML2025 paper, we present Banyan:
A recursive net that you can train super efficiently for any language or domain, and get embeddings competitive with much much larger LLMs 1/🧵
How do we learn to solve problems more efficiently?
DreamDecompiler targets more efficient #programinduction
- Reducing search breadth: training to 'compile' useful information
- Reducing search depth: learning compressed libraries
(w/ A. Palmarini and C.G. Lucas)
#ICML2024
DreamDecompiler develops a novel approach that directly leverages the neural search policy by
- Effectively 'decompiling' the knowledge learned for composing program solutions.
- Allowing the search policy to shape the search space that it will subsequently guide.
This double amortization helps
knowledge learned to reduce search breadth also reduce search depth, leading to better component selection from libraries, faster domain proficiency & improved generalisation.
Find out more at our poster (#1602) (Hall C)
arxiv.org/abs/2306.07856
Our work on auto-encoding CNPs for representation learning with @victor_p91 and @iatitov will be at #icml2024 🎉
What does an inductive bias for explicit 🌲 structure buy you?
If you're compositionally curious, check out our #EMNLP2023 paper with @zvez11, @victor_p91 and Siddharth N
We explore if explicit compositions aid multi-level representation learning arxiv.org/abs/2305.05588
1/t
Self-StrAE turned out to be pretty effective 💪
It reaches similar results to StrAE using constituency parses, and is competitive to unstructured baselines. With 430 parameters it does better than Fasttext, a Bi-LSTM, and comparably to a 6 layer RoBERTa (~4 mil parameters) 8/t