Roman Ring (@Inoryy) header
Roman Ring (@Inoryy) profile picture

[email protected] Research Engineer @DeepMind Large Scale Deep Learning Is All You Need? AlphaStar, Gopher, Flamingo, ???

Joined July 2011
Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)Photo by Roman Ring (@Inoryy)
After high school, I ended up working odd jobs, e.g., as a night shift cleaner at a fast-food restaurant. One especially nasty night, I decided that I had to at least try to get more out of my life and began self-studying coding every free minute I had, up to 20 hrs/day. 1/5
28
79
28
634
I don't think Twitter will go down soon, mostly because I believe the (ex) eng and infra teams did a good job building resilient systems. But it's unlikely I'll be posting from now on. To anyone stumbling on my profile seeking advice on AI, feel free to reach out via email.
2
1
12
Truly visionary business moves this week. I wonder if more people will finally realize the emperor has no clothes.
Replying to @mrddmia
Thank you. A thermonuclear name & shame is exactly what will happen if this continues.
1
9
A bit late to the party but this subject comes up fairly often and I have some thoughts as I spend most of my time at the boundary of Python (JAX) and C++ (XLA), and in the past have been a (very minor) contributor to DeepMind's effort to speed up Python: github.com/deepmind/s6.
It's really weird how much machine learning is done in Python, because numerical/array computing should in theory be like 50x easier in a compiled language where you can build abstractions and leave the optimization to the compiler, no?
2
1
10
However, those switches have led to access to new major features or a significant boost in productivity and I just don't see opportunities to gain either in alternatives today. Python isn't great but all it has to be is *good enough* at serving as a DSL for ML compilers.
1
The unsatisfying reality is that the day-to-day issues researchers run into are usually either at the ML framework level or at the ML compiler level, and neither would be fully addressed by a language with a cleaner API for array computing.
I feel like that blinking white guy meme coming back from a period of minimal social media to a Twitter full of stable diffusion / midjourney generated artwork. Genuinely in awe of the pace of progress.
This Post is from an account that no longer exists. Learn more
4
Hot? take: we might reach (some form of) AGI before we reach SAE J3016 L5 driving automation.
2
7
Prompt engineering might be a legitimate profession in the future.
4
6
4
65
Amazing news! TPUv4's were instrumental for some of the recent large-scale efforts at DeepMind such as Chinchilla and Flamingo.
Incredibly excited that Sundar launched the public preview of Cloud TPU v4 Pods at I/O today, with a flythrough video of a datacenter filled with them: youtu.be/nP-nMZpLM1A?t=7493! This is really three separate announcements: cloud.google.com/blog/produc…
1
4
41
Roman Ring retweeted
It is great to see the excitement about 🦩! As shown in different examples during the last few days, interacting with 🦩 has been quite fun, unique and sometimes mind blowing. However,🦩 has clear limitations as detailed in this 🧵! 1/11
4
70
14
386
Incomplete path to research engineering: - Study calculus, lin. algebra, prob. theory - Intro courses e.g. fast.ai, deeplearning.ai - Advanced courses e.g. cs231n.stanford.edu, cs224n.stanford.edu - Join communities e.g. eleuther.ai
7
109
3
574
- Find fields of interest, do literature review - Replicate interesting papers, showcase on GitHub - Think of ideas to follow up the papers - Reach out to paper authors, discuss your ideas - Implement your ideas, showcase on GitHub - Iterate to success
1
6
56
People underestimate how much AI research is bottlenecked by engineering.
93
217
48
2,944
If you want to contribute to AI research, consider joining industry labs as SWE or RE. PhD / RS is not the only way!
12
11
235
To be clear, of course we're not quite there yet. I had to lead the dialogue, correct rug <> scale, and explicitly ask about the joke. Andrej's "challenge" greatly influenced my outlook on AI and it was really exciting for me to see Flamingo handle it at all. I'll take "cute" :)
Replying to @Inoryy
Cute! not exactly convincing, but cute :)
5
3
177
10 yrs ago @karpathy wrote a blog post on the outlook of AI: karpathy.github.io/2012/10/2… in which he describes how difficult it would be for an AI to understand a given photo, concluding "we are very, very far and this depresses me." Today, our Flamingo steps up to the challenge.
82
491
144
3,706
People underestimate how much AI research is still driven by the hardware + compiler stack.
2
3
1
124
A group of Flamingos is called “flamboyance” which could be an apt description for the family of vision-language models I’m thrilled to see out in the wild! I believe using large pre-trained models in creative ways will be key and hope our work is a step in the right direction.
Introducing Flamingo 🦩: a generalist visual language model that can rapidly adapt its behaviour given just a handful of examples. Out of the box, it's also capable of rich visual dialog. Read more: dpmd.ai/dm-flamingo 1/
5
34