rewon (@rewonfc) profile picture

Here for the research papers

San Francisco, CA
Joined November 2012
Photo by rewon (@rewonfc)Photo by rewon (@rewonfc)Photo by rewon (@rewonfc)Photo by rewon (@rewonfc)
Excited to announce that we’ve raised $1.3B to build one of the largest clusters in the world and turbocharge the creation of Pi, your personal AI. forbes.com/sites/alexkonrad/…
141
303
163
2,513
929,638
It’s a big week! We’ve raised $1.3 billion and are building the world’s largest AI cluster (22k H100s). We’re grateful for our investors and new funding that will help us accelerate our mission to make personal AI available to every person in the world. inflection.ai/inflection-ai-…
67
211
287
1,460
2,768,553
We have amazing results to announce! Inflection-1 is our new best-in-class LLM powering Pi, outperforming GPT-3.5, Llama and PALM-540B on major benchmarks commonly used for comparing LLMs. inflection.ai/inflection-1
22
85
15
515
171,023
Kullback-Leibler divergence is not the same as Leibler-Kullback divergence
50
280
65
3,164
My post on SotA image generative models was released 🥳 Featured 7 notable recent papers with emphasis on: - VD-VAE - VAE + discriminator (e.g. VQGAN, DC-VAE) - Diffusion models (e.g. DDPMv2) Plus some notes on scaling (e.g. DALL-E) and evaluation. arankomatsuzaki.wordpress.co…
1
56
1
294
IMO, best empirical proof to date that AI can be creative. After this sinks in, will there be any naysayers left?
"The images are preprocessed to 256x256 resolution during training. [...] each image is compressed to a 32x32 grid of discrete latent codes using a discrete VAE that we pre-trained using a continuous relaxation." GPT + VAE + scale = impressive results! openai.com/blog/dall-e/
8
6
2
128
It breaks my 💚 when researchers tell me that VAEs don't work. My first typical question is "did you try hierarchial VAE or vanilla VAE?", the answer is usually vanilla VAE. VAEs work much better with hierarchical structures. NVAEs and this work take this to the extreme!
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images “Very Deep VAEs” achieve higher likelihoods, use fewer parameters, generate samples 1000x faster, and are more easily applied to hi-res images, compared to PixelCNN. openreview.net/forum?id=RLRX…
8
86
2
562
Posted my first paper on arXiv💥🙌 GPT-f is a Transformer-based automated theorem prover. We show that Transformer + Search is suitable to formal reasoning and continuous self-improvement 🦾 arxiv.org/abs/2009.03393
18
184
32
859
I keep seeing all kinds of crazy reports about people's experiences with GPT-3, so I figured that I'd collect a thread of them.
33
840
246
2,852
Excited to share what I've been working on with @AlecRad, @rewonfc, @ilyasut and others!
We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/
3
27
3
98
Hot Tub Christmas (with GPT-2 lyrics selected by @rewonfc ) is my own favorite -- even though the model can't figure out what's going on in the completely terrible intro. Our weird but fun new holiday tradition! soundcloud.com/openai_audio/…
1
8
27
Very happy to see such a cool application of VQ-VAEs by our friends @OpenAI! Congrats!
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
38
1
209
Excited to finally release what we've been working on since June! Jukebox is a small step in making neural nets produce music! openai.com/blog/jukebox/ Thanks to my collaborators Heewoo, @mcleavey , @_jongwook_kim, @AlecRad, @ilyasut, this work wouldn’t be possible without them!
14
43
8
190
Could a robot replace a New Yorker writer? We fed The New Yorker’s archive to an artificial-intelligence writer, which predicts text based on preceding language. Then we asked it to write for us. nyer.cm/5H11CgF
19
120
47
280
Seeing complex strategies and cooperative behaviors emerge without explicit training is really interesting. In this case, it's also adorable.
We've observed AIs discovering complex tool use while competing in a simple game of hide-and-seek. They develop a series of six distinct strategies and counterstrategies, ultimately using tools in the environment to break our simulated physics: openai.com/blog/emergent-too…
6
1
17
Incredible short sci-fi story on the creation of AGI in a multiagent system: ttapress.com/CrystalNights.p… Btw: we have multiagent team at OpenAI.
2
8
2
55
rewon retweeted
Two mechanisms for publishing GPT-2 (and hopefully future powerful models): staged release and partnership-based sharing. We're: 1. Releasing the 345M model 2. Sharing the 1.5B model with partners working on countermeasures (please apply!) openai.com/blog/better-langu…
20
233
40
668
rewon retweeted
Announcing our fourth #MuseNet "Song of the Day" winner - Chopin meets Mozart: soundcloud.com/openai_audio/… Created by @MiguelMotaPinto
7
65
17
221
Biological Structure and Function Emerge from Scaling Unsupervised Learning to 250 Million Protein Sequences (from FAIR) - unsupervised learning recovers representations that map to multiple levels of biological granularity biorxiv.org/content/10.1101/…
2
71
7
229