rewon retweeted
Excited to announce that we’ve raised $1.3B to build one of the largest clusters in the world and turbocharge the creation of Pi, your personal AI.
forbes.com/sites/alexkonrad/…
rewon retweeted
It’s a big week! We’ve raised $1.3 billion and are building the world’s largest AI cluster (22k H100s).
We’re grateful for our investors and new funding that will help us accelerate our mission to make personal AI available to every person in the world. inflection.ai/inflection-ai-…
rewon retweeted
We have amazing results to announce! Inflection-1 is our new best-in-class LLM powering Pi, outperforming GPT-3.5, Llama and PALM-540B on major benchmarks commonly used for comparing LLMs. inflection.ai/inflection-1
rewon retweeted
I've ported @rewonfc's very deep VAE (arxiv.org/abs/2011.10650) from PyTorch to JAX/Flax! Hope other JAX users find this SOTA VAE useful as a forkable baseline... github.com/j-towns/vdvae-jax.
rewon retweeted
My post on SotA image generative models was released 🥳
Featured 7 notable recent papers with emphasis on:
- VD-VAE
- VAE + discriminator (e.g. VQGAN, DC-VAE)
- Diffusion models (e.g. DDPMv2)
Plus some notes on scaling (e.g. DALL-E) and evaluation.
arankomatsuzaki.wordpress.co…
rewon retweeted
IMO, best empirical proof to date that AI can be creative. After this sinks in, will there be any naysayers left?
"The images are preprocessed to 256x256 resolution during training. [...] each image is compressed to a 32x32 grid of discrete latent codes using a discrete VAE that we pre-trained using a continuous relaxation."
GPT + VAE + scale = impressive results!
openai.com/blog/dall-e/
rewon retweeted
It breaks my 💚 when researchers tell me that VAEs don't work. My first typical question is "did you try hierarchial VAE or vanilla VAE?", the answer is usually vanilla VAE.
VAEs work much better with hierarchical structures. NVAEs and this work take this to the extreme!
Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images
“Very Deep VAEs” achieve higher likelihoods, use fewer parameters, generate samples 1000x faster, and are more easily applied to hi-res images, compared to PixelCNN.
openreview.net/forum?id=RLRX…
rewon retweeted
Posted my first paper on arXiv💥🙌
GPT-f is a Transformer-based automated theorem prover. We show that Transformer + Search is suitable to formal reasoning and continuous self-improvement 🦾
arxiv.org/abs/2009.03393
rewon retweeted
I keep seeing all kinds of crazy reports about people's experiences with GPT-3, so I figured that I'd collect a thread of them.
rewon retweeted
We found that just as a large transformer model trained on language can generate coherent text, the same exact model trained on pixel sequences can generate coherent image completions and samples. openai.com/blog/image-gpt/
rewon retweeted
Hot Tub Christmas (with GPT-2 lyrics selected by @rewonfc ) is my own favorite -- even though the model can't figure out what's going on in the completely terrible intro. Our weird but fun new holiday tradition! soundcloud.com/openai_audio/…
rewon retweeted
Very happy to see such a cool application of VQ-VAEs by our friends @OpenAI! Congrats!
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code: openai.com/blog/jukebox/
rewon retweeted
Excited to finally release what we've been working on since June! Jukebox is a small step in making neural nets produce music! openai.com/blog/jukebox/
Thanks to my collaborators Heewoo, @mcleavey , @_jongwook_kim, @AlecRad, @ilyasut, this work wouldn’t be possible without them!
rewon retweeted
Could a robot replace a New Yorker writer? We fed The New Yorker’s archive to an artificial-intelligence writer, which predicts text based on preceding language. Then we asked it to write for us. nyer.cm/5H11CgF
rewon retweeted
Seeing complex strategies and cooperative behaviors emerge without explicit training is really interesting. In this case, it's also adorable.
We've observed AIs discovering complex tool use while competing in a simple game of hide-and-seek. They develop a series of six distinct strategies and counterstrategies, ultimately using tools in the environment to break our simulated physics: openai.com/blog/emergent-too…
rewon retweeted
Incredible short sci-fi story on the creation of AGI in a multiagent system: ttapress.com/CrystalNights.p…
Btw: we have multiagent team at OpenAI.
rewon retweeted
Two mechanisms for publishing GPT-2 (and hopefully future powerful models): staged release and partnership-based sharing.
We're:
1. Releasing the 345M model
2. Sharing the 1.5B model with partners working on countermeasures (please apply!) openai.com/blog/better-langu…
rewon retweeted
Announcing our fourth #MuseNet "Song of the Day" winner - Chopin meets Mozart: soundcloud.com/openai_audio/…
Created by @MiguelMotaPinto
rewon retweeted
Biological Structure and Function Emerge from Scaling Unsupervised Learning to 250 Million Protein Sequences (from FAIR)
- unsupervised learning recovers representations that map to multiple levels of biological granularity
biorxiv.org/content/10.1101/…






























