jon_chun (@jonchun2000) header
jon_chun (@jonchun2000) profile picture

AI Researcher in GPT/LLMs, NLP, multimodal-sentiment, AI XAI/FATE and time series. Experience in Med/Fin/InsurTech, security and startups.

Kenyon College
Joined October 2017
Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)Photo by jon_chun (@jonchun2000)
This is a critically flawed paper that sacrifices science at the alter of narrative. It moves silently between fact and speculation, obscuring the inconsistency of its central claim. An autopsy 👇 Let's start with the facts, largely clear by now to those following along: (1/)🧵
The False Promise of Imitating Proprietary LLMs Open-sourced LLMs are adept at mimicking ChatGPT’s style but not its factuality. There exists a substantial capabilities gap, which requires better base LM. arxiv.org/abs/2305.15717
1
4
2
10
4,945
1920’s AI robot archives via @runwayml Gen-1 h/t reddit.com/r/aivideo/comment…
19
145
15
574
75,507
Ok… So this week: - Google Bard - Adobe Firefly - NVIDIA Foundations - Bing adds new DALL-E - Opera adds AI - Microsoft Loop - Canva puts all the AIs in - ChatGPT starts adding plugins Who’s keeping up? What did I miss? Anyone else exhausted?
140
318
62
2,076
288,013
Bain Capital Ventures just announced $1.9B in new funds 8VC just announced a $880M fund Felicis just announced a $825M fund SignalFire just announced a $900M fund GGV is raising $2.5B in new funds Khosla Ventures is raising $3B in new funds etc. This is not 2001.
26
109
22
1,213
345,011
My Twitter timeline is full of panicked takes about imminent AI apocalypse and certain doom. I think this is starting to get overplayed, and so I want to make a long thread about why I'm personally not worried yet. Get ready for a big one... 1/n
71
302
120
1,749
935,941
So many of the study techniques we learned in school were actually ineffective. Here are the result of a large meta-analysis of study techniques. You should skip highlighting, summarizing, and rereading. Instead, practice, quiz & explain. journals.sagepub.com/stoken/… AI can help! 1/
16
357
58
1,981
327,496
Very, very interesting: AI without hallucinations is very boring and much less transformative Bing now lets you chose between creative and precise mode. Here's "find me interesting connections between the theory of diffusion of innovations, Swiss cheese, and US Patent 3,387,396"
14
40
11
497
91,419
jon_chun retweeted
There are AI denialists, AI cautionists, AI boomerists and AI doomerists. I am merely a curios and fascinated AI expectationist.
36
21
7
263
43,256
Replying to @ESYudkowsky
How Far Are We From AGI?
1
1
1,725
Replying to @Hritikpawar9
This Far |----------------------| (not to scale)
2
2
85
1,794
What’s the best way to use pre-trained features when you only have a very small training dataset, (e.g. 2 images per class)? We find that projecting the features onto a good low-dimensional basis can improve generalization a lot. Paper: arxiv.org/abs/2302.05441 🧵  1/8
4
50
4
230
64,782
Replying to @nonmayorpete
Google lost $100 b in market cap after the demo failed. Not making this up cnbc.com/2023/02/08/alphabet…
1
1
4
780
What if you could fit an *entire codebase* in an LLM? 🤔 "Efficiently Scaling Transformer Inference" (11/2022) arxiv.org/pdf/2211.05102.pdf Jeff Dean + co break out all the hacks to scale PALM-540B's context length to 43,000 tokens! Here's how 👇
23
160
26
1,116
250,215
I played with @AnthropicAI assistant (AA) and compared it to @OpenAI ChatGPT TLDR: both are similar but AA is + Harder to jailbreak + Tries to be more helpful + Follows closer what we ask for + ~Better for writing in English - Worst for code - Worst in French - Longer resp. 🧵
10
63
11
283
95,524
Are Flan-T5, T0, and OPT-IML the only open-source, instruction-trained models? Maybe @AiEleuther has something in the works to compete with OpenAI's InstructGPT models?
7
9
1
107
42,709
Replying to @PeterHndrsn
The compressed database argument will fail because SD compresses 8exabytes into 8gigabytes, essentially arguing that it stole less than 0.0000001% of each low res image, which is less than 1 pixel. It's a nonsense argument.
1
1
9
452
Recommendation at family dinners: asking 5 year olds for AI art prompts.
19
167
11
1,669
221,502
Ok so I downloaded all ~322 episodes of @lexfridman podcast and used OpenAI Whisper to transcribe them. I'm hosting the transcriptions on... "Lexicap" ;) : karpathy.ai/lexicap. Raw vtt transcripts are included for anyone else who'd like to play (they are quite great!)
205
563
94
5,206
I just used 3,000 GPU-hours to test all 9 new OpenAI Whisper speech recognition models, two Talon acoustic models, and NVIDIA Nemo large and xlarge models. Whisper has a peculiar failure case. Here's what I think.
24
168
28
1,322
One way to read the Unnatural Instructions paper is as a clear indictment on human effort. For most knowledge tasks, we can now assume language models are 10x to 100x cheaper than humans. This is striking! As a society, we are unprepared for this tsunami. arxiv.org/abs/2212.09689
6
17
4
117
24,269