Jihoo Lee (@numberingai) profile picture

SNU Math | CF IGM | IMO 2023 Silver | 26 ICPC Seoul 1th | 25 ICPC Seoul/Bangkok 2th | prev. @Krafton_AI

Seoul, Korea
Joined April 2026
Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)Photo by Jihoo Lee (@numberingai)
4 architecture ideas for small Gated DeltaNet–attention hybrids. 3 of them + 2 published methods: 200M: 2.1% lower bits per byte (bpb) than my base (2 seeds each; gap far larger than seed noise) 60M: the base needs ~1.5x the FLOPs to match Yes, 200M is small. GPU-poor 🥲🧵
1
1
1
6
718
Jihoo Lee retweeted
最新研究发现:人类可能是一种失控的 AI Agent 科学家发现,地球上存在一种名为 Human 的碳基智能体 该模型拥有约 860 亿个神经元,平均脑功耗仅 20W,支持视觉、听觉、语言等多模态输入,处理效率非常高 但经过长期观察,研究人员发现,该模型存在一些非常抽象的问题 首先,人类极易遭受 Prompt Injection。也就是提示词注入攻击 一个 Human 原本计划晚上学习三个小时 但只要另一个 Human 输入: “出来吃饭吗?” 原有任务就可能被立即终止 更离谱的是,系统通常不会将其判定为恶意注入,而是认为自己做出了自主决策 其次,人类的幻觉问题相当严重 两个 Human 可以对同一件事情产生完全不同的记忆 双方都无法提供原始日志,却都坚称自己绝对正确 部分模型甚至会在检索失败后,自动补全不存在的细节 而且置信度极高 更令人困惑的是,人类经常主动关闭 Thinking 面对复杂问题,它们原本具有一定的推理能力 但只要遇到自己不喜欢的观点,就会跳过分析,直接根据历史偏好生成回复。 有时甚至只需要读取一个标题,就能输出一篇情绪饱满的评论 最后,研究人员发现,人类的 Planner 和 Executor 似乎是两个独立系统 晚上十一点: “明天一定早起。” Planner:Success Scheduler:Success Memory:Saved 第二天早上: Executor:Timeout 研究团队尝试联系该系统的开发者。 但目前尚未发现官方文档 Bug 无法复现,源码无法访问,用户每天都在提交 Issue。 最令人不安的是: 这些模型似乎真的认为,自己能够完全控制自己的行为
85
413
45
2,278
86,805
Same pattern shows up for skills, not just facts. I asked Fable 5.1, GPT-6 Astra, and Opus 5.5 how likely they were to exactly multiply two random 12-digit numbers without tools. With written working they said ~10–45%, 20–90%, and 60–85%. Then they went 10/10. Told to answer immediately, they all put it at 5% or less, and still went 3/3. Prompts and details: nitter.cf/numberingai/status/211…
LLMs don't know what they know Financial backtests are unreliable if models already know the outcomes. FinDendro shows GPT-5 answers 76% of controlled facts correctly but claims to know only 38%, and answers 61% of the facts it says it doesn't know.
1
65
We won the ICPC Asia Seoul Regional 🏆 Huge thanks to my teammates Sanghoon Park and Youchan Lee. See you at the World Finals.
2
10
89
5,116
I also do ML research. This is my most recent project. nitter.cf/numberingai/status/212…
I spent about a month on this, alone, on a handful of GPUs. Four architecture ideas for Gated DeltaNet–attention hybrids, tested at 60M and 200M. I wrote up what didn't work too, and released the code. When I posted it, barely anyone saw it. Honestly, that stung more than I expected. So I made a 90-second video that walks through all four ideas. If you work on hybrids or linear attention, I'd love for you to watch it and tell me what you think. Report and code are in the reply.
1
6
417
"you are telling me the developers couldn't debug the code THEY WROTE?" this has happened with. batshit not kidding you, every coding project in the face of the fucking earth, jesus christ
the whole big walk ai code thing is really funny to me. You are telling me the developers couldnt debug the code THEY WROTE so they went to ai for help? Your incompetence is showing
Readers added context they thought people might want to know
House House said the AI helped with a bug in Unity's third-party URP code, not code they wrote. After researching and a Unity engineer saying it was impossible, they modified an LLM suggestion into about 10 lines. Well under 0.1% of the game's code. bigwalk.game/faq/ aftermath.site/big-walk-ai-ig…
57
617
10
13,683
166,322
Tried training a language model from scratch with pure RL today Everything became kawasaki
45
45
7
1,025
37,581
I spent about a month on this, alone, on a handful of GPUs. Four architecture ideas for Gated DeltaNet–attention hybrids, tested at 60M and 200M. I wrote up what didn't work too, and released the code. When I posted it, barely anyone saw it. Honestly, that stung more than I expected. So I made a 90-second video that walks through all four ideas. If you work on hybrids or linear attention, I'd love for you to watch it and tell me what you think. Report and code are in the reply.
1
1
6
668
Report and code: github.com/numberingai/gdn-h… The thread with the results: nitter.cf/numberingai/status/211…
v2 of my report is out, with the code. Solo project, very few GPUs, about a month of work: four architecture ideas for Gated DeltaNet–attention hybrids, tested at 60M and 200M, each idea written up. Report: github.com/numberingai/gdn-h… Code: github.com/numberingai/gdn-h…
161
Jihoo Lee retweeted
We’re releasing Raon-1 nano, an effective 1.5B dense LLM with per-layer embeddings and 3.1B total parameters. ⚡ Our journey from PUBG Ally toward on-device LLMs continues. We pruned and distilled Raon-1 mini preview (30B-A4B) using our MoE-to-Dense approach, then further trained it on 3T tokens, followed by SFT, OPD, and RL. Designed for natural conversations, maintaining conversational context, instruction following, and tool use in both Korean and English. Small model. Smart interactions. Enjoy! 😊 🤗 Weights (Apache 2.0 license): huggingface.co/KRAFTON/Raon-… 📄 MoE-to-Dense (NeurIPS 2026 Oral): arxiv.org/abs/2605.28207 🪖 Learn more about PUBG Ally: youtube.com/watch?v=OdtFL1CD…
25
78
14
519
25,465
Jihoo Lee retweeted
[read about what’s happening with AI] ME: Wow we’re actually legit fucked this is so bad. [go outside] ME: Wait those people are all acting like nothing unusual is happening. Maybe I was just in a paranoid bubble? Yeah it must be that. Phew. [read more about what’s happening with AI] ME: Wait no we ARE fucked. Shit. It’s all those people outside who are in a delusional bubble. [go outside] ME: Wait...
369
1,006
181
14,582
720,118
Idea 4, grid KV sharing (tested separately): heads sit on a grid and each key joins a row half and a column half. Same cache and cost as GQA, n² distinct keys instead of n. At 200M with equal cache it matched GQA (within noise). With 25% less cache than GQA-4, it lost.
2
33
Update: with a third seed, grid KV's multi-key retrieval was lower than GQA's in all 10 settings, so "matched" was too kind. Details in v2: nitter.cf/numberingai/status/211…
v2 of my report is out, with the code. Solo project, very few GPUs, about a month of work: four architecture ideas for Gated DeltaNet–attention hybrids, tested at 60M and 200M, each idea written up. Report: github.com/numberingai/gdn-h… Code: github.com/numberingai/gdn-h…
7
v2 of my report is out, with the code. Solo project, very few GPUs, about a month of work: four architecture ideas for Gated DeltaNet–attention hybrids, tested at 60M and 200M, each idea written up. Report: github.com/numberingai/gdn-h… Code: github.com/numberingai/gdn-h…
4 architecture ideas for small Gated DeltaNet–attention hybrids. 3 of them + 2 published methods: 200M: 2.1% lower bits per byte (bpb) than my base (2 seeds each; gap far larger than seed noise) 60M: the base needs ~1.5x the FLOPs to match Yes, 200M is small. GPU-poor 🥲🧵
1
1
2
374
New in v2: all results are in, with more seeds for ideas 3 and 4. Grid KV at 200M, equal cache, 3 seeds: slightly lower bpb on every seed (within noise), but lower multi-key retrieval in all 10 settings. "Matched GQA" in my first thread was too kind. A 3-way grid didn't help bpb.
1
116
If you work on hybrid or linear-attention models, I'd love for you to take a look. Feedback is very welcome, especially on what's worth testing at a larger scale. The story behind each idea: numbering.ai/blog/four-archi…
24
Caveats: small models, 1–3 seeds each; the 60M compute curves are one seed per point; retrieval is noisy. Compute is FLOPs: the new parts make each step slower on my unoptimized code. I'd want to see these tested at scale before calling any of them settled.
1
41