Pinned Tweet
【About the Speaker】Prashant Anand @primaprashant
— Speculative Decoding in 100 Lines of Python
Prashant is the founder of Artifact Four and writes Agentic Coding Weekly. Before that, he spent six years as an ML engineer at Mercari, building production machine learning systems.
His talk is refreshingly concrete. Tools for running LLMs locally, such as llama.cpp, increasingly use a technique called speculative decoding to speed up inference. The idea is to pair a large model with a smaller one: the smaller model predicts the next few tokens, and the larger model verifies those predictions in a single pass.
In around 100 lines of Python, Prashant walks through how the technique works, why language models are slow in the first place, and how the verification process can produce exactly the same output as running the larger model on its own. He'll also run the code on real examples, exploring how many predictions get accepted and why some kinds of text are easier to predict than others.
If you've ever wanted to understand what's happening under the hood of local LLM inference, rather than just tweaking a configuration flag, this is the session for you.
3
58
little something for the coffee lovers here in tokyo
specialtycoffee.tokyo/
3
91
Prashant Anand retweeted
【スピーカー紹介】Prashant Anand — 100行のPythonで学ぶ投機的デコーディング
Prashantさん @primaprashant はArtifact Fourの創業者で、「Agentic Coding Weekly」を執筆しています。それ以前はメルカリで6年間、機械学習エンジニアとして、本番環境の機械学習システムの開発に携わっていました。
今回のテーマは、LLMの推論を高速化する「投機的デコーディング(Speculative Decoding)」。llama.cppなど、LLMをローカルで動かすツールでも活用が広がっている技術です。ポイントは、大きなモデルと小さなモデルの役割分担。小さなモデルが文章の続きを数トークン分先読みし、大きなモデルがその予測を1回の処理でまとめて確認します。
そもそも、LLMの文章生成にはなぜ時間がかかるのでしょうか。そして、大きなモデルだけで生成した場合と同じ出力を保ちながら、どうすれば処理を速くできるのでしょうか。Prashantさんは、約100行のPythonコードを通して、その仕組みを解説します。
さらに、実際にコードを動かしながら、小さなモデルの予測がどのくらい採用されるのか、文章の種類によって予測のしやすさが異なるのはなぜかも探ります。
ローカルLLMの裏側で何が起きているのかを理解したい方に、おすすめのセッションです。
#pycon_tokyo_with_pao
Prashant Anand retweeted
【About the Speaker】Prashant Anand @primaprashant
— Speculative Decoding in 100 Lines of Python
Prashant is the founder of Artifact Four and writes Agentic Coding Weekly. Before that, he spent six years as an ML engineer at Mercari, building production machine learning systems.
His talk is refreshingly concrete. Tools for running LLMs locally, such as llama.cpp, increasingly use a technique called speculative decoding to speed up inference. The idea is to pair a large model with a smaller one: the smaller model predicts the next few tokens, and the larger model verifies those predictions in a single pass.
In around 100 lines of Python, Prashant walks through how the technique works, why language models are slow in the first place, and how the verification process can produce exactly the same output as running the larger model on its own. He'll also run the code on real examples, exploring how many predictions get accepted and why some kinds of text are easier to predict than others.
If you've ever wanted to understand what's happening under the hood of local LLM inference, rather than just tweaking a configuration flag, this is the session for you.
openai models make the codebase worse and hard to read
all models especially astra add artificial line breaks to files
this is so annoying
also, once this pattern seeps into the codebase, all future agents continue to follow this pattern and codebase continuously gets worse
39
giving a talk at pycon mini on 12 Dec, come watch!
Meet the English track at PyCon mini Tokyo with PythonAsia! 🌏🐍
📅 12 December | Shibuya, Tokyo
Cheuk Ting Ho · AI & cybersecurity
David Cournapeau @cournape · scikit-learn creator
Donghee Na @dongheena92 · Python governance
Prashant Anand @primaprashant · LLM inference
Nicholas Dwiarto · CRDTs & Pydantic
Wei Lee @clleew · Airflow & AI agents
Read @_BN_LN Ben Allen’s introduction 👇
pyconjp.blogspot.com/2026/11…
#pyconminitokyo_with_pao
2
57
Prashant Anand retweeted
gihyo.jp @gihyojp にPyCon mini Tokyo with PythonAsia conferenceプログラム担当 3人によるお勧めトークセッションを掲載いただきました!
=====================
登壇者をご紹介✨(敬称略)
=====================
Cheuk Ting Ho|JetBrainsのDeveloper Advocate
David Cournapeau @cournape |scikit-learn作者
Donghee Na @dongheena92|CPythonコア開発者
KAEDE NAKAYASHIKI @yashiki0520|クラウド・IoT
k-kamijo @kei16180339|RDBMSをPythonでスクラッチ実装
山根将司|Pythonの脆弱性を発見・報告
Naoki Inada|CPythonの高速化・省メモリー化
Nicholas Dwiarto|暮らしを便利にするものづくり
ぴっかりん @ra0kley |衛星データ・3D都市モデルの分析
Prashant Anand @primaprashant|NLP・機械学習の実サービス活用
Takayuki Shimizukawa|Sphinxコミッター
Wei Lee|Apache AirflowのPMCメンバー
聴きごたえのあるセッションばかりです!
ぜひ会場へ
#pycontokyo_with_pao
ニュース「12/12開催「PyCon mini Tokyo with PythonAsia conference」お勧めのトークセッション」by @miki_mk___ , @tkoyama010 公開
gihyo.jp/article/2026/11/pyc…
It's quite ironic that both Opus 5.5 and Astra are sticklers for including the license texts in the codebase, when they themselves have been trained by ignoring all the licenses and copyrights
1
42















