Game dev, researcher, and industry analyst. Formerly @riotgames, @meta. Currently doing stuff with AI.
Los Angeles, CA
Joined February 2009
- Tweets988
- Following906
- Followers340
- Likes5.1K
The challenges of using LLMs to power NPC dialogue. Latest from Oscillating Wildy -> open.substack.com/pub/christ…
While LLMs performance on games might seem niche (and these 'benchmarks' are obviously very informal), I think performance in games is a pretty good assessment of a model's ability to reason about a system from first principles.
The models showed some flashes of goal setting, planning, optimization, and exploitation, but were mired with inconsistency.
I switched to a reasoning model. The beefy o3 model was a bit too slow for this purpose, so I took the o4-mini out for a spin instead. The results were better, but then it got stuck in a loop of insisting that I continue to save gold for no reason.
But things went off the rails pretty quickly. It would directly contradict rules, tell me to do some really stupid things, or just plain hallucinate. Like ask me to buy things in the store that weren't there.
Initially, GPT-4.1 seems to have a solid grasp of what was going on. Along with a couple pointers, it told me to buy a unit in the shop that would complement one of my own on the bench. It then told me to put those two units on the board. So far so good.
I let two OpenAI models directly read game state information each round and asked it to take actions. Having them input actions directly would get me caught by anti-cheat, so I inputed the actions myself.
An informal benchmark I love for assessing LLMs’ capacity for complex reasoning is how well they play games. So naturally I tested how well some play Teamfight Tactics ->
Christian Yoder retweeted
Introducing our first game in development, code-named Project Loki. It’s a group adventure, a competitive sandbox and a highlights factory, all at once. It’s a blend of genres players have described as “League meets Apex meets Smash.”
PLAYTEST SIGNUP @ theorycraftgames.com/
Christian Yoder retweeted
Can ChatGPT play a social deduction game? We answered the question by plugging ChatGPT into Trust No Bunny -> youtu.be/7kRCze_H3ts
Three days on Steam in early access and the reviews for Trust No Bunny have so far been fantastic. Really proud of the product we have shipped and still plenty more to come.
If you're a fan of tabletop/ social deduction games, be sure to check us out.
Trust No Bunny, our take on the social deduction genre for 5-8 players, is now available early access on Steam. We'd love for you to check us out!
store.steampowered.com/app/2…
Christian Yoder retweeted
Less than 24hrs till we’re live on Steam. Wishlist if you haven’t yet - it helps🥹store.steampowered.com/app/2… #gamedev
