Aditya Mittal retweeted
LLMs are much less capable judges for code than we might expect.
Super excited that our paper on LLM judges for code has been accepted to COLM! In it, we discuss this weakness and provide a pipeline for understanding and diagnosing issues with LLM code preferences.
This was led by two wonderful students @RyanShar01 and @AdityaM129. It was an absolute pleasure mentoring them on this project.
Aditya Mittal retweeted
The work on comparing developer and LLM biases is led by @RyanShar01 and @AdityaM129.
We show that today's LLM judges remain substantially misaligned with developer preferences in realistic coding workflows.
arxiv.org/abs/2603.24586
Aditya Mittal retweeted
New preprint alert 🚨
Can LLM agents develop video games?
We release GameDevBench, the first benchmark evaluating agentic game development in a game engine, Godot.
We also present two simple multimodal feedback mechanisms that lead to immediate performance gains.
/🧵
Aditya Mittal retweeted
Tired of evaluating LLMs on made-up problems that look nothing like real tasks?
Introducing EDIT-Bench, a code editing benchmark built from in-the-wild user interactions in VSCode.
Real-world edits are challenging: 𝗼𝗻𝗹𝘆 𝟭/𝟰𝟬 𝗺𝗼𝗱𝗲𝗹𝘀 𝘀𝗰𝗼𝗿𝗲 > 𝟲𝟬% 𝗽𝗮𝘀𝘀@𝟭.
Aditya Mittal retweeted
📣 @CopilotArena leaderboard is now live on lmarena.ai/leaderboard!
We have collected over 15k votes on 11 models (2 new models since our last blogpost release). Congrats @deepseek_ai🥇and @AnthropicAI🥇!
Which model is best for coding? @CopilotArena leaderboard is out!
Our code completions leaderboard contains data collected over the last month, with >100K completions served and >10K votes!
Let’s discuss our findings so far🧵
Aditya Mittal retweeted
Introducing Copilot Arena - Interactive coding evaluation in the wild.
Our extension lets you test top models for free, right in VSCode. Let's vote and build the Copilot leaderboard!
Download here:
marketplace.visualstudio.com…
Led by @iamwaynechi and @valeriechen_ at CMU. 1/🧵
Aditya Mittal retweeted
My prediction stands: the bioeconomy will surpass the tech economy in impact and scale. I first discussed this at SynBioBeta 2022 and was inspired by the passionate entrepreneurs I met there. This year, the top minds in synthetic biology will gather in San Jose, May 6-9. Don’t miss out — join, or follow along! synbiobeta.com/attend/synbio…
Aditya Mittal retweeted
We just raised a $28 million Series A from Coatue and @Lux_Capital to build the AWS x GitHub for biocomputing.
1/N🔻
latchbio.substack.com/p/anno…
Aditya Mittal retweeted
demo day was yesterday @theksociety.
we had people from @Google X, @thielfellowship , @SchmidtFutures, and more jam for 2 hours 🌶️
what were your highlights @ community?
Aditya Mittal retweeted
we need more ambition aimed at being part of the solutions for the world's biggest problems.
@theksociety's activate accelerator ends with just that, an earthshot sprint.
more in 🧵
Aditya Mittal retweeted
The students x students team is looking for a newsletter editor! 📝
Apply now:
form.typeform.com/to/waFHlAQ…
DM or email us ([email protected]) for more information or to ask questions.
--
#enjoywriting #student #students #writing #editor #job #team #newsletter #hiring
@BuiltWithBio Day 3 ✅
Learned about extracting sugars from stevia to make synthetic alcohol from @conagen, problems with scalability and continuous fermentation with Ellen Simmons, and problems with ML in syn bio with James Chuang, among others.
Oh, also met @ericschmidt!
Picture 1 - spoke with @ericschmidt on the role Gen Z has in the circular bioeconomy
Picture 2 - picture with extended TKSquad 😎
Picture 3 - talked with technical engineers about automation platform at @conagen
Picture 4 - met James Rosas and Emma Rooholfada about @powbio
@BuiltWithBio Day 2 ✅
Learned of bacterial transfection and chimeric plants from Vladimir Jojic, networking capabilities and language architectures from Armen Babikyan, and Adapsyn and drug discovery from Chris Dejong, among others. Excited for Day 3 and meeting others at RATC!
Aditya Mittal retweeted
the youngest and some of the most ambitious @BuiltWithBio from @theksociety
thank you @SethPeachey and Gingko for helping make this possible
Aditya Mittal retweeted
Taking a tip from @sivers and putting this out sooner than later. I love this one—I was reminded I can externalise my internals in an interesting way.
anchor.fm/arivgupta/episodes…
Going to @BuiltWithBio in Oakland for the next three days with @KarthikMittal3 @Sofiash_ @amyli0 @dev_elio @klarazietlow @michaelraspuzzi - anyone I should meet? Super interested in bioinformatics and DNA-based data storage! 👀
Aditya Mittal retweeted
I'll be in SF from April 11-18 :)
⃞ to go to the @BuiltWithBio conference with @Sofiash_ , @klarazietlow , @AdityaM129 , @KarthikMittal3 , @Sualeha_Irshad and @michaelraspuzzi
⃞ to meet cool humans and people in cancer detection/synbio!
Who are your favourite people in SF?
Aditya Mittal retweeted
Designing Microgrids in Tanzania to programming AI for crops.
A quick recap of my personal Q1.
1/13
Aditya Mittal retweeted
The writers x students discord server is finally complete! 🎉 We are currently transitioning from our writer slack to writer #discord server.
Please check the announcements channel in our writer slack or DM us for the invite link!
--
#enjoywriting #sxtudents #writing #students
