Daniel Ramos (@danieltrt7) profile picture

Software / Research Engineer

Amsterdam, The Netherlands
Joined June 2020
Photo by Daniel Ramos (@danieltrt7)Photo by Daniel Ramos (@danieltrt7)Photo by Daniel Ramos (@danieltrt7)Photo by Daniel Ramos (@danieltrt7)
🚨 Are Large Language Models Memorizing Bug Benchmarks? 🚨 There’s growing concern that LLMs for SE are prone to data leakage, but no one has quantified it... until now. 🕵️‍♂️ We measured leakage in benchmarks like Defects4J, and SWEBenchLite. arxiv.org/pdf/2411.13323 Findings👇
2
8
1
30
2,410
I’ve officially deleted more code than I’ll ever write 😄
Over 1 million lines of dead code removed. 🧹 Meet AutoClean, Uber's AI-powered system that's cleaned up 16,000+ stale feature flags using AI agents, deterministic code transformations, and validation.
2
142
Daniel Ramos retweeted
Apparently EU users don't get new AI versions till several months after US users, because AI companies have to get regulatory signoff first. Was the cost of overregulation ever so clear? Every user in Europe is several months behind, in a field that changes every several months.
386
450
97
3,920
349,466
Daniel Ramos retweeted
Thrilled to announce our new work TestGenEval, a benchmark that measures unit test generation and test completion capabilities. This work was done in collaboration with the FAIR CodeGen team. Preprint: arxiv.org/abs/2410.00752 Leaderboard: testgeneval.github.io/leader…
4
10
1
29
3,837
Looking for students! @johny_dry and I proposed Meerkat, a live, distributed, reactive PL, in an Onward! vision paper this year. We'd love to work with a Ph.D. student in the joint @CMUPortugal program, who wants to help make this a reality! Due Dec 11. dl.acm.org/doi/10.1145/36894…
2
6
15
1,799
Daniel Ramos retweeted
🚀 Curious about GitSEED accepted @SIGCSE_Virtual and how it’s transforming programming education? Check out my full thread on Bluesky for all the details! 🌟 #SIGCSE #CSforALL #GitLab 🔗bsky.app/profile/pmorvalho.b…
2
5
112
(3/4)📜 5-gram match reveals memorization: We used 5-gram match to check if models generated nearly identical outputs when given the same input. CodeGen scored 82% on Defects4J!
1
3
154
(4/4) 📈 Llama 3.1 (70B)—trained on far more data—shows less leakage than older and smaller models like CodeGen and CodeLlama. Its higher NLL and lower 5-gram match show limited signs of leakage. Read the preprint: arxiv.org/pdf/2411.13323 #AI #SoftwareEngineering #DataLeakage
3
377
Daniel Ramos retweeted
I'm excited to share that I'll be joining @UCIrvine as an Assistant Professor starting in Fall 2025! I'm looking forward to working with amazing colleagues and students to empower programmers using AI and HCI. Come join us—I’m recruiting students!
39
52
6
570
56,197
Do you work with Ansible? We want to hear from you! 🎯 This study is led by a joint team from @CarnegieMellon and @istecnico Plus, if you agree to an interview, you'll be entered into a $100 raffle! 💸 Click here to participate: cmu.ca1.qualtrics.com/jfe/fo…
7
3
10
2,526
I am on the #SoftwareEngineering #Teaching Job Market! After co-creating & co-instructing two software engineering courses at CMU I am looking for teaching-focused faculty positions in SE. If you know anyone who is hiring, please let me know! Retweets welcome.
4
14
2
28
6,565
It’s tomorrow at #PLDI2024 in Iceland/Denmark at 10:40 AM! 📅 Check out the proceedings online: dl.acm.org/doi/10.1145/36564…
Excited to present our work "A Lightweight Polyglot Code Transformation Language" next week at @PLDI2024! danieltrt.github.io/papers/p… Are you dealing with large-scale refactoring/migrations? Is your codebase multilingual? We’ve got something for you! 🧵👇 [1/n]
2
7
379
Excited to present our work "A Lightweight Polyglot Code Transformation Language" next week at @PLDI2024! danieltrt.github.io/papers/p… Are you dealing with large-scale refactoring/migrations? Is your codebase multilingual? We’ve got something for you! 🧵👇 [1/n]
1
2
1
17
1,189
At @UberEng, we’ve fully automated stale feature flag clean-up and performed API migrations with this toolkit! Our tools, built on top of PolyglotPiranha, have landed over 2000 PRs and cleaned up over 200k lines of code![4/n]
1
4
200
If you're going to be at @PLDI , come attend my talk on June 28th at 10:40! This amazing work was led by @ask1604 , Lázaro Clapp, @rajbarik, and Murali Krishna Ramanathan at @UberEng 's Programming Systems Group! [5/5]
2
7
631