Associate Professor of Cognitive Science and Computational Linguistics
Aarhus, Denmark
Joined September 2017
- Tweets108
- Following326
- Followers191
- Likes322
Some interesting results here, I think! We originally set out on with this report to evaluate the performance of open-source LLMs relative to the major closed-source alternative.
🚨Preprint🚨 We provide a systematic evaluation of the performance of ChatGPT, Open Source LLMs & supervised machine learning classifiers for text annotation. We find that "Chatbots Are Not Reliable Text Annotators" arxiv.org/abs/2311.05769. We discuss results given #openscience
Applying LLMs in humanities and social science contexts is fashionable right now - and for good reason. But with this report we try to urge some caution and reflection on what this means for open science principles.
1
90
To get a feel for how we ran the LLM classification experiments, check out this blog post by my junior colleague and co-author, Márton Kardos. He developed a Python package for doing exactly this:
medium.com/@power.up1163/zer…
102
Ross retweeted
🚨Preprint🚨 We provide a systematic evaluation of the performance of ChatGPT, Open Source LLMs & supervised machine learning classifiers for text annotation. We find that "Chatbots Are Not Reliable Text Annotators" arxiv.org/abs/2311.05769. We discuss results given #openscience
For the Friday evening crowd still mulling over Chomsky’s op-ed, here’s a key passage from our recent letter in @cogsci_soc. There are absolutely some things LLMs cannot do. But what they can do, they do very well - and that demands attention. onlinelibrary.wiley.com/shar…
If you are unsatisfied by Chomsky et al.'s painfully petrified op-ed, boy do I have a read for you! It wasn't planned, but CogSci just published a letter by @MH_Christiansen , @ross_dkm and I arguing quite literally the opposite, at least re:language. onlinelibrary.wiley.com/shar… 🧵
Ross retweeted
If you are unsatisfied by Chomsky et al.'s painfully petrified op-ed, boy do I have a read for you! It wasn't planned, but CogSci just published a letter by @MH_Christiansen , @ross_dkm and I arguing quite literally the opposite, at least re:language. onlinelibrary.wiley.com/shar… 🧵
Ross retweeted
Interested in joining @uantwerp as a postdoc in computational humanities or digital text analysis? There is a nice call from the YUFE network with as focus areas: Sustainability; Digital Society (!), Citizens Wellbeing; and European Identity. PMs welcome.
yufe4postdocs.eu/#msdynttrid…
In which we argue that cognitive science should take seriously LLMs as working models of how human-like language can be acquired without the need for a built-in grammar. Really excited to see this one published. It’s short and it’s free - give it a read! onlinelibrary.wiley.com/doi/…
In a letter in Cognitive Science, @pcontrerask,
@ross_dkm and I argue that Large Language Models - despite their many shortcomings - demonstrate that human-like grammatical language production can be learned from experience alone.
Read it for free here: onlinelibrary.wiley.com/shar…
OpenAI: "Our mission is to ensure that artificial intelligence benefits all of humanity!"
Also OpenAI:
Exclusive: In its quest to make ChatGPT less toxic, OpenAI used outsourced Kenyan laborers earning less than $2 per hour ti.me/3HfiEQ9
4
261
Once casually suggested to a colleague that I thought we could comfortably abolish peer review in its current form without any loss to scholarly rigour. They looked at me like I’d suddenly sprouted horns.
Science ran a big experiment on itself for the last ~60 years. It didn't work out.
experimentalhistory.substack…
6
Programs like ChatGPT can generate credible writing, but only because writing, and our expectations for it, has become so unaspiring, @ibogost writes. theatlantic.com/technology/a…
My addition to the flurry of ChatGPT examples making the rounds today: it makes a pretty decent go at Hallidayan (SFG) transitivity analysis
Ross retweeted
Nine months later after hatching first batch of MSc in Cognitive Science at Aarhus University in 2021: 0% unemployment. Fantastic students. Boring plot.
Ross retweeted
Very excited to help organize a workshop on Large Language Models, Cognition and Creativity at @interact_minds with @ross_dkm. Let the fun begin!
This is such a great opportunity, and you’d get to come work alongside me. Apply now!
Job opportunity: Tenured Associate Prof of Computational Linguistics @AarhusUni_int
Fantastic work environment with potential for collaborations w/ @interact_minds, Center for Humanities Computing @Nielbo23, @AIAS_dk and many others
Deadline Apr 4th bit.ly/369Rzhw
