Dynabench (@DynabenchAI) header
Dynabench (@DynabenchAI) profile picture

Official dynabench account

Joined September 2020
Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)Photo by Dynabench (@DynabenchAI)
Progress in AI continues to outpace benchmarks. Check out this new plot, inspired by @DynabenchAI, that shows just how quickly it's happening. Read more about it here: contextual.ai/plotting-progr…
5
27
8
110
29,908
Dynabench retweeted
Today, Meta researchers together with @MLCommons working group, are launching DataPerf, the first platform for building data & data-centric AI algorithm leaderboards. We're excited for how DataPerf will help to push the data-centric AI field forward ⬇️

8
25
5
100
27,656
Announcing the BabyLM 👶 Challenge, the shared task at @conll_conf and CMCL'23! We’re calling on researchers to pre-train language models on (relatively) small datasets inspired by the input given to children learning language. babylm.github.io arxiv.org/abs/2301.11796
4
72
12
260
47,301
Great talk today by @max_nlp from @ucl_nlp @CohereAI on Dynamic Adversarial Data Collection at the @kclinformatics Distributed AI group!
2
1
16
Dynabench retweeted
Ever wanted to know more about generalisation in NLP but overwhelmed with the number of papers on ArXiv? Fear not! We read 400+ papers, 600+ experiments, and designed a taxonomy 📝 to categorise the research for you! (1/n) 🧵 arxiv.org/abs/2210.03050
5
123
20
484
How can we improve benchmarking? The @DynabenchAI experiment aims to make faster progress with dynamic data collection, and today, we are pleased to introduce our next stage: @MetaAI has funded 5 exciting research proposals on the theme of "Rethinking Benchmarking”! Congrats to:
2
21
1
82
"DataPerf: Benchmarks for Data-Centric AI Development" What if instead of holding the data constant and benchmarking different models, we held the model constant and benchmarked different data pipelines? [1/7]
3
26
5
120
🧵[1/n] 😍I am very happy to present the first piece of my research on dynamic adversarial data collection (DADC). DADC leads to more trustworthy ML models. Learn how to leverage community adversarial data to make your model more robust💪🏾 huggingface.co/blog/mnist-ad…
4
25
4
103
Talking of the #DADC Community 🤝....Our social is tonight from 6:30-9:30pm @ a rooftop located on 1823 Minor Ave, Seattle (5 min walk from conference venue). There will be 🍕 and 🍹🧃 Spread the word 🗣️ + pls fill in the poll so we can get a sense of numbers 📊
82%I'm attending social
18%I'm NOT attending social
44 votes • Final results
Brb, renaming ourselves to Dynamic Adversarial Donut Collection 🍩  #NAACL #DADC22
1
2
1
13
Our audience interaction @naaclmeeting transcends online-offline worlds 🤯 Who needs a metaverse when you have @boydgraber ? 😅 #NAACL #DADC22
5
3
17
We have an incredible panel lined up for you tomorrow on the Future of Data Collection! Moderated by @adinamwilliams, panelists include @annargrs, @boydgraber, @sleepinyourhat, @tongshuangwu, @laroyo, @douwekiela & @swabhz 🤩 Post any questions you have for them below! 🚀
1
12
7
50
Grateful for the support from @MLCommons. Join us on the 14th July at #NAACL2022 to learn more about dynamic adversarial data collection, @DynabenchAI and much much more! 🚀
DADC improves AI accuracy through higher quality and more diverse data collected with human and AI collaboration. We believe this will help the community build robust ML and why are sponsoring @DADCWorkshop @DynabenchAI at #NAACL2022 mlcommons.org/en/news/dynabe…
1
8
Link to paper - "Dynabench: Rethinking Benchmarking in NLP" arxiv.org/abs/2104.14337 Also see - "Dynaboard: An evaluation-as-a-service platform for holistic next-generation Benchmarking" proceedings.neurips.cc/paper… Is that what we need? 🤔
Many of ML's major benchmarks have already become obsolete. It's getting pretty urgent to re-think ML evaluation.
2
8
Dynabench retweeted
Many of ML's major benchmarks have already become obsolete. It's getting pretty urgent to re-think ML evaluation.
8
73
12
436
Dynabench retweeted
Data collection is slow and expensive, so we give annotators a little help 🤝. Introducing Generative Annotation Assistants (GAAs) to make data collection more efficient and effective 🚀. Work will be presented at #NAACL2022 in Seattle! Paper: arxiv.org/abs/2112.09062 [1/n]
2
27
3
110
We’re happy to announce our new preprint on perturbation augmentation for fairer NLP! We trained a seq2seq control-gen model to “perturb” demographic references. We use it to pretrain & finetune LMs that are fairer, without sacrificing accuracy. We measure fairness with it too 🧵
4
36
10
172
🚨 New paper and datasets! 🚨 After sitting on my hands for many months 😬 I'm delighted that our #Hatemoji paper is going to @naaclmeeting! 😍🤩😎🆒 In a nutshell 🥜it uses human-and-model-in-the-loop learning 🤖🤝🙆 to tackle emoji-based hate A 🧵 on all our new resources 1/
6
25
9
147
Track 1 of our Shared Task kicks off on Mon 2nd May. Do you have a knack for coming up with challenging QA examples? Prove it while competing against the AI and your peers! We will also have prizes thanks to @commons_ml. Sign up at docs.google.com/forms/d/e/1F… 🚀 #NLProc #NAACL2022
2
2
11
We are delighted to announce that the awesome folks at @commons_ml are as excited by data-centric approaches as we are, and have agreed to sponsor the DADC workshop at @naaclmeeting 🥳🥳🥳 1/
2
7
3
16