
Sr. Applied Scientist @Amazon in Berlin • Previously @UKPLab @TUDarmstadt • Opinions are my own
Deutschland
Joined August 2010
- Tweets68
- Following534
- Followers341
- Likes2.3K
Andreas Rücklé retweeted
📢📢📢 The Eval4NLP workshop will take place this year at AACL 2023. Special focus: Evaluation of/with LLMs. Including a shared task on Prompting LLMs as Explainable Metrics. 📢📢📢
Direct submission deadline: 25.08.
Webpage: eval4nlp.github.io/2023/inde…
CFP: eval4nlp.github.io/2023/cfp.…
Andreas Rücklé retweeted
Happy to share the results of the Efficient NLP survey (07/2021) looking into 🌱 environmental impact , 💻 access to compute , and 🕵️ its impact on peer reviewing! Across 312 participants, we find: 👉 a high disparity wrt. compute power 👉 a high support to make things better! 🧵
Andreas Rücklé retweeted
We are excited to release Version 3 of the AdapterHub.ml framework!
We now support more architectures
- Prefix Tuning
- Compacter
- Parallel Adapter
and make it easy to UNION i.e. Mix-and-Match adapters
Our blog covers results and changes: adapterhub.ml/blog/2022/03/a…
Andreas Rücklé retweeted
Happy to announce, we just released v2.2.0 of adapters-transformers🥳
What's new:
🆕Adding adapter support for T5
🏋️AdapterTrainer for fine-tuning adapters
🔮Prediction Heads for language modeling
All changes: github.com/Adapter-Hub/adapt…
Andreas Rücklé retweeted
I’m happy to announce🍻BEIR was accepted at #NeurIPS2021 Dataset Track!!😍
We received good feedback for our diverse zero-shot #IR benchmark suite with 18 datasets and 9+ domains. We will continue to improve #BEIR, but for now, let’s celebrate!😎🍻
Link: openreview.net/forum?id=wCu6…
Andreas Rücklé retweeted
Replying to @clifapt @huggingface
Also, happy to add that our paper “What to Pre-Train on? Efficient Intermediate Task Selection” (w. @PfeiffJo & @arueckle), for which all adapters were trained, has been accepted to #EMNLP2021. Code & updated version following.
arxiv.org/abs/2104.08247
nitter.cf/clifapt/status/1383114…
In our new paper
“What to Pre-Train on? Efficient Intermediate Task Selection”
we investigate efficiently selecting the best intermediate training tasks for a target task in an adapter-based transfer setup.
w. @PfeiffJo @arueckle
tinyurl.com/WhatToPreTrainOn
Andreas Rücklé retweeted
Tomorrow at the ACL Business Meeting we will have a panel discussion (14:30–15:30 UTC+0, all registered for ACL can attend) to get feedback on policy recommendations relating to the trends of large computational budgets and efficiency in NLP.
This is an exciting panel including:
Andreas Rücklé retweeted
“How Good is Your Tokenizer?
On the Monolingual Performance of Multilingual Language Models”
by Philip Rust, @IGurevych, @PfeiffJo, @seb_ruder, @licwu
🕐: Session 9A, 03.08, 11:00 — 12:00 GMT
📃: aclanthology.org/2021.acl-lo…
🎥: underline.io/events/167/sess…
Andreas Rücklé retweeted
We have just released v2.1.0 of adapter-transformers!
What's new:
🤗Integration into @huggingface's Model Hub (⬇️)
🗂 Split input batches between adapters
🔄Automatic prediction head conversion
All changes: github.com/Adapter-Hub/adapt…
Andreas Rücklé retweeted
Extremely honored by the best long paper award at EACL 2021 @eaclmeeting! Paper (with my dear friend and co-author @licwu):
aclweb.org/anthology/2021.ea…
Check out BEIR, our new IR benchmark with 17 diverse datasets from different domains + tasks! The paper thoroughly compares nine(!) different IR models and provides interesting insights such as cosine vs dot product for dense IR. Work by @Nthakur20 @Nils_Reimers @abhesrivas
🚨New paper alert 🚨
🍻 BEIR: a heterogeneous benchmark for IR. 17 datasets, 9 tasks with diverse domains. 9 SOTA retrieval models evaluated in a zero-shot setup.
w/ @Nils_Reimers @arueckle @abhesrivas, IG at @UKPLab
pdf: arxiv.org/abs/2104.08663
More details, code 👇
#NLProc
SciGen is a *very* challenging dataset for table-to-text generation, which requires models to reason over scientific tables. Thanks to everyone who participated in the annotation process, your help was highly appreciated!
This Post is from an account that no longer exists. Learn more
Check out our new paper where we broadly study how to select intermediate pre-training tasks for adapter-based transfer learning. The most complex methods are not always the best! Work with @clifapt @PfeiffJo
In our new paper
“What to Pre-Train on? Efficient Intermediate Task Selection”
we investigate efficiently selecting the best intermediate training tasks for a target task in an adapter-based transfer setup.
w. @PfeiffJo @arueckle
tinyurl.com/WhatToPreTrainOn
New paper w/ @GregorGeigle @Nils_Reimers where we study approaches to efficiently identify suitable QA agents specialized in different kinds of questions! In a simulated environment, TWEAC is not only effective but also scales well to hundreds of QA agents.
Check out our new publication:
“TWEAC: Transformer with Extendable QA Agent Classifiers”
We propose a meta-QA to find the most fitting QA systems within a pool for a given question.
w/ @Nils_Reimers @arueckle IGurevych
arxiv.org/abs/2104.07081
Code & data released as well
Andreas Rücklé retweeted
Recent Advances in Language Model Fine-tuning
New blog post that takes a closer look at fine-tuning, the most common way large pre-trained language models are used in practice.
ruder.io/recent-advances-lm-…
Andreas Rücklé retweeted
Very cool new toolkit built on top of adapter-transformers
We are happy to introduce our new Python toolkit - Trankit, a light-weight transformer-based toolkit for multilingual natural language processing that can process raw text and support fundamental NLP tasks for 56 languages. github.com/nlp-uoregon/trank….
Andreas Rücklé retweeted
Check out our new paper:
"UNKs Everywhere: Adapting Multilingual Language Models to New Scripts"
We propose a series of methods that enable adaptation of pretrained multilingual models to low-resource languages
and unseen scripts.
\w @licwu IG @seb_ruder
tinyurl.com/UNKsev
Andreas Rücklé retweeted
These tutorial slides on "High Perf NLP" are really impressive. Every slide is current to the minute. Amazing set of diagrams.
gabrielilharco.com/publicati…
(@gabriel_ilharco @Tim_Dettmers @IuliaTurc @kentonctlee Felipe Ferreira Cesar Ilharco)
Andreas Rücklé retweeted
Very nice demo session by Clifton Poth @PfeiffJo and @arueckle on @AdapterHub. There are some cool colab notebooks using @huggingface for how to use adapters for fine-tuning on down-stream tasks.
adapterhub.ml/emnlp/
Andreas Rücklé retweeted
Make sure to meet us at our demo session on the @AdapterHub at #EMNLP2020.
We have a new blog post coming with it including some cool Colab scripts for you to play around with:
adapterhub.ml/emnlp
Gather-2M:
Nov 17
10:00-12:00 UTC / 11:00-13:00 CET
virtual.2020.emnlp.org/paper…
































