
CTO at Theia Insights. PhD from @cambridge_cl. Co-organiser of fever workshop (https://nitter.cf/t.co/9FTkHRwec2)
Cambridge, England
Joined October 2013
- Tweets764
- Following581
- Followers1.2K
- Likes1K
James Thorne retweeted
Hi #FactChecking NLPers! As we are preparing to wrap up our image-text claims shared task, a reminder that @FEVERworkshop is welcoming submissions on all related topics with deadline on Dec19. openreview.net/group?id=eacl… fever.ai/
Excited to announce the 9th @FEVERworkshop collocated with #EACL2026 in Morocco: fever.ai/index.html with a new shared task on #factchecking image-text claims with evidence from the web using the AVerImaTeC dataset: arxiv.org/abs/2505.17978 Deadline 2nd of December! And...
James Thorne retweeted
⁉️Why do reward models suffer from over-optimization in RLHF?
We revisit how representations are learned during reward modeling, revealing “hidden state dispersion” as the key, with a simple fix!
🧵
Meet us at @icmlconf!
📅July 16th (Wed) 11AM–1:30PM
📍East Hall A-B E-2608
James Thorne retweeted
Pleased to announce the next @FEVERworkshop at ACL2025! Regular workshop papers (ARR and direct submissions) due 15th of April! And new shared task focusing on reproducible and efficient verification of real world claims! Check fever.ai/ and get keen!
James Thorne retweeted
📢 Excited to share our new paper with @fabreetseo: "Beyond Position: How Rotary Embeddings Shape Representations and Memory in Autoregressive Transformers"!
arxiv.org/abs/2410.18067
Keep reading to find out how RoPE affects Transformer models beyond just positional encoding 🧵
James Thorne retweeted
We are hosting wonderful NLP colleagues at KAIST on their way to ACL Bangkok! 🤩 On-site registration is closed, but the talks will be broadcast on Zoom. Please join us!
Date/Time: Aug 10, 2024, 10:05-12:30 KST (UCT+9)
Parallel Session 1: Advanced Language Models and AI Capabilities, Moderator @seo_minjoon
kaist.zoom.us/my/aliceatkais…
10:05 – 10:30 @mohitban47 (UNC Chapel Hill): Multimodal Generative LLMs: Unification, Planning Agents, Evaluation
10:35 – 11:00 @IAugenstein (U of Copenhagen): Revealing the Parametric Knowledge of Language Models
11:05 – 11:30 Yoon Kim (MIT): Efficient Alternatives to Attention for Sequence Modeling
11:35 – 12:00 @kaiwei_chang (UCLA): From Learning through Labels to Learning through Language
12:05 – 12:30 @PangWeiKoh (UW): Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
Parallel Session 2: Specialized Applications and Societal Impact of NLP, Moderator @NoSyu
z.nosyu.kr
10:05 – 10:30 @VioletNPeng (UCLA): Controllable and Creative Natural Language Generation
10:35 – 11:00 @computermacgyve (Oxford University): Information Integrity and Generative AI: Challenges and Opportunities
11:05 – 11:30 @sunipa17 (Google): Geo Cultural Representation in NLP
11:35 – 12:00 @CamachoCollados (Cardiff University): Practical and Specialised NLP Solutions: The Case of Social Media
12:05 – 12:30 @TristanNaumann (Microsoft Research): Biomedical NLP / Real World Evidence in the Era of Generative AI
James Thorne retweeted
Aligning a diffusion model on preference data WITHOUT a reference model could be nice no?
So, @krasul and I are ideating the use of ORPO to align SDXL 1.0 on PickAPic.
Diffusion ORPO with LoRA 💫
Code and model ⬇️ huggingface.co/sayakpaul/dif…
James Thorne retweeted
🔥Mixtral-8x22B-base + ORPO🔥
7k data & 1.3 hours to build a strong human-aligned 140B chat model🦾
👉IFEval: 65%
👉BBH: 59%
👉MT-Bench: 8.17
More models will be added to the Zephyr-ORPO collection with @argilla_io and @huggingface , stay-tuned😃
The new Mixtral-8x22B base model is a total beast for fine-tuning and has produced some of the highest scores I've ever seen on IFEval and BBH 🤯
We teamed up with @argilla_io and @kaist_ai to cook up a brand new recipe for Zephyr models 🪁
huggingface.co/HuggingFaceH4…
🧑🍳 Align the base model with Odds Ratio Preference Optimisation (ORPO). This novel algorithm does not require an SFT step to achieve high performance and is thus much more computationally efficient than methods like DPO and PPO
🦫 Use a brand new dataset of 7k high-quality, multi-turn preferences that has been developed by our friends at @argilla_io: huggingface.co/datasets/argi…
As usual, we are open sourcing the training code in the Alignment Handbook for the community to build on: github.com/huggingface/align…
This has been a epic speed run with @jiwoohong98 @nlee288 @alvarobartt - now I can finally sleep 😂
James Thorne retweeted
Great work! I’m glad our benchmarks KoBBQ, KOLD, and CLIcK are used to evaluate HyperCLOVA_X. Making progress in Korean LLM! 🇰🇷🇰🇷🇰🇷
arxiv.org/abs/2307.16778
arxiv.org/abs/2205.11315
arxiv.org/abs/2403.06412
Happy to share that our #HyperCLOVA_X tech report was released!
James Thorne retweeted
This is actually huge:
- No SFT stage (e.g., Zephyr used 200k examples)
- Preference tuning with 7K examples only (other models trained with at least 60k samples)
I've put a lot of care & love building the DPO version of the amazing Capybara dataset from @ldjconfirmed so I'm really pleased to see these results.
Let's double down on useful open data for OSS AI developers and researchers
📢New model, Mistral-ORPO-Capybara-7k in ORPO collection!🧵
With 💡ORPO💡 + 7k Capybara preference pair by @argilla_io🔥 + Mistral (7B), you can get the human-aligned chat model within 2.5 hours of fine-tuning👀
👉AlpacaEval 2.0 (LC): 15.9%
👉MT-Bench: 7.44
👉IFEval: 61.27%
James Thorne retweeted
🚀Calling all astronaut economists! Introducing Theia Theme Connect, where you can travel the cosmos by connecting companies and themes. Can you navigate through the stars and matrix of data before your oxygen runs out? 🔗Click here to launch into action: connect.theiainsights.com/?u…
James Thorne retweeted
I can’t keep up with all these new papers and datasets from my students 😊 @j6mes and I wanted to give #kaist undergrad students research experience with LLMs in their own languages, and this is one of the outcomes of that project!
James Thorne retweeted
A group photo from the poster presentation of »AmbiFC: Fact-Checking Ambiguous Claims with Evidence«, co-authored by @Max_Glockner (@UKPLab), Ieva Staliūnaitė (@Cambridge_CL), @j6mes (@kaist_ai) , Gisela Vallejo (@unimelb), @vlachos_nlp (@Cambridge_CL) and @IGurevych. #EMNLP2023
James Thorne retweeted
⭐️Congratulations to everyone at KAIST AI who have a total of 20 papers accepted to #NeurIPS2023! Check them here 🧵docs.google.com/document/d/1…
James Thorne retweeted
⭐️Congratulations to everyone at KAIST AI who has a total of 11 papers accepted to #EMNLP2023! Check them here 📷🧵
docs.google.com/document/d/1…
James Thorne retweeted
Come and enjoy on Fri, Oct 27 at 6 pm, as @RMBandService and @ktigers_ent do a joint performance at 📍Gwanghwamun Plaza in front of the Sejong Center to celebrate 🇬🇧🇰🇷140yrs.
James Thorne retweeted
Excited to present 🔥Prometheus, a fully open-source evaluator LM that is on par with GPT-4 evaluation when the “appropriate” reference materials are appended!
* Could generalize to customized score rubrics
* Shows high correlation with both human evaluators & GPT-4 evaluation
James Thorne retweeted
Are open-sourced LLMs really good? 👀
We introduce FLASK🧪, a fine-grained evaluation based on skill sets! Even SOTA open-sourced LLMs such as LLaMA2 Chat 70B lag behind proprietary LLMs for some abilities. 🤯
Paper: arxiv.org/abs/2307.10928
Demo: kaistai.github.io/FLASK
James Thorne retweeted
⭐️Congratulations to everyone at KAIST AI who have a total of 24 papers accepted to #iclr2023, including 9 spotlights, with papers in #nlproc #computervision #machinelearning #rl and more! ⭐️ Check them here 🧵docs.google.com/document/d/1…
James Thorne retweeted
Can we afford to deploy 🤯175B+ models for CoT reasoning? Maybe not, but thankfully, they are remarkable at 🧑🏫 teaching small models.
We show that a fine-tuned 0.3B student can sometimes outperform InstructGPT, and more... on an ✨academic budget ✨
arxiv.org/abs/2212.10071 🧵
James Thorne retweeted
What an inspiring day @akbc_conf today! Amazing speakers and a fun vibe throughout. Looking forward to tomorrow’s schedule, always @BarbicanCentre. Also, we have some space left for the social event @BrewDogTowerH - grab your ticket! eventbrite.com/e/akbc-2022-t…





























