
👩🎓 PhD, 🧠 AI Researcher, 📺 ML videos creator on YouTube: https://nitter.cf/t.co/yd92HLx342 ▶ @AICoffeeBreak
- Tweets425
- Following252
- Followers1.4K
- Likes1.5K

ALT Illustration of the test principles of self-consistency tests and unfaithful model answers, simplified for brevity. Model input is italicised. Autom. Eval.: Test can be evaluated automatically, i.e., without semantic evaluation of the generated explanation; Annot. Free: No annotated data needed. Data Agnostic: Test is applicable to any dataset/task. Test Type: Tested samples yield i) a fail/pass or ii) a continuous value as faithfulness measure; Expl. Type: Applied to post-hoc or CoT NLE. ✓ / ✗: Fulfils / does not fulfil the property. ①: Needs a helper model trained on task-specific data. ③: Needs manual checking whether the model mentions the bias in the explanation or not. ③: Needs annotated data for incorrect answers proposal. ④: Requires a few-shot prompted helper model for some edits. ⑤: ComVE input reconstruction requires annotation for the sentences against common sense.























