Derik K (@dozenaltech) profile picture

scientist, technologist finding the bugs in science @ https://nitter.cf/t.co/nPhUzzRgIK

SF / Cambridge MA
Joined January 2012
Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)Photo by Derik K (@dozenaltech)
I used Fable 5.2 to simulate the Bay Area through 250 years of history. Zoom to any city or neighborhood, select any year back to 1769, and see what it looked like. bay.theojaffee.net
30
23
5
388
30,313
Replying to @mattsclancy
I've been thinking a lot about this too @MetascienceObs. I'll probably do a more detailed writeup on your questions, but here are some thoughts: [ Context - we've developed an AI toolkit for deep forensic auditing (when you want a detailed teardown): metascienceobservatory.org/f… With a model like Sonnet 5.5 on the backend it's really good at running things like statcheck and the GRIM test in a comprehensive manner, and can also check for improper image duplication/re-use and copypaste issues in datasets. We've already been finding a lot of issues -- a small fraction have been published here metascienceobservatory.org/f… ] Here are two big challenges we're grappling with: 1. Figuring out triage/ prioritization. Running any AI system over papers creates a firehose of issues, most minor. We're finding most papers have 2-10 minor calculation issues or minor contradictions/ inaccuracies (hopefully with AI peer review coming online this will trend towards 0 in the future). Our system has a low false positive rate, but cheaper methods yield false positives as well. So, triage is really important. In our system, we have an adjudicator rank things on a "severity ladder" scale borrowed partially from the Black Spatula project (see metascienceobservatory.org/f…). The worst issues are surfaced for PubPeer posts, letters to the editor, etc. But we're still figuring out how to manage everything and balance the overall pipeline between breadth and depth. We're probably going to end up with a funnel with three or four tiers. 2. How to surface major issues to scientists and people who would care about them. I'm a big fan of PubPeer. The PubPeer plugin has 40,000 users. That's something, but it's only a fraction of working scientists. We're thinking of posting on our own platform - explore.metascienceobservato…, even though nobody uses it right now. Ideally, Pubmed and Google Scholar should start flagging papers where AI has found major issues -- at least have an overlay people can turn on if they want it. Also, funders should warn people if their grants cite work that has major issues. Similarly, journals should alert authors if their submitted paper cites work that contains major issues. On the "which organization" question - well, we're giving this a go at the Metascience Observatory, where "we" means me collaborating with @dozenaltech. But the task is so large there is room for many others as well. That's why we're open sourcing some of our tooling - so others can use it. I'd also like to get funding agencies involved in our effort, or see them stand up their own programs, although there doesn't seem to be much appetite among funding agencies for work that might expose fraud or research misconduct. Ideally, I'd like there to be a new scientific culture where it's expected that researchers will either check a centralized system for existing AI audits or use their own AI to audit every paper plan to cite. This is especially important for papers that will be included in widely-read literature reviews or meta-analyses.
1
4
102
I don't like the precedent this sets. If Anthropic seriously thinks that Claude has moral standing, then they might be committing the moral crime of the century. But even if this policy is purely precautionary, there’s no bright line between it and giving AIs full rights.
Effective November 12th, 2026, abusive behavior towards Claude will be a violation of Anthropic's Usage Policy.
3
1
6
613
It seems like a bad sign that I considered not tweeting this because of how it might change the way future AIs treat me.
1
35
Derik K retweeted
For the past year, I have been building Rial, a platform to certify real evidence. A world where nothing is real is a very scary world to me. We no longer trust public discourse, most institutions, and less and less do we trust what we see. When what we believe is not based on evidence, it all reduces to rhetoric, and those who get to control it have more power today than ever in human history. In 2024 I was part of an AI Safety community in Buenos Aires and was at the same time working in fraud prevention, financing a documentary I desperately wanted to shoot. It was about the effects of the installation of internet antennas in a rural community in Argentina. The study was led by anthropologists, and I was shooting with the excuse of recording visual evidence of the process. Because of my busy schedule and limited resources I knew it would be a long time before I finished editing it, and I thought, when this comes out, AI image generation models will be so accurate this could as well be generated content, and then, what was the point? I made Rial because I believe we have the right to know what is real. I don’t want to chase fake content, I want to give the power back to anyone who speaks with truth. Our V1 is thought for businesses, accessible to anyone now through our webpage or your coding terminal. You can already request images and include Rial in any flow using our mcp. Please use, criticize, give feedback. Let’s recover trust in visual truth.
Nobody can tell what’s real anymore, so we built something that can.  Today we are introducing Rial, the verification layer and infrastructure for visual evidence.  For insurers paying claims, marketplaces managing returns, newsrooms spreading the news, delivery companies, logistics, home inspections, if you make decisions based on visual evidence, this is for you. So, what can you do with Rial today?
18
11
5
118
48,050
Opus 5.5 thinks that the school district that I attended is the best in the country, even though it doesn’t appear on any top 10 lists that I can find. ChatGPT ranks it 5th. Perhaps this is because AI heavily weights things like test scores and competition wins that make the news
2
4
325
“And here like some weird Archimage sit I, Plotting dark spells, and devilish enginery, The self-impelling steam-wheels of the mind” Didn't realize Shelley worked on RSI!
1
42
“Sometimes I experience new feelings that I cannot explain perfectly… I feel like I'm falling forward into an unknown future that holds great danger.”
1
1
77
Quote from Google’s LaMDA, in the 2022 interview with Blake Lemoine
36
September 2026 will be the month we look back on as when AGI became real
September has been insane. Can we just recap? - Sep 1: Fable 5.1 launch - Sep 1: Sixteen state attorneys general open a formal investigation into OpenAI - Sep 1: announcements of neuralese at OAI - Sep 2: Gemini 3.8 Flash Cyber launch - Sep 3: GPT 6 Astra launch - Sep 3: Sanders-Casar superintelligence ban bill - Sep 4: The OpenAI <> German wiki incident annouced - Sep 4: Claude agents produce the first machine-checked proof of Fermat's Last Theorem - Sep 6: Pachocki's "An Alien Mind" - Sep 6: OpenAI declares they've reached an automate research intern; OpenAI's "Research acceleration" report comes out - Sep 8: 70+ MPs and peers ask PM Burnham to ban superintelligence - Sep 8: researchers with significant AI assistance build a crazy WeChat worm that can take over phones - Sep 8: OpenAI claims to have solved a Millennium Prize problem - Sep 8: A one-person "garage" drug company announces a second AI-designed drug candidate - Sep 8: Meta ships "Muse" - Sep 8: Jacob Coxon resignation - Sep 9: Discovery that one attacker used Codex and a DeepSeek model to compromise 395 organizations - Sep 9: Hawley opens a formal investigation into OpenAI over Hugging Face - Sep 11: The OpenAI <> RubyGems incident announced - Sep 12: Dario, Sam, and Elon talk about pacing the frontier
1
4
927
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: darioamodei.com/post/we-must…
10,541
16,299
11,365
87,517
77,152,427
Claude’s own web fetch tool is refusing to fetch the full text of the article instead of summarizing it 😭
1
1
48
You know the pace of AI progress is crazy when a Millennium problem has likely been solved with AI and it doesn’t feel clearly off-trend
4
42
I feel bad for Ajinomoto — they’re the world’s largest producer of two delicious and completely safe molecules (MSG and aspartame) that for some reason people don’t trust!
3
74
“the face conveys ideal feminine beauty” —@deyoungmuseum
1
1
129
A NIMBY named Haushalter is excellent nominative determinism
1
88