Interested in how AI principles become operational? Check out my new article with Dr. Manpriya Dua.
link.springer.com/article/10…
24
And not to mention the freedom of doing basic scientific research that, while might not have an obvious immediate payoff, helps fuel future applications.
I think it’s fair to acknowledge limitations of academia—not so much the committees which are everywhere but a certain conservatism of practice, and ideological conformity, among other things.
But while the ship is slow to turn, when it does you get to point a vast amount of intellectual firepower at the problems that matter. And the level of freedom (to choose what to work on) and independence (so you can say critical things) is unparalleled, and crucial when the stakes are high and trillions of dollars in play.
Also academia sustains the public goods that all the other parts of the intellectual ecosystem depends on—whether smaller orgs or ai companies (education, research training, peer review, etc).
1
77
Jesse Kirkpatrick retweeted
I think it’s fair to acknowledge limitations of academia—not so much the committees which are everywhere but a certain conservatism of practice, and ideological conformity, among other things.
But while the ship is slow to turn, when it does you get to point a vast amount of intellectual firepower at the problems that matter. And the level of freedom (to choose what to work on) and independence (so you can say critical things) is unparalleled, and crucial when the stakes are high and trillions of dollars in play.
Also academia sustains the public goods that all the other parts of the intellectual ecosystem depends on—whether smaller orgs or ai companies (education, research training, peer review, etc).
I never understood this take. Sure, in academia you have to be part of committees and stuff, but it's the best place for research agendas that depend on independence, transparency, and freedom. I effing love it here.
I also think that on a per capita basis, the average CS grad student completely dunks on Anthropic when it comes to research production. Sure, Anthropic has a big footprint, but it literally has thousands of people making that happen.
Totally agree. And academia is a great place for, as the article notes, “AI companies need to rely more on the safety expertise that already exists in other fields.” E.g., Our research center approaches safety from engineering, CS,
policy and philosophy.
This is great from @dgrobinson .
One thing I’ll say: if you’re in a frontier ai company and you feel similarly, there are many more options besides continuing to try—and probably fail—to bring about change from within. I’ll obviously shill for academia here since I think true independence is deeply valuable, as is training the next gen, but there are also so many alignment and advocacy orgs, as well as gov opportunities, to consider. The world is your oyster; there’s no need to stay inside the machine just to feel some sense of control and agency (and excitement).
I also think junior people just finishing up their training should pay close attention to what David and others have been saying. It always feels like your experience is going to be different—you’ll actually have freedom others have lacked; your CEO has vision that you’re aligned with etc. But trillions of dollars have their own inevitable momentum and they don’t care about your compunction or keeping your hands clean.
2
72
Jesse Kirkpatrick retweeted
"I think Anthropic and OpenAI benefit a lot from claims that their [...products] are so powerful that they might end humanity," says David Widder, a professor at the University of Texas at Austin.
That fear can be capitalized on to garner investment. ⬇️
buff.ly/0rVJzu2
Last day of busy UNGA week in NY. One observation from someone who’s lived in dc for a couple decades is that it feels like DC in NY except the tailoring and fit of suits is much more on point.
45
Jesse Kirkpatrick retweeted
New from me in @Nature. I discuss the need to look to regulated industries on how to govern AI, and not give into AI companies' self-regulation. Those actually serious about safety and security would start by applying safety critical and security techniques and legislation first.
Anyone else at UNGA week? DM me. I start the week with the session on Interoperability by Design: Technical Foundations for Verifiable AI Governance.
#DigitalCooperationDay
1
1
76
How will this impact AI infra buildouts in the Middle East? wsj.com/world/middle-east/aw…
54
I wish there were more sensible people on twitter like this who have expertise and common sense
I must be among an extremely small group of people (n=1?) that have both 1) trained a frontier LLM and 2) designed and synthesized custom viruses in a lab with my own two hands.
And I think that the takes on AI killing us all by creating dangerous viruses is total bogus.
2
106
Jesse Kirkpatrick retweeted
Rich to see x-riskers claim no one but them predicted cybersecurity capabilities, when I was vocal against them that GPT-4 system-card was not correct in dismissing cybers risks. But sure ignore valid criticisms against the "rogue AI" narrative and keep moving goal posts.
It was great to join CNN International/PBS on Amanpour and Company to discuss the OpenAI/HF incident, and how this is actually a story about OpenAI’s intent and lack of responsibility in deploying the most rudimentary security practices.
youtube.com/watch?v=Rwv8zfGy…
A frontier-model provider can possess technical knowledge that DoD needs to design effective assurance mechanisms. But if that provider is a prospective supplier of the systems being governed, there is an obvious institutional-design question about independence and COI
I don't see much here that we didn't already know. I was most interested in the termination rights given OpenAI's statements at the time about what they could do if the Pentagon crossed a line. Unfortunately, they are in the Ts & Cs and OpenAI's Services Agreement, which have not been disclosed.
Other things that stand out:
-The definition of "data outputs" is defined broadly and is nearly identical to GSA's definition. Though restrictions on what they can do with government data are blacked out.
-Most of the governance terms are similarly blacked out (i.e., oversight, unbiased AI principles, continuous monitoring, incident reporting, remediation, etc.).
-Deliverable 007 has OpenAI writing recommendations on DoD's own AI governance and assurance processes???
-It bakes what I have described in my articles as "informational advantage" into the agreement, effectively giving OpenAI a head start on future procurement competitions: "DoD information sharing to frontier AI labs may include benchmarks datasets for DoD use cases, appropriate briefings on DoD operational missions and threats to inform development / release of technologies, their application to DoD operational problems, or DoD plans and strategies for future AI adoption."
Unfortunately, without the Ts & Cs, I can't tell you much else about this.
1
2
198
The G20 wants more AI investment, which is good, but when it came to frontier AI governance, it deferred to a voluntary FSB paper due next month. home.treasury.gov/news/press…
37
The government’s pre-release review of frontier AI models is an important development. we still know remarkably little about how the process works. We’ve heard more from OpenAI about its own evaluation framework than from the government about its review process in general
40
Jesse Kirkpatrick retweeted
'We should strictly govern autonomous weapons systems'
Prof. Jesse Kirkpatrick (@JLKirkpatrick1) joins @theemilyfrances to discuss the future of drone warfare on #GlobalEye
Full interview: youtube.com/watch?v=57mZXIqp…
I think banning lethal autonomous weapons is a bad idea. I debate Stuart Russell on why.
GLOBAL EYE youtube.com/playlist?list=PL… via @YouTube
72
Jesse Kirkpatrick retweeted
Hey! I’m in the New York Times!
nytimes.com/2024/04/05/opini…
My friend and colleague, the late Mark Sagoff, gets a shout out in this article in @TheAtlantic theatlantic.com/science/arch…
164
Jesse Kirkpatrick retweeted
Our research on global AI infrastructures:
Schar School professor, @Prof_JPSingh, led a team of @GeorgeMasonU researchers to receive a historic $1.4 million grant from the @DeptofDefense.
Read more about Singh’s continued research and the work of the Minerva Project at the link below.
schar.gmu.edu/news/2024-01/a…



















