Andrew Donnellan

when i first started getting into AI policy a couple yrs ago, i remember telling a friend — “you know, the ML researchers don’t act like the techies i usually work with. they’re mystics. their online presence is very … self-styled prophet.” and i think that was the beginning of my awareness that the cyber and ai safety camps had different intellectual philosophies and incentives. the cyber folks generally approach systems as things that can be understood, with the assumption that risks can be mitigated through standard engineering / “defense in depth” techniques. meanwhile, many ML and alignment folks seem to not only emphasize — but sometimes delight in — the unknowability of “grown not engineered” AI systems. they focus on emergent behaviors their creators don't fully understand. and these philosophies naturally flow from the respective systems these groups have experience with! conventional software *is* pretty intelligible, esp relative to neural networks. but obviously these sensibilities can enable distinct failure modes — overconfidence in current security measures, and fatalism about the usefulness of existing security measures. like many of the people quoted here, i think the best way for each of these groups to develop and grow their own understanding is to work together. the cyber folks could teach the alignment folks a lot about practical security measures, the alignment folks can identify new threat vectors and assess how safeguards are holding up with evolving capabilities. we need the insights of both fields, tribalism impoverishes us all.