Experts warn AI industry’s existential fears overshadow urgent issues

The latest catalyst: Jacob Coxon, a safety researcher at Anthropic, resigned on September 9, warning that leading AI labs are barreling toward self-improving superintelligence without adequate safeguards. The existential alarm bells keep getting louder Coxon’s resignation didn’t happen in a vacuum. Anthropic CEO Dario Amodei published a 3,800-word essay calling for a global slowdown of AI development. The piece cited a buffet of nightmare scenarios: loss of control over advanced systems, mass economic disruption, and the potential for bioterrorism enabled by increasingly capable models. Paul Christiano, a respected voice in AI alignment research, went further. He stated the industry is on course for near-term catastrophic loss of control, with some researchers estimating a greater than 10% probability of human extinction within a decade. These warnings build on a foundation laid more than a decade ago by figures like Stephen Hawking and Elon Musk, and they’ve been reinforced by surveys of AI researchers that consistently peg the median estimate for extremely negative outcomes at around 5%. The case for looking at what’s already broken The existential framing has drawn sharp critici...