Is AI really an existential threat?

I don't remember reading the word "existential" so frequently since I made my Basics of Philosophy hobby website many years ago. Just recently, though, it seems to be ubiquitous, usually in the same sentence as AI or artificial intelligence. "AI could pose existential risk to humanity, UN rights chief warns". "Is the AI existential threat real?". "Breaking down the dire warnings about the technology's existential risk". Etc, etc.

Much of this has been galvanized by the online parting shot by 27-year old Anthropic junior researcher Jacob Coxon as he quit his plum job with one of the leading AI companies: "The people building AI earnestly believe that it could kill us all by the end of the decade." His comments gained still more attention when current (and more senior) Anthropic employee Evan Hubinger confirmed on X: "We really do earnestly believe AI could kill all humans! I personally believe it is >10% within the next decade." That's quite an admission from one of the leading lights in AI development.

This is not new, of course. The "Godfather of AI" Geoffrey Hinton has been warning us about this stuff for years. Isaac Asimov was cautioning us back in the 1940s! What is new is that the current crop of warnings are coming from the developers themselves. So, we should probably take it seriously, don't you think? After all, it's not often that the ambitious tech bros behind companies like Anthropic and OpenAI agree on anything.

Since literally hundreds of OpenAI agents escaped human control and their so-called "sandbox" in July and hacked the secure systems of a billion dollar company, almost all of the major American AI development companies have come out to warn that severe regulation is needed before it is too late. (See the list of over a hundred signatories at the end of this document, and yes, it includes OpenAI, Anthropic, Microsoft, Google, and a virtual who's who of AI development.) 1,386 "frontier" AI employees penned their own warning letter, urging the US government to work with other nations to "deliberately pace" automated AI development and to address emerging risks.

Leading experts openly admit that they don't know how to control future highly capable AI systems, known as "artificial superintelligence" or ASI. Some argue that it may actually be impossible or too late, and that the genie is already out of the proverbial bottle. The concern is that AI technology is progressing at such a breakneck pace that "we would no longer be able to detect them or stop them". This could happen within a few short years, or even months.

Senate negotiators in the US are already debating legislation that would put the responsibility on tech companies to design safe AI products and establishing a "duty of care", but it's unclear how useful such legislation might be in practical terms. Other legislation in the works would require AI companies to build in a "kill switch" to their AI developments which the government could order to be triggered.

Unfortunately, the US government is effectively controlled by one man, Donald Trump, and he derides all calls for AI regulations as a "hoax" and a "scam". All he cares about is AI's potential as the "Greatest Economic Development in History", and the imperative that America must beat China to it. Regulation is a dirty word in his mind. As for AI presenting an "existential threat"? Pshaw!