AI should be about humans harnessing machines

Probability can be a funny thing and very much in the eye of the beholder. So, when Evan Hubinger, a top safety researcher at Anthropic, warned that artificial intelligence is advancing at a rapid pace without adequate safeguards and that there is consequently a 10 percent chance it “could kill all humans” within the next decade, the response was somewhat mixed between dread and dismissiveness.

Usually, this level of risk is read as a relatively low probability. If a weather forecast suggested a 10 percent chance of rain, for example, you might decide to skip the burden of carrying an umbrella. However, a one-in-10 chance that humanity could be wiped would be a completely different scenario. At the very least, it would be worth taking a deep breath, stopping and assessing the risks.

Hubinger was not the only one to deliver this message of doom. His intervention was, in fact, a response to another post on X by Jacob Coxon, who, in a lengthy thread, outlined his reasons for resigning as a researcher from Anthropic after also working at OpenAI. He concluded that “they are racing straight to self-improving superintelligence and gambling with our lives.”

Some have equated this moment in world history with the one in which the implications of nuclear technology first became apparent to those developing it. In a 1939 letter to President Franklin D. Roosevelt, written by Leo Szilard and signed by Albert Einstein, the authors emphasized the potential of the new technology. It stated that “recent work in nuclear physics made it probable that uranium may be turned into a new and important source of energy,” but also warned that it could be used “for the construction of extremely powerful bombs.” They were right, beyond their wildest fears.

Technology itself is neutral. It is humans who will ultimately determine whether it becomes a force for good or for harm
Yossi Mekelberg

As is often the case with technological revolutions, for some, AI is the be-all and end-all remedy for all of humankind’s ills and challenges. For others, it represents the end of social order as we know it, if not the ultimate instrument of humanity’s destruction. Between the euphoria of finding a magic wand for all our challenges and the not entirely unsubstantiated fear of ultimate destruction lies a relatively simple truth: technology itself is socially, morally and ethically neutral. It is those who invent it, introduce it into society and, especially, those responsible for regulating it who will ultimately determine whether it becomes a force for good or for harm.

It is imperative that we, as societies, domestic as well as global, learn from the rapid introduction of the internet and social media, which have transformed our social lives on every level and not necessarily for the better. What we have experienced so far could look like a mere innocent blip compared with the power of AI. What we cannot afford is a binary debate in which one side wants to discard this technology altogether, while the other wants to accelerate its development with no brakes and deal with the consequences as we go. Both approaches are potentially damaging and counterproductive.

Before we even get into the most extreme scenarios of destruction, we should consider the more immediate consequences. Even relatively simple tasks that AI could perform, such as scanning documents, writing emails or developing fitness plans in place of instructors, could eliminate many thousands of entry-level jobs and leave people unemployed, bringing inevitable social consequences. We have already witnessed this on a smaller scale with digitalization and automation, which can leave people without jobs, income or self-esteem.

When people feel abandoned by economic and technological change, the consequences can extend far beyond unemployment and include social resentment and anger directed at the wrong people within society and the state itself.

There is, without doubt, much to look forward to. In healthcare and diagnostics, for example, AI could improve our ability to identify diseases and assist medical professionals. Machines taking on high-risk jobs could reduce human exposure to danger and, in some circumstances, reduce human error. Yet what is also terrifying about AI is that while it may prevent human mistakes, it could also bypass human judgment altogether.

What is terrifying about AI is that while it may prevent human mistakes, it could also bypass human judgment altogether
Yossi Mekelberg

The foundations of society are built on mutual trust, honesty and integrity, as well as social bonds and a sense of belonging. Yet AI could make it possible for misinformation and manipulation to be spread quickly, cheaply and anonymously, making it increasingly difficult to distinguish reality from fabricated information. It could make cybersecurity more difficult and create new security risks. It could also lead to a concentration of power in the hands of individuals and organizations that control the most capable AI systems, allowing them to gain substantial economic, political or informational influence.

There are also profound concerns about privacy, surveillance and intrusion. In the week after a co-pilot reportedly attempted to bring down an aircraft, one has to ask whether AI could eventually take over aspects of aviation, air traffic control or other critical public services. Education is already confronting similar questions, with students submitting work generated by AI and raising the question of whether the grade should belong to the student or the AI platform that produced the work.

But perhaps one area with immediate and far-reaching implications is how AI will change the battlefield. On the one hand, AI could make warfare more precise and potentially reduce harm to noncombatants. But in the wrong hands, or when operating autonomously without morality, consciousness or any genuine understanding of international humanitarian law, the prospect of machines deciding who lives and who dies would change not only warfare as we know it but humanity itself.

Eventually, autonomous weapons could potentially select and engage targets with little or no direct human intervention. This creates a fundamental problem of accountability: if AI makes a lethal mistake, who is responsible? The programmer, the commander, the manufacturer, the political leadership, the state or society as a whole?

Moreover, as we have already witnessed with technologies such as cruise missiles, there is a risk that increasing physical distance from the battlefield can lower the psychological and political barriers to using force. AI could introduce an unknown multiplier to that problem.

It could also accelerate military engagement when antagonistic actors in the international arena face a crisis, potentially creating situations in which machines respond to machines at the expense of human judgment, as we are already seeing in the realm of cyberwarfare.

This is not a call to stop AI research or its implementation. It is a call to develop and deploy it at a pace that allows society to discuss where it wants AI to be implemented, where it should not be implemented and at what pace. We need time to absorb what this technology really means for every walk of life before deciding on the next steps.

The question should not be whether humans or machines will win. It should be whether humans can harness machines without losing control of the society they were created to serve. Is AI a tool in the hands of humanity or a Frankenstein’s monster that ultimately decides humanity’s future?