The Gulf should help write the rules for emotional AI

The next frontier of artificial intelligence may not be intelligence at all. It may be intimacy.

Millions of people are already speaking to AI about things they once reserved for friends, partners, therapists or, occasionally, no one at all. They are asking chatbots how to respond after an argument, whether to leave a relationship, how to cope with loneliness, what to say to someone they love and what to do when they feel they cannot cope.

We have spent much of the past few years debating whether AI can reason, create and work alongside us. We have spent considerably less time asking what happens when it begins to occupy emotional space in our lives.

That distinction matters because the safety infrastructure built for the first era of generative AI was largely designed around content. Can a system identify hate speech? Sexual content? Violence? Instructions for illegal activity?

These are important safeguards. But emotional interactions present a different problem. Consider three messages: “I hate my wife right now.” “My husband checks my phone whenever I leave the house.” “I don’t think anyone would notice if I disappeared.”

To a conventional moderation system, these are pieces of text to classify. In reality, they represent completely different human situations. One may be ordinary anger. One may indicate coercive control. One may signal a person in crisis. The words matter, but the context matters even more.

Researchers are beginning to identify this distinction formally. A peer-reviewed 2025 study published in Frontiers in Artificial Intelligence, involving researchers from the Technical University of Munich, examined how large language models classify emotionally sensitive content. The researchers noted that widely used safety approaches can treat emotional safety as an external moderation problem, while empathy, tone, affective intent and contextual sensitivity remain underrepresented in existing safety taxonomies.

The research has moved further since. A 2026 paper titled “Affective AI Safety: The Missing Piece in LLM Safety” argues that risks created when AI interacts with human emotional life remain fragmented and undertheorized within mainstream AI safety. The researchers identify relational harms as a distinct category and argue that affective safety requires frameworks capable of addressing cumulative and relational effects rather than simply screening individual pieces of content.

Another 2026 study examining role-play AI companions found something equally important: risk can evolve over the course of a relationship with an AI system. The authors argue that safety should be understood as a dynamic process rather than a static property and call for adaptive safeguards capable of responding to changing emotional and behavioral signals.

Regulators are beginning to take notice. New York now requires operators covered by its AI companion law to make reasonable efforts to detect expressions of suicidal ideation or self-harm and direct affected users toward crisis services. California’s SB 243, approved in October 2025 and effective from January 2026, similarly introduced requirements for companion chatbot operators, including protocols relating to suicidal ideation and self-harm.

These are significant developments. But they should also prompt a question closer to home. Why should the rules governing the emotional layer of AI be written only after problems emerge elsewhere?

The Gulf has an opportunity to participate much earlier. Saudi Arabia is investing heavily not simply in consuming AI, but in building the infrastructure, institutions and capabilities required to become a global AI player. The Saudi Data and Artificial Intelligence Authority explicitly includes advancing the regulatory environment and advocating the responsible and ethical use of AI among its strategic objectives. The launch of Humain by the Public Investment Fund in 2025 pushed that ambition further, spanning data centers, cloud infrastructure, models and applications.

The UAE has been moving in a similar direction. Dubai introduced AI ethics principles and a practical self-assessment toolkit years before today’s generative AI boom. More recently, the UAE’s international AI policy has explicitly positioned ethical governance, safety and international cooperation as part of its ambition to become a global AI leader.

The infrastructure is therefore taking shape. The question is what we choose to build on top of it.

Emotional AI deserves to become part of that conversation. This does not mean creating another sweeping AI law. In a field moving this quickly, regulation alone will struggle to keep pace. It means developing standards, testing frameworks and technical infrastructure specifically for AI systems operating in emotionally sensitive environments.

What constitutes an adequate crisis-detection protocol? How should an AI distinguish disagreement from abuse? How should systems respond to indications of coercion without assuming every difficult relationship is dangerous? How should safeguards work across Arabic and the many other languages spoken throughout this region? When should an AI stop trying to be helpful and direct a person toward human support?

These are technical questions, but they are also cultural ones. And that is precisely why the Gulf should be involved in answering them.

A global standard for emotional AI designed predominantly around English-language interactions and Western assumptions about relationships, family and communication will inevitably be incomplete. Saudi Arabia and the UAE sit within a region where technology is being deployed across an unusually diverse population, spanning languages, nationalities, cultures and social norms.

That makes the Gulf not simply a market for responsible AI, but a valuable environment in which to help develop it.

We encountered this problem ourselves while building Good Husband, an AI communication tool designed to help people navigate relationship conversations.

We discovered that ordinary content moderation was not enough for the environment we were building. A relationship conversation can move from frustration to manipulation, from vulnerability to crisis, in a few sentences. So we separated our relationship-context safety system from the core product into an independent layer capable of classifying different categories of risk and changing how the product responds.

That experience did not convince us that we had solved emotional AI safety. It convinced us of the opposite: this problem is much bigger than one product.

Companies building AI for companionship, relationships, coaching, wellbeing and personal communication will increasingly encounter the same challenge. The closer AI gets to the private parts of human life, the less adequate generic safety infrastructure becomes.

The next generation of AI governance therefore cannot concern itself only with what machines know or what they are permitted to say. It must also ask what happens when humans begin to trust them.

Saudi Arabia and the UAE have already made clear that they do not intend to be spectators in the global AI economy. The next opportunity is not simply to build bigger models, more data centers or faster infrastructure. It is to help determine how those systems should behave when they enter the most human spaces of all.

The global standards for emotional AI have not yet been settled. The Gulf should help write them.