How Gulf-developed large language models like Jais are bringing Arabic into the AI mainstream

Special How Gulf-developed large language models like Jais are bringing Arabic into the AI mainstream
As Gulf states aim to become AI leaders by investing in R&D and startups (Supplied/MBZUAI)
Short Url
Updated 09 October 2023
Follow

How Gulf-developed large language models like Jais are bringing Arabic into the AI mainstream

How Gulf-developed large language models like Jais are bringing Arabic into the AI mainstream
  • ChatGPT understands inquiries in Arabic, but answers can sound unnatural or fail to convey the right message
  • Now homegrown LLMs can capture linguistic nuances and even comprehend dialects and cultural references

DUBAI: When ChatGPT made its debut last year, the artificial intelligence program caused a global sensation, as users found themselves communicating with a machine that could pass as another human being.

However, the enthusiasm among techies in the Arab world was somewhat diminished by ChatGPT’s limited grasp of Arabic, in part the result of the language’s complexity, diacritical markings, inflection system and regional dialects.

Although ChatGPT, which is based on a large language model, or LLM, can understand inquiries in Arabic and is able to translate, especially when using Modern Standard Arabic, answers can come across as unnatural, while literal translations do not always convey the right message.

That is why Jais, an LLM designed to support Arabic, was unveiled in July, bringing one of the world’s most widely spoken, though occasionally overlooked, languages into the AI mainstream.

Jais, a name that recalls the UAE’s highest peak in Ras Al-Khaimah, is the brainchild of a team of academics and engineers who embarked on the project because they felt too few LLMs were credibly multilingual.




The Ameca humanoid robot greets visitors at Dubai's Museum of the Future. (AFP)

Downloadable on the machine learning platform Hugging Face, Jais is the result of a collaboration between Cerebras Systems, Mohamed bin Zayed University of Artificial Intelligence, or MBZUAI, and a subsidiary of the Abu Dhabi-based G42 called Inception.

“It is vital that large language models are developed for languages other than English to ensure that innovation is accessible to everyone,” Andy Jackson, CEO of Inception, told Arab News.

“A quality Arabic LLM is critical for all sectors, businesses and organizations, as well as individuals. Innovation thrives when we collaborate, and Jais sets a new standard for AI advancement in the Middle East, ensuring that the Arabic language, with its depth and heritage, finds its voice within the AI landscape.

“Jais demonstrates our commitment to excellence, and our dedication to democratizing AI and promoting innovation.”

LLMs are functional machine learning models that use deep learning algorithms to process and understand natural human language. These models are then trained on large amounts of text data to learn patterns in the language.

These programs, which are rapidly proliferating in the wake of ChatGPT’s success, are capable of generating text on a seemingly endless array of subjects, producing everything from academic papers to poetry.

What is especially impressive about them is their ability to create responses to questions that are so convincingly human-like in almost any language, including coding.

But in order to make those languages sound convincing, native-speaking human programmers are often required to provide a critical layer of context and understanding that can enhance accuracy and reliability.

“Jais is purpose-built for the Arabic language and excels in capturing its intricacies and nuances, ensuring highly accurate and contextually relevant responses — a distinct advantage over general-purpose models,” said Jackson.




AI programs that are responsive to the Arabic language could widen access to a transformational new technology. (MBZUAI)

“This specialization is a pivotal development, opening up opportunities for governments, industries, and individuals across the Arab world to tap into the potential of generative AI.”

Currently considered among the foremost Arabic LLMs, Jais, a 13-billion parameter model, was trained on a newly developed 395-billion-token Arabic and English dataset on Condor Galaxy, one of the largest cloud AI supercomputers in the world, launched by G42 and Cerebras in July using 116 billion Arabic tokens and 279 billion English tokens.

“Jais was born in Abu Dhabi and offers more than 400 million Arabic speakers the opportunity to harness the potential of generative AI,” Preslav Nakov, professor and deputy department chair of Natural Language Processing at MBZUAI, told Arab News.

“It will facilitate and expedite innovation, highlighting Abu Dhabi’s leading position as a hub for AI, innovation, culture preservation and international collaboration.”

As an open-source model, Jais is expected to engage scientists, academics and developers to accelerate the growth of a an Arabic language AI ecosystem. It could also serve as a model for other languages now underrepresented in mainstream AI.

FASTFACTS

• Large language models, or LLMs, are a type of AI that can mimic human intelligence.

• Arabic is spoken by 400m people, but accounts for 1 percent of total global online content.

• Jais was created by Cerebras, MBZUAI, and a subsidiary of G42 called Inception.

“Jais outperforms existing Arabic models by a sizable margin,” said Nakov. “It is also competitive with English models of similar size despite being trained on significantly less English data.

“This exciting result shows that the model’s English component learned from the Arabic data and vice versa, opening a new era in LLM development and training.”

In Jais’s development, significant attention was devoted to pre-processing Arabic text, enhancing support for the language’s unique features, including its writing style and word order.

Jais also maintains a balanced Arabic-English dataset focus for optimal performance, offering a marked improvement over models with a limited Arabic text presence.

Its developers say Jais, unlike other models, captures linguistic nuances and even comprehends various Arabic dialects and cultural references.

“Jais facilitates faster customization for specific Arabic-focused use cases and addresses data ownership concerns by being based in the UAE, offering a reassuring solution for local enterprises,” said Inception CEO Jackson.




LLMs are functional machine learning models that use deep learning algorithms to process and understand natural human language. (Supplied)

The UAE’s Ministry of Foreign Affairs and Ministry of Industry and Advanced Technology, Abu Dhabi’s National Oil Company and Department of Health, Etihad Airways, First Abu Dhabi Bank, and global technology group e& are planning to utilize Jais, offering valuable insights to enhance the model and its applications across their industries.

Given the strong digital transformation efforts by several of the Arab Gulf governments, accompanied by huge investments in high-tech industries and homegrown tech startups, AI programs that are responsive to the Arabic language could widen access to a transformational new technology and challenge the monopoly of a clutch of Silicon Valley companies.

Last month, Technology Innovation Institute, an Emirati research center in Abu Dhabi, released Falcon 180b, an open-source AI model. Established in 2020, TII released Falcon 40b, the first version of its flagship open-source AI model, in May this year, after unveiling Noor, an Arabic-based AI model, last year.

According to a report in The Economist magazine, TII is the applied-research arm of the Advanced Technology Research Council, a government agency that employs an 800-strong multinational staff working on subjects from biotechnology and robotics to quantum computing.

“We are entering the game to disrupt the core players,” Faisal Al-Bannai, secretary-general of the ATRC, told The Economist, adding that TII will build new proprietary models and applications catering for specific fields such as medicine and law.

For its part, Saudi Arabia launched its National Strategy for Data and Artificial Intelligence in October 2020, aiming to become a global leader in the field as it seeks to attract $20 billion in foreign and local investments by 2030.

The Kingdom is also determined to future-proof its workforce, initially by training and developing a pool of 20,000 AI and data specialists. In May this year, Deloitte’s AI Institute was officially launched at the Experience Analytics conference in Riyadh.

Just last week Saudi Arabia launched a National Olympiad for Programming and Artificial Intelligence open to all middle- and high-school pupils. An estimated 300,000 students will be selected from 3 million participants for training in programming and AI, according to media reports.




The hope is that the advent of AI and the automation of rapid translation will be a game changer for Arabic content. (LEAP)

The initiative is a collaboration between the Saudi Data and Artificial Intelligence Authority, in collaboration with the Ministry of Education and King Abdulaziz and His Companions Foundation for Giftedness and Creativity (Mawhiba).

Saudi Arabia’s adoption of digitalization and emerging technologies is forecast to contribute about 2.4 percent to its gross domestic product by 2030, according to a recent report by global consultancy firm PwC.

In terms of average annual growth in the contribution of AI by region, Saudi Arabia is expected to grab a 31.3 percent share in the technology’s expansion between 2018 and 2030, the PwC report added.

“AI is developing rapidly, and its impact will be felt more and more across all sectors and areas of life,” said MBZUAI’s Nakov. “In this context, it is vital that the Arab world has access to an advanced LLM that can be adapted and utilized across all sectors.

“The rapid advancement of AI means that organizations that fail to adapt and start using AI sooner rather than later will be left behind, which makes it even more essential for the Arab world to have access to quality LLMs.”

Beyond its business applications, however, a crucial aspect of a program such as Jais is its ability to champion neglected languages, preserve them in a fast-changing economy, and promote digital inclusivity.

Although Arabic is an official language in 22 countries and is partly spoken in 11 others, it accounts for just 1 percent of total global online content, according to Jais’s creators. The hope is that the advent of AI and the automation of rapid translation will be a game changer.

By placing the language at the forefront of the AI revolution, Jais and its successors could help to maintain Arabic’s global prominence and its distinctive cultural significance in the digital age.


Israel PM says ‘intense’ fighting with Hamas in Rafah ‘about to end’

Israel PM says ‘intense’ fighting with Hamas in Rafah ‘about to end’
Updated 18 sec ago
Follow

Israel PM says ‘intense’ fighting with Hamas in Rafah ‘about to end’

Israel PM says ‘intense’ fighting with Hamas in Rafah ‘about to end’
  • “The intense phase of the fighting against Hamas is about to end,” Netanyahu said in an interview with Israel’s pro-Netanyahu Channel 14

JERUSALEM: Israeli Prime Minister Benjamin Netanyahu said on Sunday that the Israeli military’s heavy fighting against Hamas militants in the southern Gaza city of Rafah is nearly over.
Netanyahu, in his first interview with an Israeli network since the war with Hamas broke out on October 7, said troops would soon be deployed to the northern border with Lebanon but for “defensive purposes.”
“The intense phase of the fighting against Hamas is about to end,” Netanyahu said in an interview with Israel’s pro-Netanyahu Channel 14.
“It doesn’t mean that the war is about to end, but the war in its intense phase is about to end in Rafah,” he said.
“After the end of the intense phase, we will be able to redeploy some forces to the north, and we will do that. Primarily for defensive purposes but also to bring the (displaced) residents back home,” Netanyahu said.
Tens of thousands of Israelis have been displaced from northern Israel which has seen near-daily exchanges of fire between Israeli forces and Lebanese Hezbollah militants since the war in Gaza began.
Netanyahu said he would not agree to any deal that stipulates an end to the war in Gaza, indicating that he was open to a “partial” deal that would facilitate the return of some hostages still held there, if not all.
“The goal is to return the kidnapped and uproot the Hamas regime in Gaza,” he said.
United States officials have raised doubts over Israel’s goal of completely destroying Hamas, and on Wednesday Israel’s top army spokesman, Rear Admiral Daniel Hagari, said Hamas cannot be eliminated.
“To say that we are going to make Hamas disappear is to throw sand in people’s eyes,” Hagari said.
He said Hamas is an ideology and “we cannot eliminate an ideology.”
When asked about the post-war situation in Gaza, Netanyahu said Israel will have a role to play in the near term.
“It’s clear military control in the foreseeable future will be ours,” he said, before giving his most-detailed comments yet on the post-war situation.
Earlier this month two war cabinet members Benny Gantz and Gadi Eisenkot left the government after Netanyahu failed to deliver a post-war plan for Gaza as demanded by Gantz.
The United States has also pointed to the need for a post-war plan that would help ensure Israel’s long-term security.
“We also want to create a civilian administration, if possible with local Palestinians, and maybe with external backing from countries in the region, to manage humanitarian supply and later on civilian affairs in the Strip,” the prime minister said.
“At the end of it, there’s two things that need to happen: we need ongoing demilitarization by the IDF (army) and the establishment of a civilian administration.”
The Gaza Strip has been gripped by more than eight months of war since Hamas’s unprecedented attack on Israel, which resulted in the deaths of 1,194 people, mostly civilians, according to an AFP tally based on Israeli official figures.
Militants took 251 hostages, 116 of whom remain in Gaza, including 41 the army says are dead.
Israel’s military offensive on Gaza has since killed at least 37,598 people, also mostly civilians, according to the health ministry in the Hamas-run territory.
Tens of thousands of Israelis have consistently rallied against Netanyahu and his government, demanding early elections and a deal to return hostages.
But Netanyahu said that if his government falls, “a left-wing government will be established here, which will do something immediately: establish a Palestinian state that is a Palestinian terrorist state that will endanger our existence.”


Jordanian, German officials discuss ties, Palestinian cause

Jordanian, German officials discuss ties, Palestinian cause
Updated 37 min 35 sec ago
Follow

Jordanian, German officials discuss ties, Palestinian cause

Jordanian, German officials discuss ties, Palestinian cause
  • Safadi expressed his gratitude for Germany’s support in managing the Syrian refugee crisis

AMMAN: Jordan’s lower house speaker met the vice president of the German parliament on Sunday to discuss ties, Jordan News Agency reported.

In his meeting with Aydan Ozoguz, Speaker Ahmed Safadi highlighted the “strong and multifaceted” ties between Jordan and Germany, and called for increased cooperation.

He expressed his gratitude for Germany’s support in managing the Syrian refugee crisis, highlighting the significant burden Jordan bears due to regional instability and the influx of refugees.

During the meeting, Safadi warned that the failure to achieve the Palestinian right to an independent state remains a major source of regional instability.

Ozoguz agreed on the importance of German-Jordanian relations and said that her visit aimed to build mutual trust. She highlighted Germany’s commitment to collaborating with Jordan to develop viable solutions to regional challenges, with the ultimate goal of achieving a two-state solution in the Israel-Palestine conflict.

Khaldoun Haynam, chairman of the parliamentary foreign affairs committee, also addressed the meeting, highlighting the importance of adequate international support for Jordan.

He praised Jordan’s efforts in assisting refugees and urged the international community to support the UN Relief and Works Agency for Palestine Refugees so that it can continue to help Palestinian refugees.
 


Without food or toys, a Gaza family tries to survive

Without food or toys, a Gaza family tries to survive
Updated 43 min 23 sec ago
Follow

Without food or toys, a Gaza family tries to survive

Without food or toys, a Gaza family tries to survive
  • Desperate mothers fight for their survival with no end to the conflict in sight

JABALIA: Surrounded by a sea of rubble, the Palestinian Al-Balawi family in northern Gaza hang blankets above the ruins of their home to create a makeshift tent that provides shade from the searing summer heat.

The family are struggling to feed themselves in the Jabalia refugee camp after the nearly nine months of war that have followed Hamas’s Oct. 7 attack.

Gazans are suffering from severe shortages of supplies, including food and water, alongside Israel’s bombardment of the territory, forcing desperate mothers like Umm Siraj Al-Balawi to fight for their survival with no end to the conflict in sight.

“There are no vegetables or fruits. No vitamin intake. When you get sick, you stay in bed for two or three weeks to recover,” said the 33-year-old.

“This war must stop because it is a war of displacement. It is a war of annihilation.”

Jabalia has been hit particularly hard in recent weeks, with Israeli forces carrying out a massive bombardment campaign, part of a fierce ground offensive in northern Gaza — an area the military had previously said was out of the control of Hamas militants.

Israeli forces retrieved the bodies of some hostages from Jabalia and, in May, reported “perhaps the fiercest” fighting there since the start of the war.

Desperation among Gaza’s 2.4 million population has increased as the fighting has raged, with warnings from humanitarian agencies that they are unable to deliver aid.

Vital food supplies have piled up and are undistributed on the Palestinian side of the Kerem Shalom crossing, a key conduit for aid to enter Gaza.

Israel says it has let supplies in and called on agencies to step up distribution, while aid organizations, including the UN, say they have been unable to pick up supplies because of a breakdown in civil order in Gaza.

The war broke out after the Oct. 7 attack.

The misery of Gazans has only been exacerbated by Israel’s bombing raids, which it says are to destroy the infrastructure of Hamas.

On Saturday, at least 24 people died after huge strikes in two Gaza City neighborhoods.

The strikes left several residential complexes in rubble, while the Israeli military said it had targeted two Hamas military infrastructure sites.

“People are getting displaced from house to house, tent to tent, school to school,” said Umm.

“They (Israelis) instructed people to head to Rafah before instructing them to evacuate Rafah. They are doing the same in Khan Younis. Until when?”

The Al-Balawi family’s dire situation leaves them scrabbling in a wasteland of debris for items like pillows and food.

“The situation was very, very difficult (before the war). And it worsened after the war,” said Umm’s husband Abu, 34.

He pulls a pink cushion from the wreckage of buildings, passing it to his wife, who beats it to clear it of dust.

Elsewhere, he uses a spade to claw back mounds of rubble before finding a red teddy bear for his young son.

He then leads his children through a destroyed street to get hold of much-sought-after water before heading back to their tent, where his children share some bread and beans from a bowl.

“There is a scarcity of food and water. We can barely find food for our children. Diseases have spread in all the areas where the displaced are gathered.”

The horrors of war are apparent for their nine-year-old boy Siraj, despite his age.

“We can’t find clothes. We have no clothes,” he complains.

“There are no toys,” he adds, holding up a damaged doll. “We have no house.”


Lebanese party on despite threat of war

Lebanese party on despite threat of war
Updated 56 min 32 sec ago
Follow

Lebanese party on despite threat of war

Lebanese party on despite threat of war
  • Tensions have risen on the Israel-Lebanon frontier for almost two weeks
  • “This is Lebanon and this is our story. Nothing changes. We survived the July war”

BEIRUT: In the buzz of a trendy Beirut neighborhood, the din of bars and laughter blend together, far from the border violence with Israel further south and fears of all-out war.
“I’m 40 years old, and each year they tell us that war will break out this summer,” Elie, a financial consultant who did not give his last name, said in a bar in the Lebanese capital with other locals chatting beside him.
“What we see in the street is different from what we hear in the media,” he said. “What the foreign press is reporting makes people think that Lebanon is at war.”
Since the beginning of the war between Israel and Hamas in the Gaza Strip on October 7, the Palestinian Islamist movement’s ally, Lebanon’s Hezbollah, has been exchanging nearly-daily fire with Israel over the border.
Tensions have risen on the Israel-Lebanon frontier for almost two weeks, after Israel’s killing of one of Hezbollah’s most important commanders.
UN Secretary-General Antonio Guterres has warned of a catastrophe “beyond imagination,” and France and the United States have been working for de-escalation.
Israel’s Foreign Minister Israel Katz said Hezbollah would be destroyed in “total war” and the country’s army approved “operational plans for an offensive in Lebanon.”
The following day, the head of Hezbollah, Hassan Nasrallah, warned that “no place” in Israel would be spared by the group’s weapons in the event of full-blown war.
While the risk of the border conflict overflowing into the rest of the country comes up in conversations, it does not seem to bother the partiers in the Christian neighborhood of Mar Mikhael.
In the Beirut neighborhood well-known for its bars, lit-up with multi-colored lights, glasses clink and customers dance to the rhythm of remixed Arab and Western pop songs played at full blast by a DJ.
“This is Lebanon and this is our story. Nothing changes. We survived the July war,” Elie said, referring to a war between Israel and Hezbollah in 2006.
“In all the past crises, people continued to stay out late, whether during the Covid-19 pandemic or the explosion at Beirut port” in the summer of 2020.
The blast killed more than 200 people, destroyed swathes of the capital including Mar Mikhael, and accelerated the economy’s collapse.
Elsewhere in the city, locals brought their children to a street festival organized in the center of Beirut, carrying on late into the night.
“Despite all the threats, we are a people who love life,” Abir Atallah told AFP, amid the laughter of children in front of the stage.
While according to the United Nations more than 95,000 Lebanese have been displaced by the conflict around the Israel border, the spectre of war does not stop people in other parts of the country from living normally.
“We live day by day. Of course, people are afraid, but we rely on God,” said Mira Makhlouf, who sells toys for children.
“Lebanese love to party,” she said, adding that she has no intention of leaving the country if a full-scale war breaks out.
While the biggest events held every summer in Lebanon were canceled this year because of the conflict in the south, some organizations chose to continue with thir plans.
Arab singers are flocking to Lebanon to perform. More than 20,000 people in mid-June attended a concert in Beirut by Egyptian pop star Amr Diab.
Foreigners continue to travel to the country for the summer festival season, despite warnings from several countries that their citizens should not visit Lebanon.
“I do not think that a war will break out, and we are not afraid. Otherwise, we would not see this crowd,” Nayla Haddad said at the festival.
“Every two weeks, we organize a festival in a new place,” she said, smiling.


$230m US humanitarian pier in Gaza operational for only 12 days

$230m US humanitarian pier in Gaza operational for only 12 days
Updated 23 June 2024
Follow

$230m US humanitarian pier in Gaza operational for only 12 days

$230m US humanitarian pier in Gaza operational for only 12 days
  • Pier has allowed for the delivery of roughly 250 truckloads of aid, less than half of the pre-war daily deliveries to Gaza

LONDON: The $230 million floating pier built by the US military for seaborne humanitarian deliveries to Gaza has been operational for only 12 days since its inauguration on May 17, The Guardian reported on Sunday.

On March 7, US President Joe Biden announced that the temporary pier “would enable a massive increase in the amount of humanitarian assistance getting into Gaza every day.”

The construction of the two necessary structures — a floating dock anchored offshore and a pier connected to the Gazan coast — took more than two months and involved about 1,000 soldiers, sailors and several ships, including the Royal Navy’s landing ship, Cardigan Bay, which served as accommodation.

Since its launch, the pier has allowed for the delivery of approximately 250 truckloads of aid, equating to 4,100 tonnes of supplies, which is less than half of the pre-war daily deliveries to Gaza. The aid arriving by sea has often remained on the beach due to a lack of trucks for distribution, a result of security concerns.

Rough seas in the eastern Mediterranean have posed unexpected challenges, rendering the joint logistics over-the-shore system less effective than anticipated. The structure was designed to operate in sea conditions up to “sea state 3,” with waves between 0.5 and 1.25 metres. However, it sustained damage during a storm on May 25 and has faced unseasonably choppy waters since then.

After repairs in Ashdod, Israel, the pier resumed operations on June 8 but faced further interruptions. It was dismantled again on June 14 as a precaution against impending storms. Despite being reinstalled, there are reports suggesting that the pier’s vulnerability to weather might lead to it being dismantled early, possibly as soon as next month.

“They just miscalculated,” Stephen Morrison, a senior vice-president at the Center for Strategic and International Studies, told The Guardian. “They didn’t fully understand what was going to happen with the weather … so the DoD [Department of Defence] walks away, humiliated in a fashion.”

While acknowledging the difficulties, the Pentagon has not confirmed plans for an early termination of the mission.

“We have not established an end date for this mission as of now, contrary to some press reporting on the matter,” chief spokesperson Maj Gen Patrick Ryder told The Guardian on Thursday.

The floating pier was intended to provide an alternative means of delivering aid to Gaza, bypassing Israeli land restrictions. However, aid workers expressed concerns that the significant resources invested in the effort detracted from political pressure on Israel to open land crossings, which remain the most effective way to deliver aid.

Ziad Issa, head of policy and research at Action Aid, noted a decline in aid deliveries to Gaza, with an average of fewer than 100 trucks arriving daily in early June.

The severe security conditions have hindered the distribution of aid in Gaza. The Rafah crossing from Egypt has been closed since May 7, following an Israeli military offensive, and the alternative Keren Shalom crossing in southern Israel has proved dangerous due to the volatile situation.

“It’s unsafe for aid workers and trucks to move because of the ongoing bombardments on Gaza,” Issa told The Guardian. The Israelis announced a “tactical pause” last week to allow an aid corridor through southern Gaza, but Issa said: “We haven’t seen any difference since these tactical pauses have come in place.”