Why AI is so difficult to apply in finance
The past decade has seen a revolution in the field of artificial intelligence stemming from advancements in machine learning, deep learning and computer architecture.
This has been demonstrated in many sectors, but when it comes to the financial sector there seems to be some reluctance in applying AI as those involved in trading prefer human elements that can distinguish many variables rather than emotionless AI input.
Recent developments have been applied to a wide variety of fields such as computer vision, natural language processing, drug discovery, bioinformatics, self-driving vehicles, to name a few. Thinking about the difficulty involved in application raises the question of what AI can do in a financial setting, with its complexities and the economic risk of going wrong, as in the end a finance employee has to justify his or her actions and not blame AI for basic mistakes.
The issue of data quality is foremost in the financial sector. In the financial world, abundance of data is not an issue. Data can easily be collected from a wide variety of sources such as instrument prices, news articles, stock fundamentals, social media posts, macroeconomic data, ESG data, credit card transactions, and so on. Some of this data is classified as structured and typically has a numerical quantity and a well-defined structure (e.g. stock prices). Structured data is relatively easy to feed into an ML model whereas unstructured data often requires extra processing to extract meaningful information (e.g. news articles, TV interviews, social media posts or images). The difficulty of this information-extraction process is clear if we take a news article, for example, that discusses “apples” as a commodity. While a human would relatively easily identify that the news article is not talking about Apple (the company) but rather “apple” the fruit in a commodity transaction, it is non-trivial to build an intelligent system that can replicate this feat.
The greater concern is not the quantity of data but rather its quality and usefulness, specifically the signal-to-noise ratio of the dataset. In a system as complex as the stock market, the reality is that the signal is highly drowned out by the noise and this can be both internal for the company and external from related geopolitical and economic factors. Therefore, a key challenge is building an intelligent system that can extract the meaningful signal in this sea of noise. Financial analysts predominantly solve this with mathematical tools and techniques to reliably discern signal from stochastic fluctuations.
In Saudi Arabia and the Gulf, where traditional and Islamic finance prevails, and complex hedging and futures trading is offered by a few national banks, incorporating AI into financial data analysis still has some way to go, but competition from established foreign banks could change that.
Dr. Mohamed Ramady
A more finance-specific problem is related to the time series nature of finance data — that is, events on Tuesday have to be analyzed with the knowledge of other events that happened on Monday. Only an experienced human can do this.
The time series nature of financial data makes the data “non-stationary.”
“Stationary” refers to data, which largely stays the same over time. For example, we could train AI to recognize images of animals. If we show AI an array of images, of say lions, it will learn that if it looks like a lion, hunts like a lion and runs like a lion, then it probably is a lion. But most importantly, whether the picture is from 1950 or 2021, they will contain similar features that can be picked up by AI. These features are stationary, and are what AI uses to identify what the picture represents. This is of no use in a fast-moving financial market.
Compare this with financial data, which exhibits highly non-stationary behavior. This phenomenon is often summarized by the mantra: “Past performance is no guarantee of future results,” as every good banker likes to remind his customers. There can be many patterns that arise, such as the stock price of a company going up if oil futures go up and bonds go down. However, there is absolutely no guarantee that this will be the case again in the future. As AI mostly makes its decisions from past results, the ever-changing nature of the financial market poses a significant obstacle for any AI system.
Assuming that one can handle these problems, the next issue is the timescale for which AI can produce profitable results. The reality is that the financial market is a complex dynamic system that will eventually autocorrect any profitable opportunities when saturated.
Any predictable pattern that AI could find may be washed out eventually as prices correct themselves. The only solution to this is to build a constantly evolving system that can keep up with the dynamical market. It will take time before AI becomes more embedded in the financial sector. In Saudi Arabia and the Gulf, where traditional and Islamic finance prevails, and complex hedging and futures trading is offered by a few national banks, incorporating AI into financial data analysis still has some way to go, but competition from established foreign banks could change that.
• Dr. Mohamed Ramady is a former senior banker, and professor of finance and economics at King Fahd University of Petroleum and Minerals, Dhahran.