The investment industry is undergoing a transformation that is largely attributable to technological advancements. Investment professionals are integrating new technologies, such as machine learning (ML), across the investment process, including portfolio construction. Many asset managers are beginning to incorporate ML algorithms in the portfolio optimization process seeking more efficient portfolios than would be possible under traditional methods, such as mean-variance optimization (MVO). These trends necessitate a fresh look at how ML is altering the portfolio construction process.
Investors will benefit from a basic understanding of ML algorithms and the impact these algorithms have on their portfolios. Ultimately, the strategies used by asset managers to construct client portfolios have a direct impact on the end investor. So investors should have sufficient awareness of these methods as they continue to gain in popularity. This article aims to provide an overview of the role ML algorithms play in the portfolio optimization process.
Background
The term ‘machine learning’ was first used by A.L. Samuel in 1959. Samuel conducted an experiment by training a computer to play checkers and concluded that the computer exhibited significant potential to learn. These results paved the way for further research on this topic and led to the development of increasingly powerful and sophisticated ML algorithms over the following decades. As a result, many industries, including investment management, have adopted these technologies in recent years.
ML algorithms are particularly useful when it comes to analyzing high dimensional data or datasets with non-linear relationships, which is becoming increasingly common with the rise of unstructured data and other alternative data sources. The two main categories for ML are supervised learning and unsupervised learning. With supervised learning, the ML algorithm detects patterns between a group of features (i.e., input variables) and a known target variable (i.e., output variable)[1]. This is referred to as a labeled dataset because the target variable is defined. In unsupervised learning, however, the dataset is unlabeled, and the target variable is unknown. Thus, the algorithm seeks to identify patterns within the input data. Exhibit 1 describes some of the common ML algorithms currently used by investment professionals.
Exhibit 1: Common Machine Learning Algorithms in Investment Management.
| ML Algorithm | Description |
| Least Absolute Shrinkage and Selection Operator (LASSO) | A form of penalized regression that includes a penalty term for each additional feature included in the regression model. The goal of this regularization technique is to create a parsimonious regression model by minimizing the number of features and to increase the accuracy of the model. |
| K-Means Clustering | Divides data into k clusters. Each observation in a cluster should have similar characteristics to the other observations, and each cluster should be distinctly different from the other clusters. |
| Hierarchical Clustering | Two types: bottom-up hierarchical clustering, which aggregates data into incrementally larger clusters, and top-down hierarchical clustering, which separates data into incrementally smaller clusters. This results in alternative ways of grouping data. |
| Artificial Neural Networks (ANNs) | A network of nodes that contains an input layer, a hidden layer, and an output layer. The input layer represents the features, and the hidden layer is where the algorithm learns and processes the inputs to generate the output(s). These algorithms have many uses, including speech and facial recognition. |
Investment professionals expect new analytical methods to be highly disruptive to the investment industry in the coming years. Respondents to a 2022 survey of more than 2,000 CFA Institute members predicted that new analytical methods like ML will be the most significant disruptor to job roles in the next five to 10 years among respondents. Exhibit 2 displays this result, along with other expected disruptors to job roles.
Exhibit 2. Factors Expected to Significantly Disrupt Job Roles in the Next 5 – 10 Years.
Portfolio Optimization
The development of neural networks in the 1960s laid the groundwork for many of the alternative methods to portfolio optimization using ML. In addition, the emergence of “expert systems”[2] has led investment professionals to rely increasingly on machines to help with solving complex problems. Some of the early uses of expert systems in finance include trading and financial planning expert systems.
The use of ML algorithms in the portfolio construction process has grown in popularity in recent years as investment professionals seek additional ways to enhance portfolio returns and gain a competitive edge. In particular, integrating ML algorithms in the portfolio construction process can address the challenges and limitations of traditional portfolio optimization methods, such as MVO.
One major limitation of MVO is that it only considers the mean and variance of returns when optimizing a portfolio and does not account for skewness in returns. In reality, however, investment returns tend to exhibit skewness. Specifically, research has shown that growth stocks have higher positive skewness in their returns, on average, than value stocks. To account for potential non-normality in investment returns, some investment professionals have opted to construct portfolios using mean-variance-skewness optimization models, or even mean-variance-skewness-kurtosis optimization models. These models, however, result in multi-objective optimization problems. ANNs can efficiently create mean-variance-skewness optimal portfolios to address this limitation.
Another shortfall of MVO is that it prevents investors from expressing their views on future asset performance. An investor, for instance, might expect bonds to outperform equities in the next six months. The Black-Litterman (1992) model enables investors to incorporate these perspectives into the portfolio optimization process. An alternative approach is to integrate the Black-Litterman (1992) model with ANNs, which has the potential to generate high benchmark-relative returns without taking excess risk.
The inputs in MVO are sensitive to measurement errors, which is especially true for expected return estimates. Thus, MVO has the potential to produce “optimal” portfolios that perform poorly. Reverse optimization can be a useful alternative to develop more accurate expected return estimates. Investment professionals can then use these improved estimates as inputs in traditional MVO to generate more efficient asset allocations. Investment professionals can also use ML algorithms to predict stock returns and incorporate these estimates in MVO. Alternatively, a recent study developed an enhanced portfolio optimization approach, which consists of using a correlation shrinkage parameter to improve estimated Sharpe ratios and then creating optimal portfolios based on these estimates.
Finally, a major challenge in portfolio optimization is estimating the covariance matrix, especially for high dimensional data. LASSO models can address this challenge by producing more accurate estimates of the covariance matrix than traditional methods, which is a critical input for MVO.
Conclusions
What are the implications of these trends for investment professionals? Clearly, the investment industry is rapidly evolving in response to new technologies. Investment professionals anticipate new analytical methods such as ML will significantly disrupt job roles in the coming years. As a result, practitioners are beginning to integrate ML algorithms across all areas of the investment process.
Many asset managers are attempting to gain a competitive advantage by creating portfolios with higher returns for a given level of risk (i.e., higher Sharpe ratios) through the integration of ML algorithms in the portfolio optimization process. Additionally, ML algorithms can overcome many of the challenges and limitations of traditional portfolio optimization methods, which has led investment professionals to seek more efficient portfolio construction methods. Investors will benefit from greater awareness of these trends to better understand the impact of new optimization methods on their portfolios.
[1] In some cases, the dataset might have more than one target variable.
[2] An expert system describes a computer program that can solve a complex problem normally solved by human experts. See: Expert system | AI, Knowledge Representation & Reasoning | Britannica