Newsletter: How Machine Learning Enhances Market Volatility Forecasting Accuracy
Explore how modern machine learning models can significantly improve accuracy in forecasting market volatility trends.
Machine learning has many applications in finance, such as asset pricing, risk management, portfolio optimization, and fraud detection. In this issue, I discuss the use of machine learning in forecasting volatility.
Web-only posts Recap
Below is a summary of the web-only posts I published during last week.
Effectiveness of Regulatory Interest Rate Risk Measures
Can AI Replace Human Programmers?
Harvesting the Equity Risk Premia Through Options
Short-Selling Leveraged Exchange-Traded Funds
Tail Risk Hedging Using Put Options: Is It Effective?
Seasonality in Return Skewness: the Day-of-Week Effect
Using Machine Learning to Predict Market Volatility
The unpredictability of the markets is a well-known fact. Despite this, many traders and portfolio managers continue to try to predict market volatility and manage their risks accordingly. Usually, econometric models such as GARCH are used to forecast market volatility.
In recent years, machine learning has been shown to be capable of predicting market volatility with accuracy. Reference [1] explored how machine learning can be used in this context.
Findings
-Machine learning models can accurately forecast stock return volatility using a small set of key predictors: realized volatility, idiosyncratic volatility, bid-ask spread, and returns.
-These predictors align with existing empirical findings, reinforcing the traditional risk-return trade-off in finance.
-ML methods effectively capture both the magnitude and direction of predictor impacts, along with their interactions, without requiring pre-specified model assumptions.
-Large current-period volatility values strongly predict higher future volatility; small values have a muted or negative impact.
-LSTM models outperform feedforward neural networks and regression trees by leveraging temporal patterns in historical data.
-An LSTM using only volatility and return history over one year performs comparably to more complex models with additional predictors.
-LSTM models function as distribution-free alternatives to traditional econometric models like GARCH.
-Optimal lag length remains critical in LSTM performance and must be selected through model training.
-The study reports an average predicted realized volatility of 44.1%, closely matching the actual value of 43.8%.
-Out-of-sample R² values achieved are significantly higher than those typically reported in related volatility forecasting literature.
In short, the paper aimed to demonstrate the potential of machine learning for modeling market volatility. In particular, the authors have shown how the LSTM model can be used to predict market volatility and manage risks. The results suggest that this is a promising alternative approach to traditional econometric models like GARCH.
Reference
[1] Filipovic, Damir and Khalilzadeh, Amir, Machine Learning for Predicting Stock Return Volatility (2021). Swiss Finance Institute Research Paper No. 21-95
Machine Learning Models for Predicting Implied Volatility Surfaces
The Implied Volatility Surface (IVS) represents the variation of implied volatility across different strike prices and maturities for options on the same underlying asset. It provides a three-dimensional view where implied volatility is plotted against strike price (moneyness) and time to expiration, capturing market sentiment about expected future volatility.
Reference [2] examines five methods for forecasting the Implied Volatility Surface of short-dated options. These methods are applied to forecast the level, slope, and curvature of the IVS.
Findings
-The study evaluates five methods—OLS, AR(1), Elastic Net, Random Forest, and Neural Network—to forecast the implied volatility surface (IVS) of weekly S&P 500 options.
-Forecasts focus on three IVS characteristics: level, slope, and curvature.
-Random Forest consistently outperforms all other models across these three IVS dimensions.
-Non-learning-based models (OLS, AR(1)) perform comparably to some machine learning methods, highlighting their continued relevance.
-Neural Networks forecast the IVS level reasonably well but perform poorly in predicting slope and curvature.
-Elastic Net, a linear machine learning model, is consistently outperformed by the non-linear models (Random Forest and Neural Network) for the level characteristic.
-The study emphasizes the importance of model selection based on the specific IVS characteristic being forecasted.
-Performance evaluation is supported using the cumulative sum of squared error difference (CSSED) and permutation variable importance (VI) metrics.
-The research highlights the utility of Random Forest in capturing complex, non-linear patterns in IVS dynamics.
-Accurate IVS forecasting is valuable for derivative pricing, hedging, and risk management strategies.
This research highlights the potential of machine learning in forecasting the implied volatility surface, a key element in options pricing and risk management. Among the five methods studied, Random Forest stands out as the most consistent and accurate across multiple IVS features.
Reference
[2] Tim van de Noort, Forecasting the Characteristics of the Implied Volatility Surface for Weekly Options: How do Machine Learning Methods Perform? Erasmus University, 2024
Closing Thoughts
These studies highlight the growing effectiveness of machine learning in financial forecasting, particularly for market volatility and implied volatility surfaces. Models like LSTM and Random Forest demonstrate clear advantages over traditional methods by capturing complex patterns and dependencies. As financial markets evolve, leveraging such tools offers a promising path for enhancing predictive accuracy and risk management.
Featured Resource
Anton Vorobets just released a new quant course on Portfolio Construction and Risk Management. The course focuses on applied aspects of quantitative and quantamental investment management, drawing on perspectives from sophisticated multi-asset investors.
To learn more about the course, follow the link below:
Educational Video
Machine Learning for Realised Volatility Forecasting
In this video, Eghbal Rahimkina and Ser-Huang Poon explore how machine learning can improve realized volatility forecasting. They compare traditional models like HAR with ML approaches using data from limit order books and news sentiment. Their results show that ML models, especially LSTMs, perform better on normal trading days but struggle during market jumps. The presentation also covers data cleaning, model setup, and evaluation methods, offering a clear view of how advanced techniques can enhance financial forecasting.
Around the Quantosphere
- The End Of The Quant? How AI Is Democratizing Financial Analysis (forbes)
- Jane Street barred from Indian markets as regulator freezes $566 million over Nifty 50 manipulation claims (cnbc)
-Inside the high-frequency trading floor, where grads are paid $250,000 (AFR)
-How to get a capital markets job in an investment bank (Financial Careers)
-Implied Vols Collapse As Stocks Hit New Record High (Cboe)
-Hedge fund returns halfway through 2025: How big names like Citadel, Balyasny, and more have managed this year's markets. (Bundle)
-Hedge Fund Strategy Built on Catastrophes Taps Hot New Trend (Yahoo Finance)
-Hedge funds dive into private credit. (Financial Times)
Recent Newsletters
Below is a summary of the weekly newsletters I sent out recently
-Predicting Corrections and Economic Slowdowns (11 min)
-Rethinking Leveraged ETFs and Their Options (12 min)
-Using Skewness and Kurtosis to Enhance Trading and Risk Management (10 min)
-Gold Ratios as Stock Market Predictors (11 min)
-Volatility of Volatility: Insights from VVIX (11 min)
Refer a Friend
If you like this newsletter, then help us grow by referring a friend or two. As a token of appreciation, we'll send you PDFs that include links to our blog posts about financial derivatives, time series analysis and trading strategies, along with the accompanying Excel files or Python codes.
1 referral:
https://harbourfronts.com/wp-content/uploads/2024/12/fin_deriv-1.gif
2 referrals:
https://harbourfronts.com/wp-content/uploads/2024/12/risk_trading.gif
Use the referral link below or the “Share” button on any post.
Disclaimer
This newsletter is not investment advice. It is provided solely for entertainment and educational purposes. Always consult a financial professional before making any investment decisions.
We are not responsible for any outcomes arising from the use of the content and codes provided in the outbound links. By continuing to read this newsletter, you acknowledge and agree to this disclaimer.