A New Method for Forward Testing?

A trading system is a set of rules for deciding when and how to buy or sell assets. These systems can be used to trade stocks, bonds, commodities, or other assets. Trading systems can be manual or automated. Manual systems require the trader to interpret market conditions and make decisions about when to buy or sell. Automated systems use computer algorithms to make trading decisions. Automated systems are usually designed through backtesting, which is a method of testing how well a system would have performed in the past.

Backtesting is an important step in the system design process. However, it alone is not enough. In order to test how the trading system would perform in real trading conditions, traders use paper trading, live trading, and forward testing. Forward testing is a type of simulation in which the trading system is tested using unseen data. This allows traders to see how the system would have performed if it had been used in real trading conditions. Forward testing is an important step in the system design process. It allows traders to assess the performance of the system and make improvements before using it in live trading.

Reference [1] proposed a so-called DNN-forwardtesting method for forward testing. The authors pointed out,

In this paper, we propose a stock market trading system that exploits deep neural networks as part of its main components improving a previous work (Letteri et al. [2022b]).

In such a system, the trades are guided by the values of a pre-selected technical indicator, as usual in algorithmic trading. However, the novelty of the presented approach is in the indicator selection technique: traders usually make such a selection by backtesting the system on the historical market data and choosing the most profitable indicator with respect to the known past. On the other hand, in our approach, such most profitable indicator is chosen by DNN-forwardtesting it on the probable future predicted by a deep neural network trained on the historical data.

So basically, they utilized historical data to make predictions and then used the predicted data to choose the best indicators for their trading system. The chosen systems were then tested using real data. As far as forward testing is concerned, the proposed method is just ordinary out-of-sample testing.

The authors also claimed,

As discussed in the paper, neural networks outperform the most common statistical methods in stock price prediction: indeed, their predicted future allows to make a very accurate selection of the indicator to apply, which takes into account trends that would be very difficult to capture through backtesting.

To validate this claim, we applied our methodology on two very different assets with medium volatility, and the results show that our DNN forwardtesting-based trading system achieves a profit that is equal or higher than the one of a traditional backtesting-based trading system.

We examined Tables 8 and 9 in the paper and observed that the authors used the results of only 3 and 1 trades, respectively, to support their claim.

Let us know what you think in the comments below.

References

[1] Ivan Letteri, Giuseppe Della Penna, Giovanni De Gasperis, Abeer Dyoub, DNN-ForwardTesting: A New Trading Strategy Validation using Statistical Timeseries Analysis and Deep Neural Networks, 2022, arXiv:2210.11532