I tried the common story of predicting the Nikkei 225 using deep learning (backtest)

change history

** 2016/8/14: Added the result when the signals of the three methods are synthesized. ** ** ** 2016/8/14: Added win rate. ** **

Overview

Last time (I tried the common story of using Deep Learning to predict the Nikkei 225), the return of the Nikkei 225 stocks on the previous day I tried to predict the rise and fall of the Nikkei Stock Average the next day using. This time, for the time being, it is an operation back test edition, so I tried to simulate what would happen if it was actually operated using the predicted return.

Disc breaker

As mentioned in the previous article, I am not responsible for any damages caused by actual operation using this method.

Operational premise

Originally, it is necessary to consider commissions, trading time, trading turnover, etc., but this time, we simply said, "Forecast at the closing price on the previous day, make a trading decision momentarily, and momentarily Nikkei 225. It makes a very unrealistic assumption that you can buy or sell. (Well, isn't it because the calculation is troublesome?)

The summary is as follows.

• If you have no position and you want to predict "rise", "buy"
• Hold for "rise" predictions while holding a position
• If you have a position and you predict "down", the position will be canceled.
• If you have a "down" prediction with no position, do nothing

Last result

The previous result was as follows. I tried three methods: Random Forest, Multilayer Perceptron (MLP), and Convolutional Neural Network (CNN).

• Random forest

• Multilayer Perceptron (MLP)

• Convolutional neural network

From an AUC perspective, the order is CNN> MLP> RF.

Operational backtest

The backtest period will be daily from 2008 to the latest. The first is 1000, and each is indexed. (Left axis) It also shows the difference against the Nikkei 225 index. (Right axis)

RF simulation results

MLP simulation results

CNN simulation results

Comparison of three methods

Results from synthetic signals of CNN, MLP, RF (Added 2016/08/14)

CNN is the best model performance, so let's synthesize RF and MLP around this. As a conditional expression

signal = (CNN signal) & (RF signal|MLP signal)

It feels like. In other words, only if you say that the CNN goes up and that both of the other two models go up, you decide that the final signal goes up.

Various statistics (added on August 14, 2016)

The result is synthetic> CNN> MLP> RF (in terms of risk / return). Even if the Nikkei average is Long Hold, it is a 52% win, so it seems that the rise and fall during this period is slightly closer to the uptrend. However, among them, all methods are superior to that, and 56% for synthesis is a fairly good number.

Summary

• It may change if you change the backtest period, but CNN seems to be the best in terms of performance.
• Since the risk-return result is the same as the order of AUC, there is a possibility that the operational result will be better if AUC can be raised.
• CNN is riding the wave of Abenomics by quickly detecting and avoiding the downturn of the Lehman shock, but its performance is not good in the current situation.
• It may be interesting to look at the behavior when the return is large, and conversely when the negative is large.
• Synthetic signals may smell a little after the fact, but even if it is actually used, it would be better to have multiple models. (Eva's magic)