There are many libraries for backtesting cistre with Python, but it is difficult for people who entered from MetaTrader to understand, so I wrote the code for backtesting after practicing Python.
Backtesting Systre with Python
However, in the first version, I wrote that it works first, so it was quite wasteful and the execution speed was slow, so I tried to improve it a little this time.
In the case of stock prices, there are many things that can be downloaded directly from Yahoo! etc., but in the case of FX, data of multiple time frames such as 5 minutes or 15 minutes may be used, so 1 minute as basic data. I want foot data.
In that case, the data is also large, so I think it is more convenient to read the data downloaded in advance. Here, download it as sample data from the following site.
There are several formats for the same data, but I will download the data in Generic ASCII format because I use the pandas function.
Now, let's read the downloaded EUR / USD 1-minute data for 2015'DAT_ASCII_EURUSD_M1_2015.csv' with the pandas function. However, if this is left as it is, the market price at the beginning of the week will start on Sunday, so delay it by 7 hours and start from 00:00 on Monday.
import numpy as np
import pandas as pd
dataM1 = pd.read_csv('DAT_ASCII_EURUSD_M1_2015.csv', sep=';',
names=('Time','Open','High','Low','Close', ''),
index_col='Time', parse_dates=True)
dataM1.index += pd.offsets.Hour(7) #7 hour offset
Next, create the data of any time frame from the data of 1 minute bar. Again, you can easily do this by using the pandas resample ()
, ʻohlc () `functions.
#A function that creates four-legged data in a timeframe specified by tf from df data
def TF_ohlc(df, tf):
x = df.resample(tf).ohlc()
O = x['Open']['open']
H = x['High']['high']
L = x['Low']['low']
C = x['Close']['close']
ret = pd.DataFrame({'Open': O, 'High': H, 'Low': L, 'Close': C},
columns=['Open','High','Low','Close'])
return ret.dropna()
ohlc = TF_ohlc(dataM1, 'H') #Creation of 1-hour data
The keyword specified for tf
is
'T'
'H'
'D'
'W'
'M'
So, for 15 minutes, just write '15T'
. With this, you can create even minutes of data. Here, I tried to make 1-hour data.
The technical indicators used in the trading rules of Systre are created on GitHub. You can use it by fetching only indicators.py. I'll omit the source code, but I've avoided using pandas for internal iterations and used Numba, so I think there are no more time-consuming functions. ~~ However, parabolic SAR takes some time due to the algorithm. ~~
For example, a 10-bar moving average and a 30-bar moving average can be written as:
import indicators as ind #indicators.Import of py
FastMA = ind.iMA(ohlc, 10) #Short-term moving average
SlowMA = ind.iMA(ohlc, 30) #Long-term moving average
You can use the pandas plot function to display the created technical index on the chart, but since you may want to scale it, you often see it on market-related sites HighCharts. Let's use the Python library displayed in /).
Here, I will use the easy-to-install pandas-highcharts.
Installation is
pip install pandas-highcharts
Is OK. The code to display the closing price of FX and the two moving averages is as follows.
from pandas_highcharts.display import display_charts
df = pd.DataFrame({'Close': ohlc['Close'], 'FastMA': FastMA, 'SlowMA': SlowMA})
display_charts(df, chart_type="stock", title="MA cross", figsize=(640,480), grid=True)
When you do this, you will see a chart like this.
If you display one year's worth, the moving averages will hardly overlap, so if you zoom in properly, it will look like this, and you will be able to see the difference in the moving averages.
It's quite convenient.
As an example of a trading system, take the classic moving average crossing system. The trading rules are
It's as simple as that. This signal is an entry signal, and the exit signal for closing a position is that the buy exit is the same as the sell entry and the sell exit is the same as the buy entry. This is the so-called transfer trading.
#Buy entry signal
BuyEntry = ((FastMA > SlowMA) & (FastMA.shift() <= SlowMA.shift())).values
#Sell entry signal
SellEntry = ((FastMA < SlowMA) & (FastMA.shift() >= SlowMA.shift())).values
#Buy exit signal
BuyExit = SellEntry.copy()
#Sell exit signal
SellExit = BuyEntry.copy()
Here, each signal is a bool type array of numpy. If you think about it normally, you would probably judge the signal while turning the time series data, but in the case of Python, it is faster to process it collectively in an array, so put the signal in an array.
We will finally backtest. Input historical data and the above trading signal, and output the actual trading price and profit / loss as time series data.
def Backtest(ohlc, BuyEntry, SellEntry, BuyExit, SellExit, lots=0.1, spread=2):
Open = ohlc['Open'].values #Open price
Point = 0.0001 #Value of 1pip
if(Open[0] > 50): Point = 0.01 #1 pip value of cross circle
Spread = spread*Point #Spread
Lots = lots*100000 #Actual trading volume
N = len(ohlc) #FX data size
BuyExit[N-2] = SellExit[N-2] = True #Finally forced exit
BuyPrice = SellPrice = 0.0 #Selling price
LongTrade = np.zeros(N) #Buy trade information
ShortTrade = np.zeros(N) #Sell trade information
LongPL = np.zeros(N) #Profit and Loss of Buy Position
ShortPL = np.zeros(N) #Sell position profit or loss
for i in range(1,N):
if BuyEntry[i-1] and BuyPrice == 0: #Buy entry signal
BuyPrice = Open[i]+Spread
LongTrade[i] = BuyPrice #Buy position open
elif BuyExit[i-1] and BuyPrice != 0: #Buy exit signal
ClosePrice = Open[i]
LongTrade[i] = -ClosePrice #Buy position closed
LongPL[i] = (ClosePrice-BuyPrice)*Lots #Profit and loss settlement
BuyPrice = 0
if SellEntry[i-1] and SellPrice == 0: #Sell entry signal
SellPrice = Open[i]
ShortTrade[i] = SellPrice #Sell position open
elif SellExit[i-1] and SellPrice != 0: #Sell exit signal
ClosePrice = Open[i]+Spread
ShortTrade[i] = -ClosePrice #Sell position closed
ShortPL[i] = (SellPrice-ClosePrice)*Lots #Profit and loss settlement
SellPrice = 0
return pd.DataFrame({'Long':LongTrade, 'Short':ShortTrade}, index=ohlc.index),\
pd.DataFrame({'Long':LongPL, 'Short':ShortPL}, index=ohlc.index)
Trade, PL = Backtest(ohlc, BuyEntry, SellEntry, BuyExit, SellExit)
In the previous version, this part was divided into several stages, but this time I tried to summarize it. Originally, it would be good if we could trade only with signals, but there are cases where signals are output even if there are positions, so we also checked the existence of positions.
In the output of this function, Trade
stores the buy / sell price as a positive value and the settlement price as a negative value for each buy and sell. The realized profit / loss at the time of settlement is stored in PL
.
With the information on Trade
and PL
, you can roughly evaluate the trading system. For example, it looks like this.
def BacktestReport(Trade, PL):
LongPL = PL['Long']
LongTrades = np.count_nonzero(Trade['Long'])//2
LongWinTrades = np.count_nonzero(LongPL.clip_lower(0))
LongLoseTrades = np.count_nonzero(LongPL.clip_upper(0))
print('Number of buy trades=', LongTrades)
print('Number of winning trades=', LongWinTrades)
print('Maximum win trade=', LongPL.max())
print('Average win trade=', round(LongPL.clip_lower(0).sum()/LongWinTrades, 2))
print('Number of negative trades=', LongLoseTrades)
print('Maximum negative trade=', LongPL.min())
print('Average negative trade=', round(LongPL.clip_upper(0).sum()/LongLoseTrades, 2))
print('Win rate=', round(LongWinTrades/LongTrades*100, 2), '%\n')
ShortPL = PL['Short']
ShortTrades = np.count_nonzero(Trade['Short'])//2
ShortWinTrades = np.count_nonzero(ShortPL.clip_lower(0))
ShortLoseTrades = np.count_nonzero(ShortPL.clip_upper(0))
print('Number of sell trades=', ShortTrades)
print('Number of winning trades=', ShortWinTrades)
print('Maximum win trade=', ShortPL.max())
print('Average win trade=', round(ShortPL.clip_lower(0).sum()/ShortWinTrades, 2))
print('Number of negative trades=', ShortLoseTrades)
print('Maximum negative trade=', ShortPL.min())
print('Average negative trade=', round(ShortPL.clip_upper(0).sum()/ShortLoseTrades, 2))
print('Win rate=', round(ShortWinTrades/ShortTrades*100, 2), '%\n')
Trades = LongTrades + ShortTrades
WinTrades = LongWinTrades+ShortWinTrades
LoseTrades = LongLoseTrades+ShortLoseTrades
print('Total number of trades=', Trades)
print('Number of winning trades=', WinTrades)
print('Maximum win trade=', max(LongPL.max(), ShortPL.max()))
print('Average win trade=', round((LongPL.clip_lower(0).sum()+ShortPL.clip_lower(0).sum())/WinTrades, 2))
print('Number of negative trades=', LoseTrades)
print('Maximum negative trade=', min(LongPL.min(), ShortPL.min()))
print('Average negative trade=', round((LongPL.clip_upper(0).sum()+ShortPL.clip_upper(0).sum())/LoseTrades, 2))
print('Win rate=', round(WinTrades/Trades*100, 2), '%\n')
GrossProfit = LongPL.clip_lower(0).sum()+ShortPL.clip_lower(0).sum()
GrossLoss = LongPL.clip_upper(0).sum()+ShortPL.clip_upper(0).sum()
Profit = GrossProfit+GrossLoss
Equity = (LongPL+ShortPL).cumsum()
MDD = (Equity.cummax()-Equity).max()
print('Gross profit=', round(GrossProfit, 2))
print('Total loss=', round(GrossLoss, 2))
print('Total profit and loss=', round(Profit, 2))
print('Profit factor=', round(-GrossProfit/GrossLoss, 2))
print('Average profit and loss=', round(Profit/Trades, 2))
print('Maximum drawdown=', round(MDD, 2))
print('Recovery factor=', round(Profit/MDD, 2))
return Equity
Equity = BacktestReport(Trade, PL)
Number of buy trades= 113
Number of winning trades= 39
Maximum win trade= 440.4
Average win trade= 82.57
Number of negative trades= 74
Maximum negative trade= -169.4
Average negative trade= -43.89
Win rate= 34.51 %
Number of sell trades= 113
Number of winning trades= 49
Maximum win trade= 327.6
Average win trade= 78.71
Number of negative trades= 64
Maximum negative trade= -238.5
Average negative trade= -43.85
Win rate= 43.36 %
Total number of trades= 226
Number of winning trades= 88
Maximum win trade= 440.4
Average win trade= 80.42
Number of negative trades= 138
Maximum negative trade= -238.5
Average negative trade= -43.87
Win rate= 38.94 %
Gross profit= 7077.0
Total loss= -6054.4
Total profit and loss= 1022.6
Profit factor= 1.17
Average profit and loss= 4.52
Maximum drawdown= 1125.4
Recovery factor= 0.91
Here, each item is displayed, but when performing machine learning or optimization, it is only necessary to calculate the evaluation value.
You will often want to look at the asset curve as a system valuation. In the above function, the asset curve is output as ʻEquity`, so if you add the initial assets and make a graph, it will be as follows.
Initial = 10000 #Initial assets
display_charts(pd.DataFrame({'Equity':Equity+Initial}), chart_type="stock", title="Asset curve", figsize=(640,480), grid=True)
Finally, let's display where you bought and sold on the chart. You can display only the points you bought and sold, but I don't know how to do it in HighCharts, so I will connect the points where the position was opened and the points where it was closed with a line.
Let's convert the Trade
information to a line with the following code.
def PositionLine(trade):
PosPeriod = 0 #Position period
Position = False #Presence or absence of position
Line = trade.copy()
for i in range(len(Line)):
if trade[i] > 0: Position = True
elif Position: PosPeriod += 1 #Count the duration of a position
if trade[i] < 0:
if PosPeriod > 0:
Line[i] = -trade[i]
diff = (Line[i]-Line[i-PosPeriod])/PosPeriod
for j in range(i-1, i-PosPeriod, -1):
Line[j] = Line[j+1]-diff #Interpolate the duration of a position
PosPeriod = 0
Position = False
if trade[i] == 0 and not Position: Line[i] = 'NaN'
return Line
df = pd.DataFrame({'Open': ohlc['Open'],
'Long': PositionLine(Trade['Long'].values),
'Short': PositionLine(Trade['Short'].values)})
display_charts(df, chart_type="stock", title="Trade chart", figsize=(640,480), grid=True)
It will be displayed in High Charts in the same way, so try zooming appropriately.
Since it is a trading system, you can see that the Long position and the Short position appear alternately.
I wrote the code in Python to backtest the system so that the trading rules can be described only by technical indicators. The library that can display graphs from Python to HighCharts was quite convenient. There is still room for improvement, so I would like to continue writing when I feel like it.
The continuation is here. Backtesting FX Systre with Python (2)
The code posted in this article has been uploaded below. MT5IndicatorsPy/EA_sample.ipynb
Recommended Posts