[Python] Try optimizing FX systole parameters with random search

Optimization of FX systole parameters in Python So, I tried all the combinations of parameters of the trading system to find the optimum solution.

If the number of combinations is small, brute force is fine, but if the number of combinations is large, it becomes quite difficult. The optimization algorithm for that is [Metaheuristics](https://ja.wikipedia.org/wiki/%E3%83%A1%E3%82%BF%E3%83%92%E3%83%A5% It is known as E3% 83% BC% E3% 83% AA% E3% 82% B9% E3% 83% 86% E3% 82% A3% E3% 82% AF% E3% 82% B9). I wanted to implement a well-known genetic algorithm, but before that, I wrote a simpler random search in Python.

Preparation

Put the function to convert 1-minute data to another time frame and the technical indicator function in indicators.py, and the function to perform the previous backtest and its evaluation in backtest.py. The source can be found on GitHub. The following code will create 1 hour data in ʻohlc`.

import numpy as np
import pandas as pd
import indicators as ind #indicators.Import of py
from backtest import Backtest,BacktestReport

dataM1 = pd.read_csv('DAT_ASCII_EURUSD_M1_2015.csv', sep=';',
                     names=('Time','Open','High','Low','Close', ''),
                     index_col='Time', parse_dates=True)
dataM1.index += pd.offsets.Hour(7) #7 hour offset
ohlc = ind.TF_ohlc(dataM1, 'H') #Creation of 1-hour data

Trading system parameters to optimize

In the previous optimization, we optimized by changing two parameters within a certain range as follows.

SlowMAperiod = np.arange(10, 61) #Range of long-term moving average period
FastMAperiod = np.arange(5, 31)  #Range of short-term moving averages

At this rate, there are only 1066 ways, but I will change the number of combinations later.

Random search

In a simple random search, parameter values should be randomly generated in sequence and the one with the highest backtest evaluation should be adopted, but here, one time so that it can be applied to genetic algorithms I tried about 20 parameter combinations in each generation and tried to repeat it for several generations.

def Optimize(ohlc, SlowMAperiod, FastMAperiod):
    def shift(x, n=1): return np.concatenate((np.zeros(n), x[:-n])) #Shift function

    SlowMA = np.empty([len(SlowMAperiod), len(ohlc)]) #Long-term moving average
    for i in range(len(SlowMAperiod)):
        SlowMA[i] = ind.iMA(ohlc, SlowMAperiod[i])

    FastMA = np.empty([len(FastMAperiod), len(ohlc)]) #Short-term moving average
    for i in range(len(FastMAperiod)):
        FastMA[i] = ind.iMA(ohlc, FastMAperiod[i])
    
    M = 20 #Population
    Eval = np.zeros([M, 6])  #Evaluation item
    Param = np.zeros([M, 2], dtype=int) #Parameters
    RandomSearch(Param, Eval[:,0], SlowMAperiod, FastMAperiod) #Parameter initialization
    gens = 0 #Number of generations
    while gens < 100:
        for k in range(M):
            i = Param[k,0]
            j = Param[k,1]
            #Buy entry signal
            BuyEntry = (FastMA[j] > SlowMA[i]) & (shift(FastMA[j]) <= shift(SlowMA[i]))
            #Sell entry signal
            SellEntry = (FastMA[j] < SlowMA[i]) & (shift(FastMA[j]) >= shift(SlowMA[i]))
            #Buy exit signal
            BuyExit = SellEntry.copy()
            #Sell exit signal
            SellExit = BuyEntry.copy()
            #Backtest
            Trade, PL = Backtest(ohlc, BuyEntry, SellEntry, BuyExit, SellExit) 
            Eval[k] = BacktestReport(Trade, PL)
        #Alternation of generations
        Param = RandomSearch(Param, Eval[:,0], SlowMAperiod, FastMAperiod)
        gens += 1
    Slow = SlowMAperiod[Param[:,0]]
    Fast = FastMAperiod[Param[:,1]]
    return pd.DataFrame({'Slow':Slow, 'Fast':Fast, 'Profit': Eval[:,0], 'Trades':Eval[:,1],
                         'Average':Eval[:,2],'PF':Eval[:,3], 'MDD':Eval[:,4], 'RF':Eval[:,5]},
                         columns=['Slow','Fast','Profit','Trades','Average','PF','MDD','RF'])

The difference from the last time is that we prepared an array Param to store the combination of parameters in each generation. Then, the set of parameters with the highest evaluation value (total profit / loss in this case) is saved (elite save) for each generation, and the remaining parameters are randomly generated repeatedly.

A function that randomly generates parameters with elite storage can be written as:

#Random search with elite save
def RandomSearch(Param, Eval, SlowMAperiod, FastMAperiod):
    Param = Param[np.argsort(Eval)[::-1]] #sort
    NewParam = np.vstack((np.random.randint(len(SlowMAperiod), size=len(Eval)),
                          np.random.randint(len(FastMAperiod), size=len(Eval)))).T
    NewParam[0] = Param[0] #Elite save
    return NewParam

The method is to sort the evaluation values and set the 0th parameter again. When you do this, you get:

result = Optimize(ohlc, SlowMAperiod, FastMAperiod)
result.sort_values('Profit', ascending=False)
Slow Fast Profit Trades Average PF MDD RF
0 27 8 2507.1 264.0 9.496591 1.423497 485.1 5.168213
15 18 5 944.1 428.0 2.205841 1.110187 693.1 1.362141
19 48 14 825.4 238.0 3.468067 1.131883 927.7 0.889727
16 42 29 720.4 308.0 2.338961 1.094974 1011.8 0.711998
8 44 25 589.3 246.0 2.395528 1.089205 1141.1 0.516432
6 60 25 588.8 120.0 4.906667 1.126946 1025.0 0.574439
4 22 5 493.5 124.0 3.979839 1.105354 1106.3 0.446082
11 26 7 391.2 206.0 1.899029 1.069326 1099.8 0.355701
17 13 7 343.2 152.0 2.257895 1.069697 990.9 0.346352
18 45 20 174.5 160.0 1.090625 1.032210 1118.4 0.156026
12 38 20 169.8 170.0 0.998824 1.029754 939.8 0.180677
3 21 11 148.6 460.0 0.323043 1.016830 1203.1 0.123514
5 34 13 -342.0 170.0 -2.011765 0.939258 1160.8 -0.294624
7 42 22 -374.4 156.0 -2.400000 0.933744 1189.6 -0.314728
1 51 20 -441.4 160.0 -2.758750 0.922788 1089.9 -0.404991
9 58 27 -1485.5 222.0 -6.691441 0.787506 2394.0 -0.620510
10 19 25 -1620.0 234.0 -6.923077 0.775828 1845.7 -0.877716
13 33 16 -1625.0 400.0 -4.062500 0.827957 2307.7 -0.704164
2 44 24 -1885.7 508.0 -3.712008 0.828287 2652.2 -0.710995
14 55 12 -2048.3 398.0 -5.146482 0.776908 2813.6 -0.728000

In this way, the optimal solution has been sought, but it is natural because we have tried 20 individuals for 100 generations, that is, nearly 2000 times.

Since there are only two parameters this time, the number of combinations does not increase so much,

SlowMAperiod = np.arange(10, 161) #Range of long-term moving average period
FastMAperiod = np.arange(5, 131)  #Range of short-term moving averages

Let's increase it to 19,026 ways like this. Result is

Slow Fast Profit Trades Average PF MDD RF
0 52 48 2003.4 190.0 10.544211 1.426773 1106.4 1.810738
5 125 26 1312.6 58.0 22.631034 1.513939 896.4 1.464302
16 49 47 1203.9 62.0 19.417742 1.448046 773.0 1.557439
11 56 55 989.6 116.0 8.531034 1.245266 922.0 1.073319
14 52 34 704.0 62.0 11.354839 1.261632 527.2 1.335357
13 152 36 545.5 75.0 7.273333 1.133198 936.5 0.582488
2 140 58 457.4 196.0 2.333673 1.085960 867.1 0.527505
18 40 109 308.1 88.0 3.501136 1.073977 1435.0 0.214704
6 75 107 255.1 78.0 3.270513 1.066815 1145.1 0.222775
3 158 85 172.7 122.0 1.415574 1.036899 1530.6 0.112832
12 68 32 -622.3 90.0 -6.914444 0.836838 1629.9 -0.381803
10 120 111 -638.0 37.0 -17.243243 0.787057 1233.4 -0.517269
8 153 30 -667.3 35.0 -19.065714 0.776284 1558.8 -0.428086
7 43 74 -749.0 39.0 -19.205128 0.770245 1766.8 -0.423930
15 63 104 -774.3 92.0 -8.416304 0.835339 1692.6 -0.457462
9 154 56 -1296.9 58.0 -22.360345 0.675945 1733.3 -0.748226
1 152 24 -1315.2 74.0 -17.772973 0.664627 2130.3 -0.617378
19 155 27 -1363.8 60.0 -22.730000 0.672006 1474.4 -0.924986
17 109 40 -1478.5 62.0 -23.846774 0.651379 1784.5 -0.828523
4 148 122 -1957.4 100.0 -19.574000 0.626400 2171.4 -0.901446
Therefore, the optimum solution could not be found. However, if you repeat this several times, the optimum solution will be found. Well, if the combination is brute force, you can find a reasonable solution by random search.

Recommended Posts

[Python] Try optimizing FX systole parameters with random search
[Python] Try optimizing FX systole parameters with a genetic algorithm
Try scraping with Python.
Sequential search with Python
Binary search with python
Binary search with Python3
Try Python output with Haxe 3.2
Full bit search with Python
Backtesting FX Systre with Python (2)
Try running Python with Try Jupyter
Search engine work with python
Search twitter tweets with python
Try face recognition with Python
Streamline web search with python
Try scraping with Python + Beautiful Soup
Testing with random numbers in Python
Try to operate Facebook with Python
random French number generator with python
Learn search with Python # 2bit search, permutation search
Try singular value decomposition with Python
Try face recognition with python + OpenCV
Try frequency control simulation with Python
Try to reproduce color film with Python
Algorithm learned with Python 10th: Binary search
Try logging in to qiita with Python
Algorithm learned with Python 9th: Linear search
Try mathematical formulas using Σ with python
Try working with binary data in Python
Search the maze with the python A * algorithm
Try using Python with Google Cloud Functions
Convert FX 1-minute data to 5-minute data with Python
Try HTML scraping with a Python library
Try calling Python from Ruby with thrift
Try drawing a map with python + cartopy 0.18.0
[Continued] Try PLC register access with Python
Algorithm learned with Python 12th: Maze search
Try assigning or switching with Python: lambda
[For beginners] Try web scraping with Python
Try python
Try to predict FX with LSTM using Keras + Tensorflow Part 3 (Try brute force parameters)
nginxparser: Try parsing nginx config file with Python
Csv output from Google search with [Python]! 【Easy】
Automatically search and download YouTube videos with Python
Causal reasoning and causal search with Python (for beginners)
Try it with Word Cloud Japanese Python JupyterLab.
Try running Google Chrome with Python and Selenium
Try to solve the man-machine chart with Python
Try to draw a life curve with python
Try to make a "cryptanalysis" cipher with Python
Try to automatically generate Python documents with Sphinx
Try working with Mongo in Python on Mac
[Python3] [Ubuntu16] [Docker] Try face recognition with OpenFace
Try to make a dihedral group with Python
Try to detect fish with python + OpenCV2.4 (unfinished)
Perform a Twitter search from Python and try to generate sentences with Markov chains.
Try to implement permutation full search that often appears in competition pros with python