A note on speeding up Python code with Numba

Introduction

Comparison of moving average calculation time written in Python So, I found that the moving average (LWMA) using the for statement is slow and unusable, but there are some technical indicators of MetaTrader that can not be written without using the for statement, so I can not give up. So I tried to speed it up.

For the time being, I knew that there was Cython for speeding up, but it was that I had to rewrite the code, so when I checked other things, there was Numba. This time is a memo when I tried Numba.

Very slow code with for statement

import numpy as np
import pandas as pd
dataM1 = pd.read_csv('DAT_ASCII_EURUSD_M1_2015.csv', sep=';',
                     names=('Time','Open','High','Low','Close', ''),
                     index_col='Time', parse_dates=True)

def LWMA(s, ma_period):
    y = pd.Series(0.0, index=s.index)
    for i in range(len(y)):
        if i<ma_period-1: y[i] = 'NaN'
        else:
            y[i] = 0
            for j in range(ma_period):
                y[i] += s[i-j]*(ma_period-j)
            y[i] /= ma_period*(ma_period+1)/2
    return y

%timeit MA = LWMA(dataM1['Close'], 10)

1 loop, best of 3: 3min 14s per loop

Like previous article, it will take more than 3 minutes.

Try using Numba

Numba seems to be in Anaconda, so just import it and add @ numba.jit.

import numba
@numba.jit
def LWMA(s, ma_period):
    y = pd.Series(0.0, index=s.index)
    for i in range(len(y)):
        if i<ma_period-1: y[i] = 'NaN'
        else:
            y[i] = 0
            for j in range(ma_period):
                y[i] += s[i-j]*(ma_period-j)
            y[i] /= ma_period*(ma_period+1)/2
    return y

%timeit MA = LWMA(dataM1['Close'], 10)

1 loop, best of 3: 3min 14s per loop

Oh, the same result. It has no effect. Is Numba dedicated to Numpy by name?

Try changing pandas to numpy

I changed the input data from pandas Series type to numpy array type.

@numba.jit
def LWMA(s, ma_period):
    y = np.zeros(len(s))
    for i in range(len(y)):
        if i<ma_period-1: y[i] = 'NaN'
        else:
            y[i] = 0
            for j in range(ma_period):
                y[i] += s[i-j]*(ma_period-j)
            y[i] /= ma_period*(ma_period+1)/2
    return y

%timeit MA = LWMA(dataM1['Close'].values, 10)

1 loop, best of 3: 2.11 s per loop

This time it's faster. About 90 times. However, it is still slower than the few milliseconds of scipy.

Try to remove the if statement

Even if I compile it, it is decided that the code should be simple, so I removed the if statement. Actually, this if statement was a code that may or may not be present.

@numba.jit
def LWMA(s, ma_period):
    y = np.zeros(len(s))
    for i in range(len(y)):
        for j in range(ma_period):
            y[i] += s[i-j]*(ma_period-j)
        y[i] /= ma_period*(ma_period+1)/2
    return y

%timeit MA = LWMA(dataM1['Close'].values, 10)

100 loops, best of 3: 5.73 ms per loop

came out! millisecond. Even if there was a for statement, it could be as fast as scipy. You can do it, Python.

Summary

I was able to speed up the code that was slowed down by using the for statement by using Numba. However, it worked for numpy and had no effect for pandas.

Reference article

I thought it was slow to use the for statement in NumPy, but that wasn't the case