Let's move Cython and Numba easily

Outline

numba is a module that runs Python using a JIT (just-in-time) compiler. Cython is a programming language that can be written like Python but is precompiled like C / C ++.

I will move it for the time being

The following is the processing on Jupyter Notebook.

Anaconda, python 3.7.3 is used.

In the code that generates the nth of the Fibonacci sequence, let's compare the speed with python, numba, cython, and typed cython.

First of all, ordinary python

def python_fib(n):
    a, b = 0., 1.
    for i in range(n):
        a, b = a + b, a
    return a

Let's measure the time required to calculate the 1000th number in the Fibonacci sequence with the timeit module.

%timeit python_fib(1000)
10000 loops, best of 3: 33.6 µs per loop

It seems that it took 33.6 microseconds on average.

In this case, the command% timeit performs the following processing.

--Execute python_fib (1000) 10000 times and save the total execution time. --Execute python_fib (1000) 10000 times and save the total execution time. --Execute python_fib (1000) 10000 times and save the total execution time. --Obtain the shortest execution time of 3 executions, divide it by 10000, and output it as the shortest execution time of 10000.

The number of loops (10000 loops in this case) is automatically adjusted by the function of % timeit, and changes according to the execution speed of the code to be measured.

When compiling with Numba

numba can be installed with pip.

pip install numba

Using numba is very easy, just add a @jit decorator.

from numba import jit

@jit
def numba_fib(n):
    a, b = 0.0, 1.0
    for i in range(n):
        a, b = a + b, a
    return a

Measure the processing time.

%timeit numba_fib(1000)
The slowest run took 77684.24 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 1.26 µs per loop

1.26 microseconds! It is 26 times faster.

When using Cython

In jupyter, cython can be loaded with the following magic command.

%load_ext Cython

Compiling and optimizing by putting %% cython first.

%%cython
def cython_fib(n):
    a, b = 0.0, 1.0
    for i in range(n):
        a, b = a + b, a
    return a
%timeit cython_fib(1000)
100000 loops, best of 3: 8.22 µs per loop

8.22 microseconds. It's about four times faster without doing anything.

Finally, when compiling with cython after specifying the type.

%%cython
def cython_fib2(int n):
    a, b = 0., 1.
    for i in range(n):
        a, b = a + b, a
    return a
%timeit cython_fib2(1000)
The slowest run took 7.27 times longer than the fastest. This could mean that an intermediate result is being cached.
1000000 loops, best of 3: 984 ns per loop

0.984 microseconds! 34 times faster.

Finally

If it is a simple process like the above, it is easy to move it with numba or cython, but if it is a complicated process, there will be cases where it cannot be handled.

If you want speed, you may want to switch to another language as soon as possible, or consider multi-process or hardware selection.

Recommended Posts

Let's move Cython and Numba easily
I can't move to Python3 easily, so let's consider compatibility 2-3
Let's move word2vec with Chainer and see the learning progress
Let's use python janome easily
Let's use python's wordcloud easily!