Don't write Python if you want to speed it up with Python

Introduction

I'm going to have a lightning talk about Python at work, so I'll write something that can be read in about 5 minutes. The talk theme is "Knowledge for trying to speed up with Python". ** This is not an article that dissipates Python. ** **

Who is the target of this article

This article is intended for:

--Python beginners --People who are interested in Python but think that it is not speedy

Immediate conclusion

As the title says, if you want to speed up with Python, don't write Python (as much as possible). The purpose of this article is to reduce the amount of Python written and increase the processing speed.

Python is slow in the first place

Now, immediately, ** Python is a slow language ** compared to C ++, Java, etc. The slowness of Python is mentioned in various articles such as:

-Why is Python so slow? | POSTD -Is the Python for statement slow? --atsuoishimoto's diary -Speed comparison of Python, Java, C ++ --Qiita

In summary, Python is considered slow, mainly for the following reasons:

--Because it is an interpreted (non-compiled) language --Because it is a dynamically typed language

However, commonly considered heavy tasks such as machine learning and image processing are actively performed using Python. Why are these processes running at a speed that is practical? That's because ** most of the processing isn't written using Python **.

What do you mean

In other words, if you want to speed up Python, you can leave the heavy processing to standard functions and libraries written in C ++ instead of doing it in Python. See the two functions below. fastCode () is a function that can be executed fast. slowCode () is a slow function.

import numpy as np

#Fast code
def fastCode():
    #List is declared in list notation
    list1 = [ i for i in range(0,10000)]
    list2 = [ i for i in range(-10000,0)]

    #Use standard functions whenever possible
    listSum = sum(list1)

    #Use numpy to make a large amount of calculations into a determinant
    array1 = np.array(list1)
    array2 = np.array(list2)
    #Add all the elements of array1 and array2
    # [0, 1, 2, ...] + [-100000, -99999, -99998, ...]
    # >> [-100000, -99998, -99996, ...]
    array3 = array1 + array2
    #Multiply all elements of array1 and array2
    # [0, 1, 2, ...] * [-100000, -99999, -99998, ...]
    # >> [0, -99999, -199996, ...]
    array4 = array1 * array2

#Slow code
def slowCode():
    #Create a list using append
    list1 = []
    for i in range(0,10000):
        list1.append(i)
    list2 = []
    for i in range(-10000,0):
        list2.append(i)

    #Calculate the total by turning with a for statement without using standard functions
    listSum = 0
    for value in list1:
        listSum += value

    #Do a lot of calculations with a for statement
    list3 = []
    list4 = []
    for i , value in enumerate(list1):
        list3.append(value + list2[i])
        list4.append(value * list2[i])

You can see that fastCode () entrusts the processing to standard functions and libraries and can be coded short, while slowCode () describes the processing by itself using the for statement. I think you can. The actual processing speeds are as follows.

Standard functions and libraries are composed of ** compiled binaries ** written in C / C ++, and by letting them handle the processing, even heavy tasks can be processed at high speed. From a "speedup" perspective, Python is, so to speak, a ** language for easy use of fast libraries **.

Summary

If you want to speed up the process in Python, use the following techniques to minimize the process written in Python.

--Use standard functions and libraries written in high-speed C / C ++ --A large amount of calculations are put into a determinant and processed at once with numpy --If you can't help, create your own library using C / C ++

Finally

There are many ways to speed up Python other than those mentioned here. However, I think this is the first thing to be aware of when accelerating with Python. When you need to find more speed, let's start with your own library, function pre-reference, Cython, Numba, etc.

reference

Why is Python so slow? | POSTD Is the Python for statement slow? --atsuoishimoto's diary Python, Java, C ++ speed comparison-Qiita [Introduction to Python C API] Create an extension module in C / C ++ and call it from Python -Part 1- Python Tips for Acceleration-A Leisure Engineer's Diary