I measured the speed of list comprehension, for and while with python2.7.

at first

Which is faster, notation in the list, for and while? I was wondering, so I wrote various codes and measured them.

Execution environment

python2.7 windows7 Intel Core i5 CPU 2.4GHz Memory 4.0 GB

Measurement target

for single loop (1 million times) for double loop (1000 times x 1000 times) List comprehension (1 million times) List comprehension (1000 times x 1000 times) while single loop (1 million times) while double loop (1000 times x 1000 times)

Conditions (see the code at the end for details)

When creating a list In case of + for (with or without range)

result

List comprehension was the fastest if you just made a list. When an if statement was included, it got a little worse overall, and the for and list comprehension notation did not change much. While was clearly slowed down in a double loop. I'm not sure why. When it comes to total value calculation, list comprehension is slower than for. This seems to mean that it takes time to sum one million pieces. The result of total value calculation + if is the same regardless of whether it is a list comprehension or for. determine.png

Conclusion

You can write the list comprehension notation for the time being, but if the process does not make use of the created list (such as simply taking sum), the cost for processing the list with other functions (sum () etc.) will increase. And can be slow. While should be careful about if of multiple loops.

Experimental codes

seq1 = range(10**6)
seq2 = range(10**3)
num1 = 10**6
num2 = 10**3

v = 10**10
def for1():
  ls = []
  for i in seq1:
    ls.append(v)
  #print len(ls)

def for2():
  ls = []

  for i in seq2:
    for j in seq2:
      ls.append(v)
  #print len(ls)

def for_range1():
  ls = []
  for i in range(10**6):
    ls.append(v)
  #print len(ls)

def for_range2():
  ls = []
  for i in range(10**3):
    for j in range(10**3):
      ls.append(v)
  #print len(ls)
      
def for_listnai1():
  [v for i in seq1]

def for_listnai2():
  [v for i in seq2 for j in seq2]

def for_listnai_range1():
  [v for i in range(10**6)]

def for_listnai_range2():
  [v for i in range(10**3) for j in range(10**3)]

def forsum1():
  s = 0
  for i in seq1:
    s += v

def forsum2():
  s = 0
  for i in seq2:
    for j in seq2:
      s += v

def for_listnaisum1():
  sum([v for i in seq1])

def for_listnaisum2():
  sum([v for i in seq2 for j in seq2])

def while1():
  i = 0
  ls = []
  while i < num1:
    ls.append(v)
    i+=1
  
def while2():
  i = 0
  ls = []
  while i < num2:
    j = 0
    while j < num2:
      ls.append(v)
      j+=1
    i+=1

def whilesum1():
  i = 0
  s = 0
  while i < num1:
    s += v
    i+=1
  
def whilesum2():
  i = 0
  s = 0
  
  while i < num2:
    j = 0
    while j < num2:
      s += v
      j+=1
    i+=1
    
def forif1():
  ls = []
  for i in seq1:
    if v % 2:
      ls.append(v)

def forif2():
  ls = []
  for i in seq2:
    for j in seq2:
      if v % 2:
        ls.append(v)

def forlistnaiif1():
  [v for i in seq1 if v % 2]

def forlistnaiif2():
  [v for i in seq2 for j in seq2 if v % 2]

def whileif1():
  ls = []
  i=0
  while i <num1:
    ls.append(v)
    i+=1

def whileif2():
  ls = []
  i=0
  while i <num2:
    j=0
    while j<num2 :
      if v % 2:
        ls.append(v)
      j+=1
    i+=1
    
def forifsum1():
  s = 0
  for i in seq1:
    if v % 2:
      s+=v

def forifsum2():
  s = 0
  for i in seq2:
    for j in seq2:
      if v % 2:
        s+=v

def forlistnaiifsum1():
  sum([v for i in seq1 if v % 2])

def forlistnaiifsum2():
  sum([v for i in seq2 for j in seq2 if v % 2])

def whileifsum1():
  s = 0
  i=0
  while i <num1:
    s+=v
    i+=1

def whileifsum2():
  s=0
  i=0
  while i <num2:
    j=0
    while j<num2 :
      if v % 2:
        s+=v
      j+=1
    i+=1

Time measurement code


def exe(func,num=0):
    import time
    print "%s start:" % func
    s = 0
    for i in range(0,num): 
      start = time.time()
      exec func
      end = time.time()
      s += end - start
    print "%s finished: %d times." % (func,num)
    print s

if __name__=='__main__':
  exe("for1()",1)
  exe("for2()",1)
  exe("for_range1()",1)
  exe("for_range2()",1)
  exe("for_listnai1()",1)
  exe("for_listnai2()",1)
  exe("for_listnai_range1()",1)
  exe("for_listnai_range2()",1)
  exe("while1()",1)
  exe("while2()",1)
  
  exe("forsum1()",1)
  exe("forsum2()",1)
  exe("for_listnaisum1()",1)
  exe("for_listnaisum2()",1)
  exe("whilesum1()",1)
  exe("whilesum2()",1)

  exe("forif1()",1)
  exe("forif2()",1)
  exe("forlistnaiif1()",1)
  exe("forlistnaiif2()",1)
  exe("whileif1()",1)
  exe("whileif2()",1)

  exe("forifsum1()",1)
  exe("forifsum2()",1)
  exe("forlistnaiifsum1()",1)
  exe("forlistnaiifsum2()",1)
  exe("whileifsum1()",1)
  exe("whileifsum2()",1)

Recommended Posts

I measured the speed of list comprehension, for and while with python2.7.
I compared the speed of Hash with Topaz, Ruby and Python
I replaced the numerical calculation of Python with Rust and compared the speed
I tried to compare the processing speed with dplyr of R and pandas of Python
Python list comprehension speed
Compare the speed of Python append and map
I compared the speed of go language web framework echo and python web framework flask
I compared the speed of regular expressions in Ruby, Python, and Perl (2013 version)
List of Python libraries for data scientists and data engineers
Python netCDF4 read speed and nesting of for statements
Visualize the range of interpolation and extrapolation with python
I checked out the versions of Blender and Python
I measured the performance of 1 million documents with mongoDB
I checked the reference speed when using python list, dictionary, and set type in.
I tried to automate the article update of Livedoor blog with Python and selenium.
I tried to find the entropy of the image with python
I tried "gamma correction" of the image with Python + OpenCV
I was hooked for 2 minutes with the Python debugger pdb
I wrote the basic grammar of Python with Jupyter Lab
I evaluated the strategy of stock system trading with Python.
I want to know the features of Python and pip
I implemented N-Queen in various languages and measured the speed
Play with the password mechanism of GitHub Webhook and Python
I tried to get the number of days of the month holidays (Saturdays, Sundays, and holidays) with python
I liked the tweet with python. ..
I played with PyQt5 and Python3
Coexistence of Python2 and 3 with CircleCI (1.0)
About the basics list of Python basics
I tried scraping the ranking of Qiita Advent Calendar with Python
Speed comparison of Wiktionary full text processing with F # and Python
[Introduction to Python] I compared the naming conventions of C # and Python.
I want to output the beginning of the next month with Python
I tried to create a list of prime numbers with python
[Python] I thoroughly explained the theory and implementation of logistic regression
The story of making a standard driver for db with python.
[Python] I thoroughly explained the theory and implementation of decision trees
python note: map -do the same for each element of the list
I tried to improve the efficiency of daily work with Python
The latest NGINX is an application server! ?? I measured the benchmark of NGINX Unit with PHP, Python, Go! !!
python memo: enumerate () -get index and element of list at the same time and turn for statement
Check the existence of the file with python
Python comprehension (list and generator expressions) [additional]
I didn't know the basics of Python
I installed and used Numba with Python3.5
The third night of the loop with for
Pandas of the beginner, by the beginner, for the beginner [Python]
The Python project template I think of.
The second night of the loop with for
Calculate the shortest route of a graph with Dijkstra's algorithm and Python
I read the Sudachi synonym dictionary with Pandas and searched for synonyms
Python> sys.path> List of strings indicating the path to search for modules
[Introduction to Python] How to sort the contents of a list efficiently with list sort
I measured 6 methods to get the index of the maximum value (minimum value) of the list
[Python of Hikari-] Chapter 05-09 Control syntax (use of for statement and while statement properly)
I set the environment variable with Docker and displayed it in Python
I tried to get the authentication code of Qiita API with Python.
I vectorized the chord of the song with word2vec and visualized it with t-SNE
Effective Python memo Item 7 Use list comprehension instead of map and filter
I have 0 years of programming experience and challenge data processing with python
Find the general terms of the Tribonacci sequence with linear algebra and Python
Receive a list of the results of parallel processing in Python with starmap