sample.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from multiprocessing import Pool
from collections import Counter
#Parallel processing here
def func(n, argument1, argument2):
#The process of doubling and adding 5
return n * argument1 + argument2
def wrapper(args):
#args(i, 2, 5)Has become
return func(*args)
def multi_process(sampleList):
#Number of processes:8(Parallel processing with 8 cpu)
p = Pool(8)
output = p.map(wrapper, sampleList)
#End of process
p.close()
return output
if __name__ == "__main__":
#Perform 100 processes in parallel
num = 100
# (i, 2, 5)Is the argument
sampleList = [(i, 2, 5) for i in range(num)]
#Double the elements of sampleList and add 5
output = multi_process(sampleList)
The python version is 2.7.10. You can see it by looking at the code
-When parallel processing using ** multiprocessing , it is easy to pass it as a list.
- wrapper () ** You can pass multiple arguments by putting a function in between.
――In this sample, all the first elements of the list are doubled and 5 is added.
--If you forget to close the process (** Pool.close () **), you may open the process too much and get the error Too many open files.
Recommended Posts