[Ruby] How to decide the parallel number of parallel

less than 1 minute read

I made a note because I investigated how to determine the parallel number of parallel. The number of parallels is determined by evaluating from 1 below.

1. Number of processor cores in the system

The number of processor cores returned by Etc.nprocessors. In a 1-CPU system, even if you use parallel and you do not specify any of the items below, the parallel number will be 1. On the other hand, when using a multi-core Xoen, etc. on the server machine, the number of parallel connections may become too large and put pressure on the CPU and memory.

2. Environment variable PARALLEL_PROCESSOR_COUNT

If the environment variable PARALLEL_PROCESSOR_COUNT is set, its value is adopted. Since it does not need to be specified each time parallel is called, is it convenient when the same number of parallels is always required?

3. Parallel number specified in in_processes/in_threads

You can specify the parallel number each time with the in_processes/in_threads option of the map method. Do you have the most cases to specify here?

4. number of sources of map/each method

Even with Parallel.map([1, 2, 3], in_process: 10)..., the parallel number is only 3.