Multi tool use

Clash Royale CLAN TAG

ThreadPool and Pool for parallel processing

Is there a way to use both ThreadPool and Pool in python to parallelise a loop by specifying the number of CPUs and cores you wish to use?

For example I would have a loop execute as:

from multiprocessing.dummy import Pool as ThreadPool
from tqdm import tqdm
import numpy as np

def my_function(x):
return x + 1

pool = ThreadPool(4)
my_array = np.arange(0,1e6,1)


results = list(tqdm(pool.imap(my_function, my_array),total=len(my_array)))


For 4 cores (threads) but it I wanted to spread these out on multiple CPUs as well, is there a simple way to adapt the code?

1 Answer
1

You can just use it with no parameter and let the library decide:

from multiprocessing.pool import ThreadPool
...
pool = ThreadPool()


By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.