Posts

Showing posts with the label pool

ThreadPool and Pool for parallel processing

Image
Clash Royale CLAN TAG #URR8PPP ThreadPool and Pool for parallel processing Is there a way to use both ThreadPool and Pool in python to parallelise a loop by specifying the number of CPUs and cores you wish to use? For example I would have a loop execute as: from multiprocessing.dummy import Pool as ThreadPool from tqdm import tqdm import numpy as np def my_function(x): return x + 1 pool = ThreadPool(4) my_array = np.arange(0,1e6,1) results = list(tqdm(pool.imap(my_function, my_array),total=len(my_array))) For 4 cores (threads) but it I wanted to spread these out on multiple CPUs as well, is there a simple way to adapt the code? 1 Answer 1 You can just use it with no parameter and let the library decide: from multiprocessing.pool import ThreadPool ... pool = ThreadPool() But that will not share the process across multiple CPUs no? Only cores – user...