Home

Thread Pool Usage

Resources

  1. Explanation behind chunksize

Example

""" processes=os.cpu_count() enables us to to utilise all processing cores. chunksize=1 tells us to pass one element on each execution. If you roughly know the time expecation of each function execution, you can increase this to chunk the args into bigger pieces for one process to tackle. """ from multiprocessing import Pool def square(number): return number * number data = [1,2,3] chunksize = 1 proc_count = os.cpu_count() with Pool(processes=proc_count) as pool: result = pool.map(square, data, chunksize) # result returns [1, 4, 9]

Repository

https://github.com/okeeffed/developer-notes-nextjs/content/python/thread-pool-usage

Sections


Related