Skip to content

With Python, to simulate a genuine parallel-processing (making use of all the cores) #3

Answered by 0xmicrosoft15
mnadamod asked this question in Q&A
Discussion options

You must be logged in to vote

We have a couple of options for parallel processing in Python:

  • multiprocessing module (built-in, best for CPU-bound tasks)
  • concurrent.futures.ProcessPoolExecutor (higher-level interface)
  • joblib library (great for simple parallel loops)
  • dask library (for larger-than-memory computations)

An example of the reliable way to utilize all cores for CPU-bound tasks:
`
import multiprocessing
import time

def cpu_intensive_task(n):
"""Example CPU-bound task (calculates nth Fibonacci number)"""
if n <= 1:
return n
else:
return cpu_intensive_task(n-1) + cpu_intensive_task(n-2)

if name == 'main':
# Get number of available CPU cores
num_cores = multiprocessing.cpu_count()
print(f"Using {num_cores} CPU …

Replies: 1 comment 1 reply

Comment options

You must be logged in to vote
1 reply
@mnadamod
Comment options

Answer selected by mnadamod
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
2 participants