Vous pouvez mettre en œuvre un décorateur pour faire de vos fonctions asynchrones, même si c'est un peu délicat. L' multiprocessing
module est plein de bizarreries et apparemment arbitraire des restrictions – une raison de plus pour encapsuler derrière une interface conviviale.
from inspect import getmodule
from multiprocessing import Pool
def async(decorated):
r'''Wraps a top-level function around an asynchronous dispatcher.
when the decorated function is called, a task is submitted to a
process pool, and a future object is returned, providing access to an
eventual return value.
The future object has a blocking get() method to access the task
result: it will return immediately if the job is already done, or block
until it completes.
This decorator won't work on methods, due to limitations in Python's
pickling machinery (in principle methods could be made pickleable, but
good luck on that).
'''
# Keeps the original function visible from the module global namespace,
# under a name consistent to its __name__ attribute. This is necessary for
# the multiprocessing pickling machinery to work properly.
module = getmodule(decorated)
decorated.__name__ += '_original'
setattr(module, decorated.__name__, decorated)
def send(*args, **opts):
return async.pool.apply_async(decorated, args, opts)
return send
Le code ci-dessous illustre l'utilisation de la décoratrice:
@async
def printsum(uid, values):
summed = 0
for value in values:
summed += value
print("Worker %i: sum value is %i" % (uid, summed))
return (uid, summed)
if __name__ == '__main__':
from random import sample
# The process pool must be created inside __main__.
async.pool = Pool(4)
p = range(0, 1000)
results = []
for i in range(4):
result = printsum(i, sample(p, 100))
results.append(result)
for result in results:
print("Worker %i: sum value is %i" % result.get())
Dans un cas, je ellaborate un peu plus sur le décorateur, de la fourniture de certains moyen de le désactiver pour le débogage (tout en gardant l'avenir de l'interface en place), ou peut-être une installation de traitement des exceptions; mais je pense que cela démontre le principe assez bien.