SamatsWiki

• Diff for "ProgrammingLanguages/Python/asyncio"

Notes

• asyncio.run() gets the event loop, runs tasks until they're complete, and closes the event loop. Introduced in Python 3.7 and replaces asyncio.get_event_loop() and loop.run_until_complete(). E.g.

asyncio.run(main())

rather than:

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
finally:
    loop.close()

• CPU-intensive things should not be in async tasks. Put them into ProcessPoolExecutor, e.g.

   1 import asyncio
   2 import concurrent.futures
   3 executor = concurrent.futures.ProcessPoolExecutor()
   4 
   5 item = ... # do something CPU-intensive against this
   6 cpu_intensive_thing = ... # some CPU-intensive function
   7 
   8 
   9 async def cpu_intensive_coroutine(item):
  10     loop = async.get_event_loop()
  11     result = await loop.run_in_executor(executor, cpu_intensive_thing(item))
  12     return result
  13 
  14 result = asyncio.run(cpu_intensive_coroutine(item))

• Python 3.9 has async.to_thread() which is a higher level of the above but into a ThreadPoolExecutor. It's only useful for IO-bound tasks.

• use async.create_task to turn a coroutine into a task. Tasks can be given to gather or as_comepleted

• asyncio.gather will schedule multiple async tasks at once. Waits for all tasks to be completed, returns list of results. ( asyncio.as_completed can be looped over as tasks are completed.

   1 tasks = []
   2 tasks.append(asyncio.create_task(…))
   3 tasks.append(asyncio.create_task(…))
   4 # …
   5 for completed_task in asyncio.as_completed(*tasks):
   6     earliest_task_results = await completed_task
   7     # …

Helpers

• asynctosync and synctoasync helpers: https://github.com/django/asgiref/blob/master/asgiref/sync.py, usage: https://www.aeracode.org/2018/02/19/python-async-simplified/. Also: pip package: syncasync

Links

• A Curious Course on Coroutines and Concurrency

• Async/Await Programming Basics with Python Examples for Redis: examples of async.gather() w/ Redis API

• AsyncIO in Python: A Complete Walkthrough: great overview

• Asyncio PySheeet: reference snippets

• https://fredrikaverpil.github.io/2017/06/20/async-and-await-with-subprocesses/

• https://docs.python.org/3/library/asyncio-task.html#asyncio.gather

• https://stackoverflow.com/questions/48483348/how-to-limit-concurrency-with-python-asyncio

• https://stackoverflow.com/questions/46031558/how-to-limit-the-number-of-concurrent-processes-using-subprocess-module-in-async

• https://docs.python.org/3/library/asyncio-sync.html#asyncio.Semaphore