Differences between revisions 1 and 5 (spanning 4 versions)
⇤ ← Revision 1 as of 2020-04-07 21:42:37 →
Size: 452
Editor: SamatJain
Comment:
← Revision 5 as of 2020-06-23 07:58:26 → ⇥
Size: 2746
Editor: SamatJain
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
== Notes ==

 * `asyncio.run()` gets the event loop, runs tasks until they're complete, and closes the event loop. Introduced in Python 3.7 and replaces `asyncio.get_event_loop()` and `loop.run_until_complete()`. E.g.

{{{#!highlight python numbers=off
asyncio.run(main())
}}}

rather than:

{{{#!highlight python numbers=off
loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
finally:
    loop.close()
}}}

 * CPU-intensive things should not be in await tasks. Put them into ThreadPoolExecutor, e.g.

{{{#!highlight python
import asyncio
import concurrent.futures
executor = concurrent.futures.ThreadPoolExecutor()

item = ... # do something CPU-intensive against this
cpu_intensive_thing = ... # some CPU-intensive function


async def cpu_intensive_coroutine(item):
    loop = async.get_event_loop()
    return (await cpu_intensive_thing(item))

result = asyncio.run(cpu_intensive_coroutine(item))
}}}

 * use `async.create_task` to turn a coroutine into a task. Tasks can be given to gather or as_comepleted
 * `asyncio.gather` will schedule multiple async tasks at once. Waits for all tasks to be completed, returns list of results.
 ( `asyncio.as_completed` can be looped over as tasks are completed.

{{{#!highlight python
tasks = []
tasks.append(asyncio.create_task(…))
tasks.append(asyncio.create_task(…))
# …
for completed_task in asyncio.as_completed(*tasks):
    earliest_task_results = await completed_task
    # …
}}}

== Helpers ==

 * `asynctosync` and `synctoasync` helpers: https://github.com/django/asgiref/blob/master/asgiref/sync.py, usage: https://www.aeracode.org/2018/02/19/python-async-simplified/. Also: [[https://pypi.org/project/syncasync/|pip package: syncasync]]

== Links ==

 [[http://www.dabeaz.com/coroutines/Coroutines.pdf|A Curious Course on Coroutines and Concurrency]]

 * [[https://redislabs.com/blog/async-await-programming-basics-python-examples/|Async/Await Programming Basics with Python Examples for Redis]]: examples of `async.gather()` w/ Redis API

 * [[https://realpython.com/async-io-python/|AsyncIO in Python: A Complete Walkthrough]]: great overview
 * [[https://www.pythonsheets.com/notes/python-asyncio.html|Asyncio PySheeet]]: reference snippets

Notes

  • asyncio.run() gets the event loop, runs tasks until they're complete, and closes the event loop. Introduced in Python 3.7 and replaces asyncio.get_event_loop() and loop.run_until_complete(). E.g. 

asyncio.run(main())

rather than: 

loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
finally:
    loop.close()

  • CPU-intensive things should not be in await tasks. Put them into ThreadPoolExecutor, e.g. 

   1 import asyncio
   2 import concurrent.futures
   3 executor = concurrent.futures.ThreadPoolExecutor()
   4 
   5 item = ... # do something CPU-intensive against this
   6 cpu_intensive_thing = ... # some CPU-intensive function
   7 
   8 
   9 async def cpu_intensive_coroutine(item):
  10     loop = async.get_event_loop()
  11     return (await cpu_intensive_thing(item))
  12 
  13 result = asyncio.run(cpu_intensive_coroutine(item))

  • use async.create_task to turn a coroutine into a task. Tasks can be given to gather or as_comepleted

  • asyncio.gather will schedule multiple async tasks at once. Waits for all tasks to be completed, returns list of results. ( asyncio.as_completed can be looped over as tasks are completed. 

   1 tasks = []
   2 tasks.append(asyncio.create_task(…))
   3 tasks.append(asyncio.create_task(…))
   4 # …
   5 for completed_task in asyncio.as_completed(*tasks):
   6     earliest_task_results = await completed_task
   7     # …

Helpers

SamatsWiki: ProgrammingLanguages/Python/asyncio (last edited 2020-06-28 04:16:00 by SamatJain)