Asyncio Tutorial

@asvetlov

BPO	34831
Nosy	@asvetlov, @cjrh, @elprans, @1st1, @willingc
PRs	bpo-34831: Asyncio tutorial #9748

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2018-09-28.11:36:17.041>
labels = ['3.8', 'type-feature', '3.7', 'docs', 'expert-asyncio']
title = 'Asyncio Tutorial'
updated_at = <Date 2019-06-19.12:39:06.198>
user = 'https://github.com/cjrh'

bugs.python.org fields:

activity = <Date 2019-06-19.12:39:06.198>
actor = 'cjrh'
assignee = 'docs@python'
closed = False
closed_date = None
closer = None
components = ['Documentation', 'asyncio']
creation = <Date 2018-09-28.11:36:17.041>
creator = 'cjrh'
dependencies = []
files = []
hgrepos = []
issue_num = 34831
keywords = ['patch']
message_count = 14.0
messages = ['326628', '326665', '326789', '326977', '327266', '327271', '327275', '327284', '327286', '328201', '333088', '345661', '345755', '346039']
nosy_count = 6.0
nosy_names = ['asvetlov', 'cjrh', 'Elvis.Pranskevichus', 'docs@python', 'yselivanov', 'willingc']
pr_nums = ['9748']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue34831'
versions = ['Python 3.7', 'Python 3.8']

Linked PRs

Hi Yury,

As discussed, below is a very rough outline of a proposed TOC for an asyncio tutorial. No content has been written yet (only what you see below). I think we should nail down the TOC first.

Asyncio Tutorial
================

Proposed Table of Contents:

Why asyncio?
- Reason Support "bpo-" in Misc/NEWS #1: thread safety by not using threads at all.
- Reason Rename README to README.rst and enhance formatting #2: very many concurrent socket connections, which threads
  make cumbersome.
- Demo: concurrency without threads
  - (only goals here are to *briefly* introduce the syntax, and then
    demonstrate concurrency without threads)
  - show two tasks, one prints out integers 0-9, while the other
    prints out letters of the alphabet, A-I.
  - pose the question: "they're running concurrently but we're
    not using threads: how is that possible?"
  - explain that there is a thing called an "event loop" behind
    the scenes that allows execution to switch between the two
    every time one of them "waits".
  - emphasize that you can easily spot where the switch can happen,
    it is where you see the "await" keyword. Switches will occur
    nowhere else.
The difference between functions and async def functions
- async & await
  - show async def functions, compare to def functions.
  - use inspect module to show what things actually are, e.g. function,
    coroutine function, coroutine, generator function, generator,
    asynchronous generator function, asynchronous generator.
  - point out you can only use "await" inside an async def fn
  - point out that await <x> is an expression, and you can
    use it in most places any other expression can be used.
How to run async def functions
- point out there are two different issues: (a) async def functions
  can call other functions, and other async def functions using
  await, but also, (b) how to "get started" with the first
  async def function? Answer: run the event loop.
- show asyncio.run(), in particular running an async def main()
  function, which itself can call others.
Dealing with concurrent functions
- (This is maybe too similar to the new docs in the section
  https://docs.python.org/3/library/asyncio-task.html?highlight=asyncio%20run#coroutines-and-tasks)
- What if you want to run an async def function, but you don't
  want to wait for it? Answer: create a "task" for it.
- What if you want to run multiple functions concurrently, and you
  do want to wait for all of them to finish? Answer: use gather
  or wait()
Case Study: chat server/client (my proposal)
- (goal is to walk through building a realistic example of using
  asyncio)
- (This will be a lot more fun than a web-crawler. Web-crawlers are
  super boring!)
- (I'm pretty confident the size of the code can be kept small. A
  lot can be done in under 50 lines, as I showed in the case studies
  in my book)
- server uses streams API
- server receives many long-lived connections
- user can create/join a "room", and then start typing messages.
  Other connected clients in the same room will see the messages.
- client implementation has some options:
  - could use Tkinter gui, using streams API in an event loop
    on a separate thread. (This would show how asyncio isn't some
    alien thing, but is part of python. Would also show how to
    set up asyncio to work in a separate thread. Finally, would not
    require any external dependencies, only the stdlib is needed
    for the entire case study.)
  - could use a browser client, connecting back to the server
    over a websocket connection. (This might seem simpler, but
    in fact introduces a lot more complexity than just using
    tkinter. We need to bring in html, css, probably some js and
    probably also a third-party websockets python library. I feel
    like it isn't a good fit for a stdlib python tutorial, but it
    is a more modern approach.)
  - there are not a lot of other options. terminal-based client
    might be possible, but probably hard, not cross-platform, and
    will be unappealing to many people.
- When describing the code, point out:
  - you have to choose a "message protocol" (size-prefixed is fine)
  - you must put send and recv in separate tasks
  - server will "keep track" of connected clients and the room
    (or rooms?) they've joined
  - startup and shutdown, specific references to the new run()
    function
  - ?

I like this, great job!

Couple of thoughts on how we should organize this:

I think we should stick to your structure and push things to docs.python.org as soon as every next section is somewhat ready.
Every big section should probably have its own page, linking prev/next tutorial pages.
I'd organize the tutorial in a dedicated directory like "Doc/library/asyncio-tutorial/". Context: I wanted to reorganize all current "Doc/library/asyncio*.rst" files under one dir too, but decided to keep the existing file structure to avoid breaking links to the docs from SO/google). We shouldn't repeat that mistake again.

BTW, maybe we should consider using the new iPythonn async repl: https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7 What do you think about that?

Some comments on the proposed TOC:

Reason Support "bpo-" in Misc/NEWS #1: thread safety by not using threads at all.

Reason Rename README to README.rst and enhance formatting #2: very many concurrent socket connections, which threads
make cumbersome.

This section is super important to get right, as I see people ask the "why and where should I use asyncio" _all_ _the_ _time_. So I'll expand on this section in a detail.

Reason #1 and #2 are correct, but I think we should really focus on making #1 comprehensive and clear:

async/await makes all context switches visible; that makes it easy to spot race conditions and reason about your code (think about ORMs that can make 100s of queries to render one web page); you've likely seen https://glyph.twistedmatrix.com/2014/02/unyielding.html
in general, all datastructures are safe for async (we cannot say same for threads);
an async/await library means that it's safe to use it in concurrent async/await code (you can never be sure if some library is thread-safe, even if it claims that);
language constructs like 'async for' and 'async with' enable structured concurrency;

And on #2:

high-throughput IO or 1000s of long-living connections are only doable with asyncio
if you don't need to scale your code right now but might need in near future investing in async/await is wise

How to run async def functions

point out there are two different issues: (a) async def functions
can call other functions, and other async def functions using
await, but also, (b) how to "get started" with the first
async def function? Answer: run the event loop.

Just a quick note: I'd try to not mention the low-level loop APIs as long as possible (e.g. no loop.run_until_complete() etc).

Dealing with concurrent functions
- (This is maybe too similar to the new docs in the section
  https://docs.python.org/3/library/asyncio-task.html?highlight=asyncio%20run#coroutines-and-tasks)

Right, we'll need to update the asyncio-task.rst file. I think we'll collapse first two section into one ("Coroutines" and "Awaitables" into "Awaitables") and link the tutorial from that new section.

Case Study: chat server/client (my proposal)
[..]

server uses streams API

Yay for streams!

[..]
- client implementation has some options:

I never use tkinter myself :( I remember trying to use it and it didn't work on my macOS. So I'd try to either:

build a simple browser app (that would require us to implement HTTP 0.9 which can be fun);
build a terminal app;
use iPython repl to connect to our asyncio server (might end up being more complicated than the first two options).

I think we should stick to your structure and push things to docs.python.org as soon as every next section is somewhat ready.

Ok. I'll get a PR going for the start page of the tutorial.

Every big section should probably have its own page, linking prev/next tutorial pages.

I'd organize the tutorial in a dedicated directory like "Doc/library/asyncio-tutorial/".

Agree.

BTW, maybe we should consider using the new iPythonn async repl: https://blog.jupyter.org/ipython-7-0-async-repl-a35ce050f7f7 What do you think about that?

I saw Matthias' tweets about that recently too. It's cool! but...for teaching purposes it's not great to introduce a whole new complex tool (ipython) to explain a different complex tool (asyncio). My experience is that *every* single new thing that is mentioned adds cognitive load for learners. For this tutorial my feeling is to keep as much to "ordinary" Python stuff as possible, i.e., stdlib.

Just a quick note: I'd try to not mention the low-level loop APIs as long as possible (e.g. no loop.run_until_complete() etc).

For sure, I agree with you 100% on this. But I find it hard to do as soon as I have to make a real thing. I think you're right that we focus initially on only high-level stuff first (and for most of the tut). That is doable.

I think we'll collapse first two section into one ("Coroutines" and "Awaitables" into "Awaitables") and link the tutorial from that new section.

ok

Yay for streams!
I never use tkinter myself :( I remember trying to use it and it didn't work on my macOS. So I'd try to either:

build a simple browser app (that would require us to implement HTTP 0.9 which can be fun);

build a terminal app;

use iPython repl to connect to our asyncio server (might end up being more complicated than the first two options).

I too have bashed my head for many hours over the years trying to get Tkinter to work on Mac, but a lot of work has gone into this recently and the newer (release) Python's have bundled Tk 8.6: https://www.python.org/download/mac/tcltk/ (this is what learners will prob use on Mac)

Tkinter gets a bad rap, but it's really quite powerful--and portable. Runs great on a Raspberry Pi for example.

Noticing your hesitation towards tkinter ;) , I spent a few hours on Sunday sketching out my "chat server/client" idea a little more, using Tkinter for the client UI:

https://github.com/cjrh/chat

(Notice especially in the README all the different aspects of asyncio, streams etc. that we would be able to cover and explain with an actual use-case. THESE are the kinds of tricky things people desperately want help with.)

It's still rough obviously (I can prob reduce the total LOC footprint by 20% & I'm sure you can improve on some parts) but I just wanted to show you something runnable you can prod and poke to give a concrete idea of what I'm suggesting. It works on Windows, should work on Linux but I haven't tested yet.

My proposal is that we slowly build up towards this, starting with the "hello world" simple case (asyncio.run calling main() which prints out "hello world" or something), and then adding the necessary features, piece by piece, with commentary along the way on what each piece does, and why it is done in a particular way. (I specifically like to write like this: simplistic case first, and then improve incrementally)

Only requires stdlib (so we don't have to explain or link to pip/virtualenv etc. etc.)
shows a wide range of *interoperating* asyncio features in a condensed app
client has a proper GUI, i.e. "looks" like an actual application, not just an ugly CLI thing
client handles reconnection, if the server goes down and comes back later.
using signal handling to trigger shutdown (esp. the server)
signal handling works on Windows (CTRL-C and CTRL-BREAK near-instant controlled shutdown)
server is 100% asyncio (so that situation is covered), but client requires marrying two loops (so this situation is also covered), one for tkinter and one for asyncio. (This is a common problem, not just with UI frameworks but also with game programming frameworks like pygame, pyarcade and so on. Again, this is the kind of problem many people ask for help with.)
thus, an example showing how to run asyncio in a thread. (asyncio.run works great in a thread, nice job!)
an actual SSL example that works (this was surprisingly hard to find, eventually found one at PyMOTW)

I fully realise that this case study implementation might look weird and ugly, and we don't really want to mention threads at all, and we don't want to explicitly refer to the loop, or create a Future instance, etc., but this is the kind of case study that will give people guidance on how to handle these *actual problems* that they are going to come across.

If you have a look and still don't want to go this way, that's ok, I'm happy to go with a different suggestion.

I too have bashed my head for many hours over the years trying to get Tkinter to work on Mac, but a lot of work has gone into this recently and the newer (release) Python's have bundled Tk 8.6: https://www.python.org/download/mac/tcltk/ (this is what learners will prob use on Mac)

I've tried to run it and here's what I have on my system:

    ~/d/t/chat (master) » python3.7 client.py
    Traceback (most recent call last):
      File "client.py", line 7, in <module>
        from tkinter import *
      File "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/tkinter/__init__.py", line 36, in <module>
        import _tkinter # If this fails your Python may not be configured for Tk
    ModuleNotFoundError: No module named '_tkinter'

How about we write the tutorial and implement terminal clients first. Then we can have two branches of the tutorial -- one implementing a Tk client, one implementing a web client?

A CLI client is a necessary step along the way anyway, so that sounds good by me.

You suggested:

I'd organize the tutorial in a dedicated directory like "Doc/library/asyncio-tutorial/"

I had a look at the source tree, there is an existing "howto" directory; do you still prefer your suggestion of using "Doc/library/" over something like "Doc/howto/asyncio.rst"?

I set up a basic structure under "Doc/library/asyncio-tutorial" as suggested, and opened a PR to show you how that looks. When I make
more progress on a section, I'll post an update here.

I tested the Python 3.7.0 release version for Mac, the download called "macOS 64-bit installer" with checksum ae0717a02efea3b0eb34aadc680dc498 on this page:

https://www.python.org/downloads/release/python-370/

I downloaded, installed that on a mac, and the chat client app launched no problem. (I added a screenshot to my github repo readme).

Your error "ModuleNotFoundError: No module named '_tkinter'" suggests that the python you were using is a different one, because in the release download, Tk 8.6.8 is bundled and the _tkinter module was built against it and must exist.

Anyway, I just wanted to check that it does work on mac :)

Yury, based on the file paths, you appear to be running a MacPorts python3.7. Like many other third-party distributors (but unlike the python.org installers), MacPorts separates Tkinter support into a separately-installable component. Try:

port install py37-tkinter

Yury, based on the file paths, you appear to be running a MacPorts python3.7. Like many other third-party distributors (but unlike the python.org installers), MacPorts separates Tkinter support into a separately-installable component. Try:

Thanks, Ned! Yeah, I know why my python37 doesn't have Tk and how to fix it. It's just I don't think the tutorial should require some component that we know is missing when python is installed through certain channels (with default flags/config).

I kept the 3.12, 3.13 and 3.14 labels so that the tutorial is also backported. It doesn't make sense not to backport such tutorial if one is made (but the backports shoud be carefully done since they could be different depending on the available features).

I kept the 3.12, 3.13 and 3.14 labels so that the tutorial is also backported. It doesn't make sense not to backport such tutorial if one is made (but the backports shoud be carefully done since they could be different depending on the available features).

That's great aspirational thinking, though I fear that in practice we won't find volunteers to do the work for the backports, so the realistic choice is either no backport or an unreliable backport.

Maybe the "backports" could just hard-link to the 3.14 tutorial, and the latter could have a light sprinkling of "new in 3.13" and "new in 3.14" notes (but not "new in 3.12" or earlier versions).

Maybe the "backports" could just hard-link to the 3.14 tutorial

That could be a solution but I don't know how to do it without hardcoding the link. Now, since the issue has been opened for a long time, I think that 3.14 would likely be the ~~"main"~~ ¹ stable branch at the point it is written, and then we wouldn't have backports issues. Now, most of the stuff the asyncio tutorial would contain is likely to be compatible for 3.12+ so maybe the backports will be fine?

Nonetheless, I just wanted to keep the labels for myself and so that we can remember to address this when writing the tutorial (by the way, I would probably be fine with helping doing the backports, one possible issue I see is that it could create conflicts for subsequent PRs if the wording changes too much).

Sorry I meant "stable". ↩

Totally fine. We need the tutorial first, the backport can be decided later.

And 3.14 already is main.

@willingc

cc @willingc

The new Asyncio HOWTO has been added in 3964f97, I think that adding a tutorial would be quite duplicative? Can this be closed?

Tutorials and explainers (HOWTO) fall under different sections of the Diátaxis framework. It would still be useful to have a tutorial.

A

Tutorials and explainers (HOWTO) fall under different sections of the Diátaxis framework. It would still be useful to have a tutorial.

Sorry, I disagree. "A Conceptual Overview of asyncio" functions as an asyncio tutorial in its current form. I think it's just badly named a "HOWTO" because there's no final goal of creating anything; it is merely something to help beginners grasp asyncio, which corresponds to the Diátaxis definition of a tutorial.

More generally, I don't think our definition of "HOWTO" fits Diátaxis. For example, the free-threading page, the ipaddress introduction, and the descriptor guide are all marked as "HOWTO" articles, despite them not "guiding the user towards an action", per Diátaxis's definition of HOWTO.

I'm going to close this for the time being, since this issue inspired a PR that was very similar to the tutorial added in #137215.

Navigation Menu

Repository navigation

Asyncio Tutorial #79012

Linked PRs

33 remaining items

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Asyncio Tutorial #79012

Description

Linked PRs

Activity

cjrh commented on Sep 28, 2018

1st1 commented on Sep 29, 2018

cjrh commented on Oct 1, 2018

1st1 commented on Oct 3, 2018

cjrh commented on Oct 7, 2018

cjrh commented on Oct 7, 2018

cjrh commented on Oct 7, 2018

ned-deily commented on Oct 7, 2018

1st1 commented on Oct 7, 2018

33 remaining items

picnixz commented on Dec 15, 2024

gvanrossum commented on Dec 15, 2024

picnixz commented on Dec 15, 2024

Footnotes

gvanrossum commented on Dec 15, 2024

StanFromIreland commented on Aug 9, 2025

AA-Turner commented on Aug 9, 2025

ZeroIntensity commented on Feb 8, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions