Backport breaks Jupyter Notebook

[gschwaer]

Hi,
I just installed and run jupyter notebook for the first time using:

pip3 install notebook
jupyter notebook

After creating a notebook, I cannot execute any command. The backend reports the error:

[IPKernelApp] ERROR | Exception in message handler:
Traceback (most recent call last):
  File "/home/gschwaer/.local/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 261, in dispatch_shell
    yield gen.maybe_future(handler(stream, idents, msg))
  File "/home/gschwaer/.local/lib/python3.6/site-packages/tornado/gen.py", line 762, in run
    value = future.result()
  File "/home/gschwaer/.local/lib/python3.6/site-packages/tornado/gen.py", line 769, in run
    yielded = self.gen.throw(*exc_info)  # type: ignore
  File "/home/gschwaer/.local/lib/python3.6/site-packages/ipykernel/kernelbase.py", line 538, in execute_request
    user_expressions, allow_stdin,
  File "/home/gschwaer/.local/lib/python3.6/site-packages/tornado/gen.py", line 762, in run
    value = future.result()
  File "/home/gschwaer/.local/lib/python3.6/site-packages/tornado/gen.py", line 234, in wrapper
    yielded = ctx_run(next, result)
  File "/home/gschwaer/.local/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
    return callable(*args, **kwargs)
  File "/home/gschwaer/.local/lib/python3.6/site-packages/ipykernel/ipkernel.py", line 290, in do_execute
    with self._cancel_on_sigint(coro_future):
TypeError: should_run_async() got an unexpected keyword argument 'transformed_cell'

I found this commit: 54c29d3bfd8c97b190ee7270d42cc86daa58cf1d from @afshin (merged by @blink1073 of PR #605) which seems to cause this issue. If I reverse the changes of this commit everything works as expected. I think this was backported for some reason, but it is a breaking change that should require a version bump. Or maybe the jupyter devs need to restrict the ipykernel version differently from now on?

Best regards

[blink1073]

It looks like you may need to upgrade ipython (pip install -U ipython), since that API was introduced in ipython 7.17

[gschwaer]

Hm, I'm not a python expert. I tried that but it said (cropped)

$ pip3 install -U ipython
Requirement already satisfied: ipython in ./.local/lib/python3.6/site-packages (7.16.1)

So maybe the ipykernel dependencies are not up to date? Output of pipdeptree (cropped):

$ pipdeptree -p notebook
notebook==6.2.0
  - ipykernel [required: Any, installed: 5.5.1]
    - ipython [required: >=5.0.0, installed: 7.16.1]

[blink1073]

Perhaps pip install "ipython>=7.17"

[gschwaer]

$ pip install "ipython>=7.17"
ERROR: Could not find a version that satisfies the requirement ipython>=7.17
ERROR: No matching distribution found for ipython>=7.17

But regardless of that. Shouldn't ipykernel define the required dependency version of ipython as >=7.17 then?

$ pipdeptree -p ipykernel
ipykernel==5.5.1
  - ipython [required: >=5.0.0, installed: 7.16.1]

[blink1073]

Agreed, I deleted the 5.5.1 release since bumping a major version dependency in a patch release doesn't make sense.

[gschwaer]

Thank you! 👍

[blink1073]

Thanks for the report and detailed responses!

[gschwaer]

Sure, thank you for the super fast response ;) Have a nice day!

[gschwaer]

In case someone digs up this issue because he/she updated his/her dependencies in the wrong moment. Do this:

pip3 uninstall ipykernel
pip3 install ipykernel

Which should remove the 5.5.1 version and install the 5.5.0.