Upgrade IPython and tornado versions

[94rain]

Describe the current behavior
Currently supported versions of IPython and tornado are pretty old. Upgrading versions will fail, and it is not possible to run other packages that require newer versions of these packages (e.g. require IPython 7.33.0 and tornado 6.1). produce incompatibility errors and requires restarting runtime to make the new versions take into effect.

colabtools/setup.py

Lines 25 to 34 in 31fdaa9

	'ipython~=5.5.0',
	'notebook~=5.2.0; python_version < "3.0"',
	'notebook~=5.3.0; python_version >= "3.0"',
	'six~=1.15.0',
	'pandas~=0.24.0; python_version < "3.0"',
	'pandas>=1.1.0; python_version >= "3.0"',
	'portpicker~=1.3.1',
	'requests~=2.23.0',
	'tornado~=4.5.0; python_version < "3.0"',
	'tornado~=5.1.0; python_version >= "3.0"',

Describe the expected behavior
I wonder if it would be possible to upgrade the support for the IPython version to >7.33.0 and tornado to > 6.1.

What web browser you are using
Chrome 101.0.4951.64 (arm64)

[craigcitro]

We'd like to upgrade the versions soon; we don't have any specific timeline to share.

That said, I believe you should be able to upgrade (via !pip install) without any problem; I'm curious to hear if you're hitting an error with that approach.

[94rain]

Thanks for the response!

Yes, thank you for pointing that out. I just double-checked that with !pip install ipython tornado --upgrade --force-reinstall, it is indeed possible to upgrade the versions. But it will produce dependency conflicts errors and requires restarting the runtime to make the new versions take into effect. I didn't notice that the updates can still be done despite the dependency conflicts errors. I just updated the description.

dependency conflicts errors

  Attempting uninstall: tornado
    Found existing installation: tornado 5.1.1
    Uninstalling tornado-5.1.1:
      Successfully uninstalled tornado-5.1.1
  Attempting uninstall: ipython
    Found existing installation: ipython 5.5.0
    Uninstalling ipython-5.5.0:
      Successfully uninstalled ipython-5.5.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
jupyter-console 5.2.0 requires prompt-toolkit<2.0.0,>=1.0.0, but you have prompt-toolkit 3.0.29 which is incompatible.
google-colab 1.0.0 requires ipython~=5.5.0, but you have ipython 7.34.0 which is incompatible.
google-colab 1.0.0 requires tornado~=5.1.0; python_version >= "3.0", but you have tornado 6.1 which is incompatible.
Successfully installed ipython-7.34.0 prompt-toolkit-3.0.29 tornado-6.1
WARNING: Upgrading ipython, ipykernel, tornado, prompt-toolkit or pyzmq can
cause your runtime to repeatedly crash or behave in unexpected ways and is not
recommended. If your runtime won't connect or execute code, you can reset it
with "Disconnect and delete runtime" from the "Runtime" menu.
WARNING: The following packages were previously imported in this runtime:
  [IPython,prompt_toolkit,tornado]
You must restart the runtime in order to use newly installed versions.

[sammlapp]

I am having some weird issues with colab that seem to be related to this. When I install a python package (opensoundscape==0.10.2) which requires an ipython version >=8.10 I get this message that suggests colab requires exactly ipython==7.34.0

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires ipykernel==5.5.6, but you have ipykernel 6.29.5 which is incompatible.
google-colab 1.0.0 requires ipython==7.34.0, but you have ipython 8.26.0 which is incompatible.
notebook 6.5.5 requires jupyter-client<8,>=5.3.4, but you have jupyter-client 8.6.2 which is incompatible.
tensorflow-metadata 1.15.0 requires protobuf<4.21,>=3.20.3; python_version < "3.11", but you have protobuf 4.25.4 which is incompatible.
Successfully installed Deprecated-1.2.14 appdirs-1.4.4 arrow-1.3.0 aru-metadata-parser-0.1.0 asttokens-2.4.1 async-lru-2.0.4 comm-0.2.2 docker-pycreds-0.4.0 docopt-0.6.2 executing-2.0.1 fqdn-1.5.1 gitdb-4.0.11 gitpython-3.1.43 grad-cam-1.5.2 h11-0.14.0 httpcore-1.0.5 httpx-0.27.0 ipykernel-6.29.5 ipython-8.26.0 ipywidgets-8.1.3 isoduration-20.11.0 jedi-0.19.1 json5-0.9.25 jsonpointer-3.0.0 jupyter-client-8.6.2 jupyter-events-0.10.0 jupyter-lsp-2.2.5 jupyter-server-2.14.2 jupyter-server-terminals-0.5.3 jupyterlab-4.2.4 jupyterlab-server-2.27.3 matplotlib-3.9.1 nvidia-cublas-cu12-12.1.3.1 nvidia-cuda-cupti-cu12-12.1.105 nvidia-cuda-nvrtc-cu12-12.1.105 nvidia-cuda-runtime-cu12-12.1.105 nvidia-cudnn-cu12-8.9.2.26 nvidia-cufft-cu12-11.0.2.54 nvidia-curand-cu12-10.3.2.106 nvidia-cusolver-cu12-11.4.5.107 nvidia-cusparse-cu12-12.1.0.106 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.5.82 nvidia-nvtx-cu12-12.1.105 opensoundscape-0.10.2 overrides-7.7.0 pathtools-0.1.2 pillow-10.0.1 protobuf-4.25.4 pure-eval-0.2.3 python-json-logger-2.0.7 rfc3339-validator-0.1.4 rfc3986-validator-0.1.1 schema-0.7.7 sentry-sdk-2.11.0 setproctitle-1.3.3 smmap-5.0.1 stack-data-0.6.3 traitlets-5.14.3 ttach-0.0.3 types-python-dateutil-2.9.0.20240316 uri-template-1.3.0 wandb-0.13.11 widgetsnbextension-4.0.11
WARNING: Upgrading ipython, ipykernel, tornado, prompt-toolkit, pyzmq can
cause your runtime to repeatedly crash or behave in unexpected ways and is not
recommended. If your runtime won't connect or execute code, you can reset it
with "Disconnect and delete runtime" from the "Runtime" menu.

WARNING: The following packages were previously imported in this runtime:
[IPython,PIL]
You must restart the runtime in order to use newly installed versions.

Then, if I restart the runtime, it repeatedly crashes (as the message warns it might) and I cannot run the notebook. Attempting to run a cell results in a pop-up window with the message:

Error
Failed to execute cell. Could not send execute message to runtime: Error: await connected: disconnected
await connected: disconnected
Error: await connected: disconnected
at https://ssl.gstatic.com/colabor

Thus it seems I cannot update these packages and have a working colab notebook. Strangely, a collaborator using a separate notebook has no issues installing the same package and getting a working environment with ipython=8.26.0. I don't know what might cause the difference in behaviors between the two notebooks. I can also run the other notebook without a problem, but the problem reappears if I create a notebook of my own

[metrizable]

Colab uses ipython==7.34.0, ipykernel==6.17.1, and tornado==6.4.2 now.

[sammlapp]

Is it possible that the dependencies could be loosened, rather than requiring exact versions?