Crash after condacolab installation

[FrPsc]

I have this issue with running a colab notebook using condacolab.

CELL 1
import subprocess
subprocess.run( 'pip install -q condacolab'.split() )
import condacolab
condacolab.install()

CELL 2
import condacolab
condacolab.check()

This returns the following error:
AssertionError Traceback (most recent call last)
in <cell line: 2>()
1 import condacolab
----> 2 condacolab.check()

/usr/local/lib/python3.9/dist-packages/condacolab.py in check(prefix, verbose)
300 f"{prefix}/bin" in os.environ["PATH"]
301 ), f"💥💔💥 PATH was not patched! Value: {os.environ['PATH']}"
--> 302 assert (
303 f"{prefix}/lib" in os.environ["LD_LIBRARY_PATH"]
304 ), f"💥💔💥 LD_LIBRARY_PATH was not patched! Value: {os.environ['LD_LIBRARY_PATH']}"

AssertionError: 💥💔💥 LD_LIBRARY_PATH was not patched! Value: /usr/local/nvidia/lib:/usr/local/nvidia/lib64

If I skip condacolab.check and proceed to installing and importing packages:
CELL 2
import subprocess
_ = subprocess.run( 'mamba install scipy -c conda-forge --yes'.split() )
import scipy

I get a crash from colab: Your session crashed for an unknown reason.

Remarkably, in both cases, I only get this issue when executing all cells together. If instead I execute Cell 1, wait its completion, then execute the second, this runs smoothly.

[jaimergp]

Can you use the ! syntax as in the examples? condacolab requires a kernel restart in between commands, so yes, two separate cells are recommended. The "crash" might be expected.

https://github.com/conda-incubator/condacolab/tree/0.1.x#faq

[matteo-cagiada]

Hello, I have the same problem with my colab notebook. I'm following the installation procedure as described in your repo, but running the cell after the installation bring to an unexpected behaviour where the required packages are not installed. The temporary solution is to run the cell twice, which doubles the installation time (which is already 10 minutes), or to add a fake cell between the two, but this didn't happen in any of the previous installations of condacolab. Do you think it can be solved?

[jaimergp]

We haven't changed anything significant in between versions, just the default installer we point to. So it must be a Colab change. Using different cells work for me in the linked example from the readme (tried on Saturday).

[FrPsc]

Can you use the ! syntax as in the examples? condacolab requires a kernel restart in between commands, so yes, two separate cells are recommended. The "crash" might be expected.

https://github.com/conda-incubator/condacolab/tree/0.1.x#faq

Yes I can also use ! instead of subprocess, but the result is the same. I understand a crash is to be expected, but does this mean that it will not be possible to run the notebook's cells altogether when a condacolab installation is present?

For the record, I succeeded in running together the first two cells reported in the message above without the condacolab.check() returning an error. This was done by adding the two lines (indicated by the arrows) to the first cell:

! pip install -q condacolab
import condacolab
condacolab.install()
import os <--
os.environ['LD_LIBRARY_PATH'] += ':/usr/local/lib/' <--

So I've added a third cell to set the environment:

! mamba install matplotlib mdtraj openmm=7.5 -c conda-forge --yes
import pandas as pd
import os
import shutil
import numpy as np
import scipy.stats as scs
from scipy.optimize import curve_fit
from sklearn.linear_model import LinearRegression
import mdtraj as md
from simtk import openmm, unit
from simtk.openmm import app

And again, all is fine if I run the cells one by one, but if I select Run all, I get the following error on this last cell:
ImportError Traceback (most recent call last)
in <cell line: 2>()
1 get_ipython().system(' mamba install matplotlib mdtraj openmm=7.5 kneed -c conda-forge --yes')
----> 2 import pandas as pd
3 import os
4 import shutil
5 import numpy as np

7 frames
/usr/local/lib/python3.9/site-packages/pandas/core/window/ewm.py in
9
10 from pandas._libs.tslibs import Timedelta
---> 11 import pandas._libs.window.aggregations as window_aggregations
12 from pandas._typing import (
13 Axis,

ImportError: /lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.29' not found (required by /usr/local/lib/python3.9/site-packages/pandas/_libs/window/aggregations.cpython-39-x86_64-linux-gnu.so)

This happens with specific libraries. If I comment out pandas, then I get the same error on scipy. So it looks like there are problems with environmental paths after the condacolab installation and the kernel restart is needed to reset the path settings. I was wondering if there is a way to adjust these paths (similarly to the fix I've used to get condacolab.check to pass without kernel restart) after the condacolab installation, in order to avoid a kernel restart.

[jaimergp]

We need an interpreter restart to LD_LIBRARY_PATH is re-read. I don't know of any other ways for the python process to locate and link shared objects.

condacolab.install() issues the interpreter restart, and we need to wait there til it happens. You are patching os.environ at runtime, so the checks that run immediately after (while we are still waiting for the process to restart) pass, but it's just "lying to ourselves".

You are seeing those ABI issues with some libraries precisely because of that.

So in a nutshell, no, we don't support "Run All Cells". You need to issue the restart and wait.

I can probably look into adding a time.sleep() statement somewhere so users cannot run into this situation you are mentioning.