Make Grafzahl look for conda in the right place

[ureber]

Hi there,

I'm having a bit of trouble getting Grafzahl to look for miniconda in the right place. Since I had to install miniconda manually, it's not in the usual folder (r-miniconda), but some other (miniconda3). I saw that this was an issue before (#20) and tried to solve it by specifying the right path for both RETICULATE_MINICONDA_PATH and GRAFZAHL_MINICONDA_PATH. However, detect_conda() still returns FALSE.

The problem, I suspect, is that the .gen_conda_path function adds "bin" and "conda" to the path, which then points to a folder/file that doesn't exist within miniconda3. In my case, the right path would be "condabin" and then "conda" (I guess). I don't know if this is a version or system related issue, but any idea on how to fix this or some workaround would be greatly appreciated.

Thanks!

[chainsawriot]

sessionInfo(), please.

Also please tell me the output of

reticulate::conda_list()

[ureber]

Sure!

sessionInfo()

R version 4.3.1 (2023-06-16 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows Server 2019 x64 (build 17763)

Matrix products: default

locale:
[1] LC_COLLATE=German_Switzerland.1252 LC_CTYPE=German_Switzerland.1252
[3] LC_MONETARY=German_Switzerland.1252 LC_NUMERIC=C
[5] LC_TIME=German_Switzerland.1252

time zone: Europe/Zurich
tzcode source: internal

attached base packages:
[1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached):
[1] Matrix_1.5-4.1 glmnet_4.1-7 gtable_0.3.3 jsonlite_1.8.7 compiler_4.3.1
[6] Rcpp_1.0.11 grafzahl_0.0.8 assertthat_0.2.1 gower_1.0.1 splines_4.3.1
[11] scales_1.2.1 png_0.1-8 reticulate_1.30 lattice_0.21-8 ggplot2_3.4.2
[16] R6_2.5.1 shape_1.4.6 iterators_1.0.14 tibble_3.2.1 munsell_0.5.0
[21] lime_0.5.3 pillar_1.9.0 rlang_1.1.1 utf8_1.2.3 stringi_1.7.12
[26] fs_1.6.3 cli_3.6.1 withr_2.5.0 magrittr_2.0.3 foreach_1.5.2
[31] grid_4.3.1 rstudioapi_0.15.0 rappdirs_0.3.3 lifecycle_1.0.3 vctrs_0.6.3
[36] glue_1.6.2 codetools_0.2-19 survival_3.5-5 fansi_1.0.4 colorspace_2.1-0
[41] purrr_1.0.1 tools_4.3.1 usethis_2.2.2 pkgconfig_2.0.3

reticulate::conda_list() when RETICULATE_MINICONDA_PATH=C:/Users/ur21m712/AppData/Local/miniconda3/

Error in system2(conda, c("info", "--base"), stdout = TRUE) :
'"C:/Users/ur21m712/AppData/Local/miniconda3"' not found
In addition: Warning messages:
1: In conda_binary(conda) :
Supplied path is not a conda binary: ‘C:/Users/ur21m712/AppData/Local/miniconda3’
2: In conda_binary(conda) :
Supplied path is not a conda binary: ‘C:/Users/ur21m712/AppData/Local/miniconda3’

reticulate::conda_list() when RETICULATE_MINICONDA_PATH=C:/Users/ur21m712/AppData/Local/miniconda3/bin/conda 🤔

                name

1 base
2 grafzahl_condaenv_cuda
python
1 C:\Users\ur21m712\AppData\Local\miniconda3/python.exe
2 C:\Users\ur21m712\AppData\Local\miniconda3\envs\grafzahl_condaenv_cuda/python.exe

[ureber]

No, wait! Don't no what made the difference, but now I consistently get the correct (?) output for the option RETICULATE_MINICONDA_PATH=C:/Users/ur21m712/AppData/Local/miniconda3/

                name

1 base
2 grafzahl_condaenv_cuda
python
1 C:\Users\ur21m712\AppData\Local\miniconda3/python.exe
2 C:\Users\ur21m712\AppData\Local\miniconda3\envs\grafzahl_condaenv_cuda/python.exe

The slashes are all mixed up. Maybe that's an issue?

[chainsawriot]

@ureber Could you please try this in your command line environment (cmd.exe)?

C:\Users\ur21m712\AppData\Local\miniconda3\envs\grafzahl_condaenv_cuda\python.exe

Do you get a Python console?

---

Short explanation: I don't have Windows to test this package. Therefore, the support is not as great. For example, I don't know the directory structure of a miniconda install on Windows is different from other *nix systems.

detect_conda() is not a very robust test and might need to modify for Windows.

[ureber]

Yes, I get a Python console (3.10.12).

I'm afraid I can't be of much help here, as my experience with conda on Windows is limited to this server (which is not mine).

[chainsawriot]

@ureber Could you please give this a try (The GPU support for Windows is still WIP)?

remotes::install_github("chainsawriot/grafzahl@windows")

setup_grafzahl(cuda = FALSE)
detect_conda()
detect_cuda() # FALSE
model <- grafzahl(unciviltweets, model_type = "bertweet", model_name = "vinai/bertweet-base")

[chainsawriot]

If you don't like to setup the conda environment afresh.

remotes::install_github("chainsawriot/grafzahl@windows")
require(grafzahl)
Sys.setenv("GRAFZAHL_MINICONDA_PATH" = "C:\Users\ur21m712\AppData\Local\miniconda3")
detect_conda()
detect_cuda()
model <- grafzahl(unciviltweets, model_type = "bertweet", model_name = "vinai/bertweet-base")

[ureber]

Thanks for the quick support. I tried both options. First the latter one, without a new installation of conda. As before, detect_conda() returned FALSE. I then ran the setup again, which produced the following error. Running detect_conda() led to TRUE, however.

Error in py_run_file_impl(file, local, convert) :
ModuleNotFoundError: No module named 'transformers.models.mmbt'

[chainsawriot]

And yes, I found this issue while fixing this: ThilinaRajapakse/simpletransformers#1539

And the CPU support in the pull request has a temp. fix #25 to this.

In order to fix this in your GPU miniconda install now, with your Conda console

conda activate grafzahl_condaenv_cuda
python -m pip uninstall simpletransformers
python -m pip uninstall transformers
python -m pip uninstall scipy
python -m pip install simpletransformers "transformers==4.30.2" "scipy==1.10.1"
conda deactivate

[ureber]

This made it work! Both detect_conda() and detect_cuda() now return TRUE and also fine-tuning the model with grafzahl() works. I'm not entirely sure which installation it's running on, although judging by the speed, it's probably the GPU one.