Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate content away from Anaconda #782

Draft
wants to merge 34 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
bf4ee86
remove anaconda channel from build env
wwarriner Aug 14, 2024
2a1a129
disable warnings for no git revision date
wwarriner Aug 14, 2024
df16e06
change to using_conda.md
wwarriner Aug 14, 2024
47cdc8e
add conda migration faq draft
wwarriner Aug 14, 2024
44dcb84
remove anaconda from using_conda.md
wwarriner Aug 16, 2024
16b5f95
removed anaconda from getting_started page
wwarriner Aug 16, 2024
9968eac
remove anaconda from ood_jupyter
wwarriner Aug 16, 2024
9ca4f8e
remove anaconda from ood_matlab
wwarriner Aug 16, 2024
0e22837
update software/software
wwarriner Aug 16, 2024
4ee8da2
move section to software/software
wwarriner Aug 16, 2024
d90bd87
remove anaconda from ood_r
wwarriner Aug 16, 2024
83eeb51
fix typo
wwarriner Aug 16, 2024
796e5c2
remove anaconda from gpus
wwarriner Aug 16, 2024
6814a9e
remove anaconda from slurm_tutorial
wwarriner Aug 16, 2024
3fbcfd8
remove anaconda from submitting_jobs
wwarriner Aug 16, 2024
bdeec92
remove anaconda from common_software.csv
wwarriner Aug 16, 2024
c27bdd7
remove anaconda from cheaha/tutorial/index
wwarriner Aug 16, 2024
10b26df
update cheaha/tutorial/index
wwarriner Aug 16, 2024
accb89b
update cheaha/tutorial/index
wwarriner Aug 16, 2024
49c42ba
remove anaconda from pytorch_tensorflow tutorial
wwarriner Aug 16, 2024
fcea560
remove anaconda from contributor guide
wwarriner Aug 16, 2024
20acdc5
remove anaconda from storage.md
wwarriner Aug 16, 2024
e1c2a23
remove anaconda from data_management/lts/interfaces.md
wwarriner Aug 16, 2024
463386f
update uab-rc-facilities.txt
wwarriner Aug 16, 2024
f2c86eb
remove anaconda from support.md
wwarriner Aug 16, 2024
df651c4
remove anaconda from uab_cloud/installing_software.md
wwarriner Aug 16, 2024
2aeaa7e
remove anaconda from getting_containers.md
wwarriner Aug 16, 2024
9a3893b
remove anaconda from r_environments.md
wwarriner Aug 16, 2024
3144643
remove anaconda from shell.md
wwarriner Aug 16, 2024
ef60530
removed as much anaconda as I could from using_conda.md
wwarriner Aug 16, 2024
f184f9a
fixed unicode double quote character
wwarriner Aug 16, 2024
0aa4539
remove miniconda from uab_cloud/installing_software
wwarriner Aug 16, 2024
04a0633
Merge branch 'main' into feat-anaconda-to-miniforge
wwarriner Sep 6, 2024
fc08bb1
Merge branch 'main' into feat-anaconda-to-miniforge
wwarriner Oct 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
move section to software/software
  • Loading branch information
wwarriner committed Aug 16, 2024
commit 4ee8da2ae5cec491f5da9acb46b08a040cc44af7
19 changes: 7 additions & 12 deletions docs/cheaha/open_ondemand/ood_jupyter.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,22 +111,17 @@ While launching an OOD HPC Desktop Job or any OOD Applications, if the user gets

Using `conda init` causes a block of code automatically inserted into the `.bashrc` file in your `$HOME` directory. This code block may interfere with the proper functioning of various OOD applications, resulting in a VNC error. To address this issue, it is recommended to follow the instructions outlined in the [FAQ entry](https://ask.cyberinfrastructure.org/t/why-do-i-get-an-error-when-launching-an-open-ondemand-hpc-interactive-session/2496).

### Pip Installs Packages Outside of Environment
### Installing Pip Packages Outside of Your Environments

When installing packages within a `conda` environment using `pip`, it's crucial to ensure that you install `pip` within the same conda environment and use `pip` from that environment. If `pip` is used outside of `conda` or within an environment without `pip` installed, the packages are installed to `~/.local`. This can lead to unexpected package conflicts, as Python loads packages from `~/.local` before loading from `conda` environments, and shows the following error,

```bash
Requirement already satisfied: numpy in /home/$USER/.local/lib/python3.11/site-packages (1.26.3)
```
<!-- markdownlint-disable MD046 -->
!!! danger

For the above case, resolving errors involve deleting the `~/.local` directory.
Using `pip install` without loading Miniforge3 will cause hard-to-diagnose errors and broken workflows.

Here's an example of the correct procedure for installing `pip` packages within a `conda`:
Using `pip install` in the `base` environment will cause the same hard-to-diagnose errors and broken workflows.

1. Load the `Miniforge` module using `module load Miniforge3`.
1. Create or activate the desired `conda` environment. Please refer to the [`conda` documentation](../../workflow_solutions/using_conda.md#create-an-environment)
1. Install `pip` within the `conda` environment using `conda install pip` or `conda install python`. `pip` and `python` are packaged together, installing one will always install the other.
1. Use `pip` when this `conda` environment is active to install packages. Please refer to [Installing packages with `pip`](../../workflow_solutions/using_conda.md#installing-packages-with-pip)
Read more about this issue, and how to resolve it, [here](../software/software.md#installing-pip-packages-outside-of-your-environments).
<!-- markdownlint-disable MD046 -->

### Tensorflow and PyTorch GPU issues

Expand Down
19 changes: 18 additions & 1 deletion docs/cheaha/software/software.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,28 @@ Once you have loaded the Miniforge module, `conda` on Cheaha works similarly to

Using `pip install` in the `base` environment will cause the same hard-to-diagnose errors and broken workflows.

Read more about this issue, and how to resolve it, [here](../open_ondemand/ood_jupyter.md#pip-installs-packages-outside-of-environment).
Read more about this issue, and how to resolve it, [here](#installing-pip-packages-outside-of-your-environments).
<!-- markdownlint-enable MD046 -->

For more information on usage with examples, see [`conda` Environments](../../workflow_solutions/using_conda.md). Need some hands-on experience? You can find instructions on how to install PyTorch and TensorFlow using `conda` in this [tutorial](../tutorial/pytorch_tensorflow.md).

### Installing Pip Packages Outside of Your Environments

When installing packages within a `conda` environment using `pip`, it's crucial to ensure that you install `pip` within the same conda environment and use `pip` from that environment. If `pip` is used outside of `conda` or within an environment without `pip` installed, the packages are installed to `~/.local`. This can lead to unexpected package conflicts, as Python loads packages from `~/.local` before loading from `conda` environments, and shows the following error,

```bash
Requirement already satisfied: numpy in /home/$USER/.local/lib/python3.11/site-packages (1.26.3)
```

For the above case, resolving errors involve deleting the `~/.local` directory.

Here's an example of the correct procedure for installing `pip` packages within a `conda`:

1. Load the `Miniforge` module using `module load Miniforge3`.
1. Create or activate the desired `conda` environment. Please refer to the [`conda` documentation](../../workflow_solutions/using_conda.md#create-an-environment)
1. Install `pip` within the `conda` environment using `conda install pip` or `conda install python`. `pip` and `python` are packaged together, installing one will always install the other.
1. Use `pip` when this `conda` environment is active to install packages. Please refer to [Installing packages with `pip`](../../workflow_solutions/using_conda.md#installing-packages-with-pip)

## Singularity Containers

Containers are a very useful resource for installing software without needing administrator permission. Please read the full documentation about singularity and containers on our [main Singularity page](../../workflow_solutions/getting_containers.md#containers-on-cheaha).
2 changes: 1 addition & 1 deletion docs/workflow_solutions/using_conda.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ When building a `conda` environment, prefer to get all of your packages through
<!-- markdownlint-disable MD046 -->
!!! important

Make sure `pip` is installed within the `conda` environment and use it for installing packages within the `conda` environment to prevent [Pip related issues](../cheaha/open_ondemand/ood_jupyter.md#pip-installs-packages-outside-of-environment).
When using `conda` and `pip` on Cheaha, make sure you are using a custom `conda` environment and that `pip` is installed before installing `pip` packages to prevent severe [`pip` related issues](../cheaha/software/software.md#installing-pip-packages-outside-of-your-environments).
<!-- markdownlint-disable MD046 -->

<!-- markdownlint-disable MD046 -->
Expand Down