-
Notifications
You must be signed in to change notification settings - Fork 11
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Documentation geoips installation #651
base: main
Are you sure you want to change the base?
Changes from 21 commits
9c1bd78
dce320c
ba1a06b
ed06cad
516b271
059f535
b14d892
c89e13a
108ed4c
0065696
f8a9ae4
df0c7db
f15afb0
0b11853
30f39c4
469b585
4a5e1af
cd718af
6e06c24
95c026b
9c60e4d
c0e51aa
8caf778
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
:orphan: | ||
|
||
Testing | ||
======= | ||
|
||
.. note:: | ||
|
||
Setting up WSL through windows is the easiest method for running GeoIPS on | ||
a Windows system. To do so, follow directions on | ||
`Microsoft WSL Install <https://learn.microsoft.com/en-us/windows/wsl/install>`_. | ||
This requires administrator privileges. | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,155 @@ | ||
:orphan: | ||
|
||
.. dropdown:: Distribution Statement | ||
|
||
| # # # This source code is protected under the license referenced at | ||
| # # # https://github.com/NRLMMD-GEOIPS. | ||
|
||
Installing | ||
========== | ||
|
||
.. note:: | ||
|
||
Setting up WSL through windows is the easiest method for running GeoIPS on | ||
a Windows system. To do so, follow directions on | ||
`Microsoft WSL Install <https://learn.microsoft.com/en-us/windows/wsl/install>`_. | ||
This requires administrator privileges. | ||
|
||
For all installation methods, you need to setup environmental variables. | ||
|
||
The required environmental variables are: | ||
|
||
-``GEOIPS_PACKAGES_DIR`` is where GeoIPS looks for itself and its plugin packages | ||
-``GEOIPS_TESTDATA_DIR`` is where GeoIPS looks to find its test datasets | ||
-``GEOIPS_OUTDIRS`` defines where GeoIPS will write output data and imagery | ||
|
||
These are fine defaults for most users: | ||
|
||
.. code:: bash | ||
|
||
export GEOIPS_PACKAGES_DIR=$HOME/geoips | ||
export GEOIPS_TESTDATA_DIR=$GEOIPS_PACKAGES_DIR/test_data | ||
export GEOIPS_OUTDIRS=$GEOIPS_PACKAGES_DIR/outdirs | ||
|
||
If desired, the GeoIPS environment variables can be added to your | ||
default shell configuration file (which is typically stored in | ||
Comment on lines
+8
to
+35
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are we going to link to this from anywhere, or just leave it orphaned for now, for later update / completion? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ^ I think orphaned for now is fine |
||
``$HOME/.bashrc``) by running the following commands or something similar: | ||
|
||
.. code:: bash | ||
|
||
echo "export GEOIPS_PACKAGES_DIR=$GEOIPS_PACKAGES_DIR" >> ~/.bashrc | ||
echo "export GEOIPS_TESTDATA_DIR=$GEOIPS_TESTDATA_DIR" >> ~/.bashrc | ||
echo "export GEOIPS_OUTDIRS=$GEOIPS_OUTDIRS" >> ~/.bashrc | ||
|
||
Install system dependencies directly | ||
------------------------------------ | ||
|
||
If you are an expert user, you may want to install the dependencies manually. Do not do this if you already installed | ||
dependencies with Conda. | ||
|
||
Required | ||
^^^^^^^^ | ||
|
||
* ``git`` | ||
* ``openblas`` (required for scipy pip install) | ||
* ``make`` (required for pypublicdecompwt) | ||
* ``python`` >= 3.9 | ||
|
||
Optional | ||
^^^^^^^^ | ||
|
||
* ``gfortran`` (required for plugins including fortran builds) | ||
* ``gcc`` and ``g++`` (required for plugins including fortran or C builds, and ARM machines) | ||
* ``pdflatex`` (optional, for building pdf documentation) | ||
* ``wget`` (required for downloading test data) | ||
* ``make`` (optional, for building packages on ARM machines) | ||
* Test data repos can be installed in `$GEOIPS_TESTDATA_DIR` | ||
|
||
Debian | ||
^^^^^^ | ||
|
||
For example, on Debian machines, you can install the optional and required dependencies like this: | ||
|
||
.. code:: bash | ||
|
||
sudo apt-get install git make libopenblas-dev python3 | ||
sudo apt-get install gfortran gcc g++ gdal-bin libgdal-dev texlive-latex-base wget make | ||
|
||
This takes ~1gb of space. | ||
|
||
Install GeoIPS | ||
-------------- | ||
|
||
We can use ``pip`` to install all GeoIPS Python dependencies, and GeoIPS itself. | ||
|
||
First, clone the GeoIPS git repository: | ||
|
||
.. code:: bash | ||
|
||
git clone https://github.com/NRLMMD-GeoIPS/geoips.git $GEOIPS_PACKAGES_DIR/geoips | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install "$GEOIPS_PACKAGES_DIR/geoips" | ||
|
||
If you want to install GeoIPS with all optional dependencies, you can use: | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install "$GEOIPS_PACKAGES_DIR/geoips[doc,lint,test,debug]" | ||
|
||
The optional dependencies are: | ||
|
||
- ``doc``: for building the documentation with Sphinx | ||
(the documentation is also available online at | ||
https://nrlmmd-geoips.github.io/geoips/) | ||
- ``lint``: for linting the code (useful for developers) | ||
- ``test``: for running the tests | ||
- ``debug``: for debugging the code with IPython/jupyter | ||
|
||
Development | ||
----------- | ||
|
||
The installation steps for developers are the same as for normal and/or expert users, except for one step. | ||
|
||
Most developers use Conda installations, but any dependency management solution is fine. | ||
|
||
When installing geoips, please install all the extras and install in **editable** mode so that changes to the code are | ||
immediately reflected in the installed package. Eg. | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install -e "$GEOIPS_PACKAGES_DIR/geoips[doc,lint,test,debug]" | ||
|
||
See the [ADDING FUNCTIONALITY] page for more details on how to contribute to GeoIPS. | ||
|
||
Docker | ||
------ | ||
|
||
We provide a working Dockerfile that can be used to run GeoIPS in a container. | ||
|
||
The Dockerfile can be built into a Docker image by cloning the GeoIPS repository and | ||
running ``docker build``. For example: | ||
|
||
.. code:: bash | ||
|
||
git clone https://github.com/NRLMMD-GEOIPS/geoips.git geoips | ||
cd geoips | ||
docker build -t geoips . | ||
|
||
The Docker image can be run with the following command: | ||
|
||
.. code:: bash | ||
|
||
docker run -it geoips # Run the container in interactive mode | ||
|
||
Right now, the Docker image is only used for development and testing. | ||
|
||
We suggest mounting in a data directory so your containers don't get too large. | ||
|
||
If you are interested in using the Docker build | ||
for production or plugin development, please reach | ||
out to us via our contact page [CONTACT PAGE] or create an issue on GitHub [ISSUE PAGE]. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,146 @@ | ||
:orphan: | ||
|
||
.. dropdown:: Distribution Statement | ||
|
||
| # # # This source code is protected under the license referenced at | ||
| # # # https://github.com/NRLMMD-GEOIPS. | ||
|
||
Installing | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. If this is going to be our "official" installatin method that we are sending out for people to use this week, then we will need to update the README.md to point here for installation. Do we want to update this now? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is what I tested, but I'm fine sending out the old instructions and waiting on this if we still have things to discuss |
||
========== | ||
|
||
.. note:: | ||
|
||
Setting up WSL through windows is the easiest method for running GeoIPS on | ||
a Windows system. To do so, follow directions on | ||
`Microsoft WSL Install <https://learn.microsoft.com/en-us/windows/wsl/install>`_. | ||
This requires administrator privileges. | ||
|
||
For all installation methods, you need to setup environmental variables. | ||
|
||
The required environmental variables are: | ||
|
||
- ``GEOIPS_PACKAGES_DIR`` is where GeoIPS looks for itself and its plugin packages | ||
- ``GEOIPS_TESTDATA_DIR`` is where GeoIPS looks to find its test datasets | ||
- ``GEOIPS_OUTDIRS`` defines where GeoIPS will write output data and imagery | ||
|
||
These are fine defaults for most users: | ||
|
||
.. code:: bash | ||
|
||
export GEOIPS_PACKAGES_DIR=$HOME/geoips | ||
export GEOIPS_TESTDATA_DIR=$GEOIPS_PACKAGES_DIR/test_data | ||
export GEOIPS_OUTDIRS=$GEOIPS_PACKAGES_DIR/outdirs | ||
|
||
If desired, the GeoIPS environment variables can be added to your | ||
default shell configuration file (which is typically stored in | ||
``$HOME/.bashrc``) by running the following commands or something similar: | ||
|
||
.. code:: bash | ||
|
||
echo "export GEOIPS_PACKAGES_DIR=$GEOIPS_PACKAGES_DIR" >> ~/.bashrc | ||
echo "export GEOIPS_TESTDATA_DIR=$GEOIPS_TESTDATA_DIR" >> ~/.bashrc | ||
echo "export GEOIPS_OUTDIRS=$GEOIPS_OUTDIRS" >> ~/.bashrc | ||
|
||
Install Dependencies with Conda | ||
------------------------------- | ||
|
||
Conda is a package and virtual environment manager that can | ||
be used to install the dependencies of GeoIPS. | ||
|
||
We recommend using Conda if you are new to GeoIPS, programming or Python. | ||
|
||
If you are an expert user, you may want to use the manual | ||
or Docker installation methods. | ||
|
||
Get Conda | ||
^^^^^^^^^ | ||
|
||
- Download the appropriate version of `Conda | ||
<https://www.anaconda.com/download#downloads>`_, `Miniconda | ||
<https://docs.conda.io/en/latest/miniconda.html>`_, or | ||
`Miniforge/Mambaforge <https://github.com/conda-forge/miniforge#download>`_. | ||
|
||
For example, for Linux with Intel chips, one of the following might work: | ||
|
||
.. code:: bash | ||
|
||
wget https://github.com/conda-forge/miniforge/releases/latest/download/Mambaforge-Linux-x86_64.sh | ||
|
||
To find your specific version, visit the `Miniforge/Mambaforge github page | ||
<https://github.com/conda-forge/miniforge/releases/>`_. | ||
|
||
- Make the install script executable and run the installer, | ||
following the prompts. Eg.: | ||
|
||
.. code:: bash | ||
|
||
chmod u+x ./Mambaforge-Linux-x86_64.sh | ||
# After installing, follow instructions regarding conda init / restarting your terminal ! | ||
# Make sure you set env vars again if you restart your terminal ! | ||
./Mambaforge-Linux-x86_64.sh | ||
|
||
.. code:: bash | ||
|
||
rm -f ./Mambaforge-Linux-x86_64.sh | ||
|
||
Create and activate a conda environment with dependencies | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
Next we'll create a conda environment named ``geoips`` that contains all system | ||
requirements for GeoIPS. | ||
|
||
.. code:: bash | ||
|
||
mamba create -y -n geoips -c conda-forge python=3.10 openblas git gfortran | ||
|
||
.. warning:: | ||
|
||
If you are using an ARM machine, you may need to install ``pyhdf`` with conda. | ||
If you run into ``fatal error: 'hdf.h' file not found`` when installing ``pyhdf``, | ||
you may need to install ``pyhdf`` with conda. You can do this with the following command: | ||
|
||
.. code:: bash | ||
|
||
mamba create -y -n geoips -c conda-forge python=3.10 openblas git gfortran **pyhdf** | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I might just comment this line so people don't do this by default. I don't know if it matters either way, but 99% sure people will just copy/paste their way through. |
||
|
||
Activate the conda environment | ||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ | ||
|
||
You will need to run ``conda activate geoips`` (or ``mamba activate geoips``) | ||
every time you want to run or work on GeoIPS. Eg. | ||
|
||
.. code:: bash | ||
|
||
conda activate geoips | ||
|
||
Install GeoIPS | ||
-------------- | ||
|
||
We can use ``pip`` to install all GeoIPS Python dependencies, tests for GeoIPS and GeoIPS itself. | ||
|
||
First, clone the GeoIPS git repository: | ||
|
||
.. code:: bash | ||
|
||
git clone https://github.com/NRLMMD-GeoIPS/geoips.git $GEOIPS_PACKAGES_DIR/geoips | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install -e "$GEOIPS_PACKAGES_DIR/geoips[test]" | ||
|
||
If you want to install GeoIPS with all optional dependencies, you can use: | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install -e "$GEOIPS_PACKAGES_DIR/geoips[doc,lint,test,debug]" | ||
|
||
The optional dependencies are: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I would make a note here: "These optional dependencies are all required for tests to fully pass" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See my comments above 🙂 |
||
|
||
- ``doc``: for building the documentation with Sphinx | ||
(the documentation is also available online at | ||
https://nrlmmd-geoips.github.io/geoips/) | ||
- ``lint``: for linting the code (useful for developers) | ||
- ``test``: for running the tests | ||
- ``debug``: for debugging the code with IPython/jupyter (also useful for developers) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it is worth having the base_install.sh and base_test.sh bundled as part of the basic installation - it is worth having immediate feedback that the install was successful. And it's really only 3 extra lines (base_install.sh, create_plugin_registries, and base_test.sh). Oh - there is no create_plugin_registries included in this installation, so geoips is not fully functional at the end of this document. |
This file was deleted.
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
:orphan: | ||
|
||
Install GeoIPS | ||
-------------- | ||
|
||
We can use ``pip`` to install all GeoIPS Python dependencies, and GeoIPS itself. | ||
|
||
First, clone the GeoIPS git repository: | ||
|
||
.. code:: bash | ||
|
||
git clone https://github.com/NRLMMD-GeoIPS/geoips.git $GEOIPS_PACKAGES_DIR/geoips | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install "$GEOIPS_PACKAGES_DIR/geoips" | ||
|
||
If you want to install GeoIPS with all optional dependencies, you can use: | ||
|
||
.. code:: bash | ||
|
||
# Ensure geoips python environment enabled before installing geoips | ||
pip install "$GEOIPS_PACKAGES_DIR/geoips[doc,lint,test,debug]" | ||
|
||
The optional dependencies are: | ||
|
||
- ``doc``: for building the documentation with Sphinx | ||
(the documentation is also available online at | ||
https://nrlmmd-geoips.github.io/geoips/) | ||
- ``lint``: for linting the code (useful for developers) | ||
- ``test``: for running the tests | ||
- ``debug``: for debugging the code with IPython/jupyter | ||
Comment on lines
+3
to
+33
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. What is the intention for this section of the installation? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The intention is to - later - embed it in the advanced and the basic installation instructions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is the intention for this section of the getting started?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The intention is to add more information about testing geoips in the future, but that's not needed here