Skip to content

Commit

Permalink
Improved build system for cmf2
Browse files Browse the repository at this point in the history
  • Loading branch information
Philipp Kraft committed Jan 15, 2021
1 parent 125ade8 commit 7836736
Show file tree
Hide file tree
Showing 9 changed files with 405 additions and 522 deletions.
4 changes: 2 additions & 2 deletions cmf/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@
from .timetools import StopWatch, datetime_to_cmf, timerange


__version__ = '2.0.0b4.cmf2'
__compiletime__ = 'Mon Sep 23 14:41:22 2019'
__version__ = '2.0.0b5.cmf2'
__compiletime__ = 'Fri Jan 15 16:05:41 2021'

from .cmf_core import connect_cells_with_flux as __ccwf

Expand Down
344 changes: 157 additions & 187 deletions cmf/cmf_core.py
View file

Large diffs are not rendered by default.

267 changes: 116 additions & 151 deletions cmf/cmf_core_src/cmf_wrap.cpp
View file

Large diffs are not rendered by default.

79 changes: 1 addition & 78 deletions documentation/installation/CmfInstallUbuntu.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,87 +4,10 @@

# Install cmf on a Linux with sudo privileges

First you need to make sure that the dependencies of cmf are fullfilled.
It is good practise to get ppackages from your Linux distribution, if
they are available and to get only those packages, that you can't get
from your distribution to install directly in Python. Any package
installed by your OS package manager will be updated by the OS, while
other packages need to get updated by yourself.

I am working mostly with Ubuntu variants, hence i can't say anything
about other Linux package managers. However an internet search like
"apt-get fedora" will help.

## Python 2.7

You need sudo privileges to install the dependencies for the system. You
will need `python-dev` and `numpy`. Recommended are `matplotlib`,
`scipy`and `pandas`.

### Dependencies of cmf from the Ubuntu package manager

#!sh
sudo apt-get install python-dev python-numpy python-scipy python-pandas python-matplotlib

### Get dependencies from the Python Package Index (PyPI)

Follow this steps if you can't get the packages from your OS or if you
do not have sudo privileges. This text assumes you want everything only
installed for yourself and not for other users.

First check your python version:

#!sh
python --version

If the version is \<=2.7.8 then get the program `pip` to install more
packages.For higher versions, `pip` is already part of Python

#!sh
wget https://bootstrap.pypa.io/get-pip.py
python get-pip.py --user

The next steps are only needed if you did not used the `apt-get`
commands from above:

#!sh
pip install numpy --user
pip install matplotlib --user
pip install scipy --user
pip install pandas --user

### Compile cmf

Get the source code from CmfDownload and unzip it to /cmf-source

#!sh
cd cmf-source
python setup.py install --user
cd .. # Important! You cannot test your installation from the source folder

or, without multi core processor support (without OpenMP)

#!sh
cd cmf-source
python setup.py install noopenmp --user
cd .. # Important! You cannot test your installation from the source folder

## Adding geometric support to CMF

To added geometric features to cmf, you need to install the geos library
and shapely. For the geos library, you need sudo privileges

#!sh
wget http://download.osgeo.org/geos/geos-3.2.2.tar.bz2
tar -xjf geos-3.2.2.tar.bz2
cd geos-3.2.2
./configure
make
sudo make install
cd ..
pip install shapely --user

## Python 3.5
## Python 3.7 and higher

Work in progress..

Expand Down
202 changes: 106 additions & 96 deletions documentation/installation/CmfInstallWindows.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,143 +2,153 @@

[back...](@ref cmfInstall)

# Installing cmf on Windows for Python 2.7

# Python 3

## Without geometry

This is the simplest way to install cmf on Windows. It shows one path,
that should be suitable for most computers with a Windows system.
Alternatives are listed below

- Get a recent winpython distribution from
(<http://sourceforge.net/projects/winpython/>). Today the 64bit
variant should run on most computers. When in doubt - use 32bit, it
fits always.
- "Install" it to any directory on your computer, eg. C:\\Apps
- Download cmf - Binary for Python 2.7 32 bit from CmfDownload
- Open C:\\Apps\\Winpython2.732bit in your file explorer and double
click on the "Winpython Control Panel.exe"
- In that program, click on a "Add packages" and add the downloaded
cmf
- Click install, and here you go...

If you already have an existing Python installation you can also use:
variant should run on most computers.
- "Install" it to any directory on your computer, eg. C:\\Apps

Open the WinPython Command Prompt and type

pip install cmf

### More choice

What you really need to install cmf is a running python 2.7 interpreter
with the `numpy` package. Having other packages from the so called
`scipy-Stack` available, like `scipy`, `matplotlib`, `pandas` is
fine. The best way to get the scipy stack is to install a scientific
python distribution. winpython is one choice, but Anaconda and
python(x,y) are others and they work equally fine. But please follow the
instructions to add packages to these distributions, there are some
traps around.

### Adding Geometry
## Adding Geometry

Just get the shapely binary from:

<http://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely>

and install it with the pacakge manager of your python distribution. it
is the Winpython Control Panel fro winpython.
and install it with the pacakge manager of your python distribution. Eg. use
the Winpython Control Panel for winpython.

## Compile cmf for Python 3 with Visual Studio

### Prerequisites

To compile cmf2 you need the following installed on your computer:

- Buildtools for Visual Studio from https://visualstudio.microsoft.com/de/downloads/. Please check
Python and Visual Studio compiler version at https://pythondev.readthedocs.io/windows.html
- A Python (>3.6)) installation, preferably a scientific distribution like WinPython or Anaconda,
including pip
- The CMF source code. Get it using Git or download a ZIP from
GitHub.com (https://github.com/philippkraft/cmf)
- Optional: SWIG from swig.org (in the path). If you plan to change the C++ API of CMF you need to
generate the Python interface with SWIG again.
accessible from


### Installation of requirements

Open a Visual Studio Development Command Prompt and make sure the right python installation
is in the path (eg. python -VV). Navigate to the cmf source code.
It is a good idea to create a virtual environment (venv) for the compilation

Check if nmake is in the path, if you get an error message you are not using the Visual Studio
Development Command Prompt or you have not installed the Build Tools properly

nmake /?

Install the requirements

pip install -r requirements.txt

Install cmake

pip install cmake

## Compile cmf from source for python 2.7
### One-Step installation of CMF

If you want to use always the most recent version of cmf or if you like
to make changes to the C++ source files of cmf, you need to compile the
C++ files to machine language.
Installing and compiling CMF takes some time. The easiest way is to just install cmf from
the source directory using pip install. In case of problems, it is better to compile CMF
step by step (see below)

'''Note''': Compiling C++ code on Windows is not simple
pip install . -v

To compile sources, you need a compiler - the right compiler. Python 2.7
for Windows is compiled with Visual Studio 2008, and you need the same
compiler. However, you cannot buy this compiler and development tool
anymore, because it is to old. But Microsoft gives the compiler without
the integrated development environment (IDE) away for free.
The `-v` stands for verbose, to give more output

### Get the right compiler and other prerequisites
### Step by step installation of CMF

Download the right C++ compiler (called Windows SDK) here: [Microsoft
Visual C++ Compiler for
Python 2.7](https://www.microsoft.com/download/details.aspx?id=44266)
Compiling and installing of cmf consists of the following steps

And install it. To use the compiler you might need to tweak it a bit, to
have the compiler working seamlessly with Python. Save the following
batch file to as `C:\\Program Files\\Microsoft Visual Studio
9.0\\VC\\vcvarsall.bat`
1. Download the source code of the solvers in CMF (suitesparse and sundials) and build them
2. Build the CMF C++ library linking the solvers
3. Build the Python extension and link it with the libraries
4. Build the Python tools for CMF
5. Install the whole library in your Python installation
6. Test it

@echo off
REM Calls the right environment depending on your architecture
if "%1x"=="x86x" "%~dp0\bin\vcvars32.bat"
if "%1x"=="amd64x" "%~dp0\bin\vcvars64.bat"
#### Build the solvers (1)

Next you need to make the compiler a bit more "modern":
tools\install_solvers.bat

- Download and install the boost library (<http://www.boost.org>),
headers only is fine.
- Create an environmental variable called BOOSTDIR pointing to the
boost installation directory.
The source code for the solvers is downloaded to `build\extern`, the built libraries are
stored in the `lib` directory

### Compile
#### Build the CMF C++ Library (2)

- Get the cmf source code from CmfDownload and unzip it on your hard
drive
- Open a command line and navigate to the cmf source directory
tools\install_cmf_core.bat

Compiling cmf and installing it for your computer is than as simple as
typing in a command line:
There are a couple of test executables build together with cmf.
These executables can be found in the directory `bin`. The static library is together
with the solver libraries in `lib\lib`

'''Note''': Do not test the intallation in the source code folder. The
compiled library will not be found.
#### Build and link the Python extension (3)

python setup.py build_ext

If the C++ API has been changed, build the wrapper interface newly with

python setup.py build_ext swig

#### Build the Python tools

python setup.py build_py

#### Install the package to the current python version

python setup.py install

# Python 3.5
#### Test cmf

## Without geometry
Do not fire up a python command prompt and import cmf in the source directory. This will fail.

This is the simplest way to install cmf on Windows. It shows one path,
that should be suitable for most computers with a Windows system.
Alternatives are listed below
Either change the directory first or run the cmf tests with

- Get a recent winpython distribution from
(<http://sourceforge.net/projects/winpython/>). Today the 64bit
variant should run on most computers. When in doubt - use 32bit, it
fits always.
- "Install" it to any directory on your computer, eg. C:\\Apps
- Download cmf - Binary for Python 3.5 32 bit from CmfDownload
- Open C:\\Apps\\Winpython2.732bit in your file explorer and double
click on the "Winpython Control Panel.exe"
- In that program, click on a "Add packages" and add the downloaded
cmf
- Click install, and here you go...

If you already have an existing Python installation you can also use:
tools\test.bat

pip install cmf
This will take care of the necessary directory change

## Adding Geometry

Just get the shapely binary from:
### Deployment of the compiled version

<http://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely>
You can distribute your compiled version to other Windows users by building a binary wheel.

python setup.py bdist_wheel

or

and install it with the pacakge manager of your python distribution. Use
again the Winpython Control Panel for winpython.
pip wheel .

## Compile cmf for Python 3.5
The .whl file is in the `dist` directory and can be easily shared between computers.

Compiling cmf for Python 3.5 works in Windows. However, there is no
established procedure by now. The main point is, that Python 3.5 is
compiled with Visual Studio 2015, which is actually available and for
free (no money, no freedom) available for private persons, academic
institutions and small companies. It should also work with: [Visual C++
compiler 2015](http://landinghub.visualstudio.com/visual-cpp-build-tools).
You can find more references on how to compile C and C++ extensions
(like cmf) for Python on Windows
[here](https://wiki.python.org/moin/WindowsCompilers) including links to
the Microsoft Compiler without Visual Studio
#### Note:
The .whl is ony suitable for the same Python version, architecture (x64/x86)
and operating system (Windows). If you are distributing a compiled version of CMF,
it is required by the licence to give the receivers of the binary package access to
the source that has been used to compile the binary. In case of an unchanged CMF a
link to the CMF code repository is sufficient. If you have made changes to the source
code, you are responsible to provide the receiver with your source code, which inherits
the GNU Public Licence from CMF.

## Compile CMF2 for Python 3 with the Windows Subsystem for Linux (WSL)

Just follow the instructions for installing CMF on Linux
8 changes: 4 additions & 4 deletions setup.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
from distutils.sysconfig import customize_compiler
from distutils.command.build_py import build_py

version = '2.0.0b4'
version = '2.0.0b5'

branchversion = version
try:
Expand Down Expand Up @@ -429,9 +429,9 @@ def make_cmf_core():
'License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)',
'Programming Language :: C++',
'Programming Language :: Python',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7',
'Programming Language :: Python :: 3.8',
'Programming Language :: Python :: 3.9',
'Topic :: Scientific/Engineering',
'Topic :: Software Development :: Libraries :: Python Modules',
]
Expand All @@ -441,7 +441,7 @@ def make_cmf_core():
license='GPLv3+',
ext_modules=ext,
packages=['cmf', 'cmf.draw', 'cmf.geometry'],
python_requires='>=3.5',
python_requires='>=3.7',
install_requires='numpy>=1.11.1',
keywords='hydrology catchment simulation toolbox',
author='Philipp Kraft',
Expand Down
2 changes: 1 addition & 1 deletion tools/Doxyfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ PROJECT_NAME = cmf
# could be handy for archiving the generated documentation or if some version
# control system is used.

PROJECT_NUMBER = 2.0.0b4
PROJECT_NUMBER = 2.0.0b5

# Using the PROJECT_BRIEF tag one can provide an optional one line description
# for a project that appears at the top of each page and should give viewer a
Expand Down
Loading

0 comments on commit 7836736

Please sign in to comment.