Skip to content

Using local copies of LAPACK for easier packaging. #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
6cdc2b5
final changes for windows packaging
gbarter Jan 27, 2024
95cb089
windows ci build that works
gbarter May 2, 2024
df8cbf5
brew install for mac
gbarter May 2, 2024
3937226
fix openblas pkg name msys2
gbarter May 2, 2024
8745f70
add more brew and msys2 packages
gbarter May 2, 2024
82b0aae
looking for libgomp
gbarter May 2, 2024
29ab786
try mac and windows again
gbarter May 2, 2024
bdcfef3
try mac and windows again
gbarter May 2, 2024
2af43de
try mac and windows again
gbarter May 2, 2024
32f53b5
try mac and windows again
gbarter May 2, 2024
b8e9874
try mac and windows again
gbarter May 2, 2024
f586417
try mac and windows again
gbarter May 2, 2024
ca8878e
try mac and windows again
gbarter May 2, 2024
2345276
try mac and windows again
gbarter May 2, 2024
fccecbe
try mac and windows again
gbarter May 2, 2024
d300792
improvements from wisdem
gbarter May 22, 2024
f07a44a
switch to using local lapack routines to simplify packaging
gbarter May 23, 2024
5710602
add new file to meson source list
gbarter May 23, 2024
3b47c47
tweaks
gbarter May 23, 2024
0ee1ee0
one step closer
gbarter May 23, 2024
c785c7e
quick switch back to setuptools
gbarter May 23, 2024
3efc647
need setup file. . .
gbarter May 23, 2024
cedece7
similar problems
gbarter May 23, 2024
1226902
debugging windows
gbarter May 23, 2024
3eafeb7
removing pip windows for now, everythiing seems to be working fine, b…
gbarter May 23, 2024
a2fddcd
remove tmate
gbarter May 23, 2024
3ca6630
forgot a step
gbarter May 23, 2024
8194245
set next version number
gbarter May 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions .github/tools/cibw_repair_wheel_command_windows.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
#!/usr/bin/env bash

set -e
set -x

WHEEL=$1
DEST_DIR=$2

python -m pip install delvewheel
python -m delvewheel show "$WHEEL" && python -m delvewheel repair -w "$DEST_DIR" "$WHEEL" --no-mangle-all
82 changes: 49 additions & 33 deletions .github/workflows/CI_pyHAMS.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,50 +16,63 @@ jobs:
fail-fast: false #true
matrix:
os: ["ubuntu-latest", "macOS-latest"] #, "windows-latest"]
python-version: ["3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11", "3.12"]

steps:
- name: Setup GNU Fortran
# if: false == contains( matrix.os, 'windows')
uses: awvwgk/setup-fortran@v1 #modflowpy/install-intelfortran-action@v1 #
if: false == contains( matrix.os, 'windows')
uses: awvwgk/setup-fortran@v1

- name: Install compiler
if: false == contains( matrix.os, 'windows')
id: install_cc
uses: rlalik/setup-cpp-compiler@master
with:
compiler: clang

- name: Install mingw-w64 on Windows
if: contains( matrix.os, 'windows')
uses: msys2/setup-msys2@v2
with:
path-type: inherit
update: true
install: >-
mingw-w64-x86_64-cmake
mingw-w64-x86_64-pkg-config
mingw-w64-x86_64-gcc
mingw-w64-x86_64-gcc-libs
mingw-w64-x86_64-gcc-fortran

- name: checkout repository
uses: actions/checkout@v4

- name: Mac openmp
if: contains( matrix.os, 'mac')
run: |
brew install llvm

- name: Set compilers
if: false == contains( matrix.os, 'windows')
run: |
echo "CC=${{ steps.install_cc.outputs.cc }}" >> $GITHUB_ENV
echo "CXX=${{ steps.install_cc.outputs.cxx }}" >> $GITHUB_ENV

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
uses: actions/setup-python@v5
id: cp
with:
python-version: ${{ matrix.python-version }}
update-environment: true

- name: Windows openblas
if: contains( matrix.os, 'windows')
run: |
#vcpkg install openblas
pacman -S mingw-w64-x86_64-openblas64 mingw-w64-lapack
#'${{ steps.cp.outputs.python-path }}' -m pip install pylib-openblas

- name: Mac openblas
if: contains( matrix.os, 'mac')
run: |
#export PKG_CONFIG_PATH="/usr/local/opt/openblas/lib/pkgconfig:$PKG_CONFIG_PATH"
sudo ln -s /usr/local/opt/openblas/lib/libopenblas* /usr/local/lib/

- name: Linux openblas
if: contains( matrix.os, 'ubuntu')
run: |
sudo apt-get install libopenblas-dev

#- name: Setup tmate session
# if: contains( matrix.os, 'windows')
# uses: mxschmitt/action-tmate@v3

- name: Pip Install pyHAMS
run: |
'${{ steps.cp.outputs.python-path }}' -m pip install --upgrade pip
'${{ steps.cp.outputs.python-path }}' -m pip install meson-python meson numpy ninja wheel
'${{ steps.cp.outputs.python-path }}' -m pip install meson-python meson numpy ninja wheel setuptools
'${{ steps.cp.outputs.python-path }}' -m pip install --no-build-isolation -e . -v

#- name: Setup tmate session
# if: contains( matrix.os, 'windows')
# uses: mxschmitt/action-tmate@v3

- name: Test run
run: |
Expand All @@ -78,7 +91,7 @@ jobs:
fail-fast: false #true
matrix:
os: ["ubuntu-latest", "macOS-latest", "windows-latest"]
python-version: ["3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11", "3.12"]

steps:
- name: checkout repository
Expand All @@ -96,19 +109,22 @@ jobs:
auto-activate-base: false

# Install dependencies of WISDEM specific to windows
# NOTE: If you install m2w64-toolchain, it includes m2w64-pkgconfig which supercedes the conda one
# The m2w64 pkgconfig fails to find openblas and the install fails
# By only includes these compiler packages, the build succeeds
- name: Add dependencies windows specific
if: contains( matrix.os, 'windows')
run: |
conda install -y m2w64-binutils m2w64-gcc m2w64-gcc-libgfortran m2w64-gcc-fortran libpython
conda install -y m2w64-toolchain libpython

# Install dependencies of WISDEM specific to windows
- name: Add dependencies unix specific
if: contains( matrix.os, 'ubuntu')
run: |
conda install -y libgomp
gfortran --version

- name: Add dependencies mac specific
if: contains( matrix.os, 'mac')
run: |
conda install -y compilers
conda install -y compilers llvm-openmp
gfortran --version

# Install
Expand Down
74 changes: 0 additions & 74 deletions .github/workflows/Publish_pyHAMS.inactive

This file was deleted.

125 changes: 125 additions & 0 deletions .github/workflows/Publish_pyHAMS.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
name: Build and upload to PyPI
# https://github.com/pypa/cibuildwheel/blob/main/examples/github-deploy.yml
# Best comparable example: https://github.com/pdfo/pdfo

# Build on every pull request (no need for every push) and release change:
on: [pull_request, release]

jobs:
build_wheels:
name: Build wheels on ${{ matrix.os }}
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-12, macos-13, macos-14]

steps:
- name: Set up QEMU
if: runner.os == 'Linux'
uses: docker/setup-qemu-action@v3
with:
platforms: all

- name: Setup GNU Fortran
if: false == contains( matrix.os, 'windows')
uses: awvwgk/setup-fortran@v1

- name: Install compiler
if: false == contains( matrix.os, 'windows')
id: install_cc
uses: rlalik/setup-cpp-compiler@master
with:
compiler: clang

- name: Install mingw-w64 on Windows
if: contains( matrix.os, 'windows')
uses: msys2/setup-msys2@v2
with:
path-type: inherit
install: |
mingw-w64-x86_64-gcc
mingw-w64-x86_64-gcc-fortran

- name: Checkout
uses: actions/checkout@v4

- name: Build wheels linux
if: contains( matrix.os, 'ubuntu')
uses: pypa/cibuildwheel@v2.18.1
env:
CC: ${{ steps.install_cc.outputs.cc }}
CXX: ${{ steps.install_cc.outputs.cxx }}

- name: Build wheels mac-12
if: contains( matrix.os, 'macos-12')
uses: pypa/cibuildwheel@v2.18.1
env:
CC: ${{ steps.install_cc.outputs.cc }}
CXX: ${{ steps.install_cc.outputs.cxx }}
CIBW_ENVIRONMENT_MACOS: MACOSX_DEPLOYMENT_TARGET="12.0"

- name: Build wheels mac-13
if: contains( matrix.os, 'macos-13')
uses: pypa/cibuildwheel@v2.18.1
env:
CC: ${{ steps.install_cc.outputs.cc }}
CXX: ${{ steps.install_cc.outputs.cxx }}
CIBW_ENVIRONMENT_MACOS: MACOSX_DEPLOYMENT_TARGET="13.0"

- name: Build wheels mac-14
if: contains( matrix.os, 'macos-14')
uses: pypa/cibuildwheel@v2.18.1
env:
CC: ${{ steps.install_cc.outputs.cc }}
CXX: ${{ steps.install_cc.outputs.cxx }}
CIBW_ENVIRONMENT_MACOS: MACOSX_DEPLOYMENT_TARGET="14.0"

- name: Build wheels windows
if: contains( matrix.os, 'windows')
uses: pypa/cibuildwheel@v2.18.1

- uses: actions/upload-artifact@v4
with:
name: cibw-wheels-${{ matrix.os }}-${{ strategy.job-index }}
path: ./wheelhouse/*.whl

build_sdist:
name: Build source distribution
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Build sdist
run: pipx run build --sdist

- uses: actions/upload-artifact@v4
with:
name: cibw-sdist
path: dist/*.tar.gz

upload_pypi:
needs: [build_wheels, build_sdist]
runs-on: ubuntu-latest
environment: pypi
permissions:
id-token: write # IMPORTANT: this permission is mandatory for trusted publishing
# upload to PyPI on every tag starting with 'v'
#if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')
# alternatively, to publish when a GitHub Release is created, use the following rule:
if: github.event_name == 'release' && github.event.action == 'published'
steps:
- uses: actions/download-artifact@v4
with:
# unpacks all CIBW artifacts into dist/
pattern: cibw-*
path: dist
merge-multiple: true

- name: Upload to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
#with:
# user: __token__
# password: ${{ secrets.pypi_password }}
# # To test: repository_url: https://test.pypi.org/legacy/

10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ There is cylinder test case that demonstrates usage and outputs in ``test/test_c

## Prerequisites

pyHAMS requires a Fortran compiler and OpenBLAS / MKL / LAPACK. The python wrapper currently supports GNU and Intel compilers. HAMS can be built with Flang, but that is not yet recognized by pyHAMS.
pyHAMS requires a Fortran compiler and OpenBLAS / MKL / LAPACK. We strongly recommend the [Miniforge](https://github.com/conda-forge/miniforge?tab=readme-ov-file#miniforge3) distribution of `conda` to satisfy package dependencies, as the traditional Anaconda distribution can be slow and struggle with mapping out dependencies.

## Install (as a library)

Expand All @@ -17,13 +17,17 @@ To install pyHAMS as a library that can be used by WEIS or RAFT in the backend,

## Install (from source)

If you would like to build the project locally from source for easier access to the underlying methods and tests, do:
If you would like to build the project locally from source for easier access to the underlying methods and tests, we still recommend using `conda` to satisfy dependencies.

$ git clone https://github.com/WISDEM/pyHAMS.git
$ cd pyHAMS
$ conda env create --name pyhams-env -f environment.yml
$ conda activate pyhams-env
$ conda install -y compilers # (Mac/Linux without other compilers)
$ conda install -y m2w64-toolchain libpython # (Windows only)
$ pip install .

If developer/editable mode, do the same `git clone` step, but on install do:
If developer/editable mode, replace the final step with:

$ pip install --no-build-isolation -e .

Expand Down
3 changes: 1 addition & 2 deletions environment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,7 @@ dependencies:
- meson-python
- ninja
- numpy
- openblas
- openmp
- pip
- pkg-config
- python
- setuptools
Loading