Skip to content

Commit de4452c

Browse files
jlarsen-usgsjlarsen-usgs
authored
modflowapi interface (MODFLOW-ORG#8)
* update package: manual variable address assembly updated to use xmipy get_variable_addr() * update additional manual variable address assembly statements * Refactor code and added functionality: * add stress_period_start, stress_period_end Callbacks * fix ApiModel __repr__ * added Exchanges, TDIS, ATS, and SLN support * added ScalarInput and ScalarPackage support * update autotests * added parallel testing support through pytest-xdist * updated markers and split the extensions tests from the mf6 examples tests * added a test for ATS * update setup.cfg * update ci.yml * update(ListInput): add auxvar to stress_period_data when auxiliary variables are used * Allow None to be passed to stress_period_data.values to disable stresses for a package * updates: ApiModel, ApiSimulation, run_simulation * added a `totim` property on `ApiSimulation` and `ApiModel` * added docstrings to ApiModel property methods * updated termination message in run_simulation * added a finalize callback to Callbacks and run_simulation * add support for AUXNAME_CST * add(Head Monitor Example): Add a head monitor example application * ApiModel: adjust X based on nodetouser * ApiPackage: enforce lower cased variable names in get_advanced_var * ArrayPointer: trap for arrays that are not adjusted by reduced node numbers (ex. idomain) * update setup.cfg * try reformatting the xmipy installation instructions
1 parent fe1289b commit de4452c

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

96 files changed

+8932
-62
lines changed

‎.github/workflows/ci.yml

Lines changed: 116 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -26,10 +26,10 @@ jobs:
2626

2727
# check out repo
2828
- name: Checkout repo
29-
uses: actions/checkout@v2.3.4
29+
uses: actions/checkout@v3
3030

3131
- name: Setup Python 3.8
32-
uses: actions/setup-python@v2
32+
uses: actions/setup-python@v4
3333
with:
3434
python-version: 3.8
3535

@@ -39,15 +39,15 @@ jobs:
3939
4040
- name: Base installation
4141
run: |
42-
pip install . --use-feature=in-tree-build
43-
42+
pip install -e .
43+
4444
- name: Print version
4545
run: |
4646
python -c "import modflowapi; print(modflowapi.__version__)"
4747
4848
4949
lint:
50-
name: linting
50+
name: lint
5151
runs-on: ubuntu-latest
5252
strategy:
5353
fail-fast: false
@@ -59,24 +59,27 @@ jobs:
5959
steps:
6060
# check out repo
6161
- name: Checkout repo
62-
uses: actions/checkout@v2.3.4
62+
uses: actions/checkout@v3
6363

64-
# Standard python fails on windows without GDAL installation. Using
64+
# Standard python fails on Windows without GDAL installation. Using
6565
# standard python here since only linting on linux.
6666
# Use standard bash shell with standard python
6767
- name: Setup Python 3.8
68-
uses: actions/setup-python@v2
68+
uses: actions/setup-python@v4
6969
with:
7070
python-version: 3.8
71+
cache: 'pip'
72+
cache-dependency-path: setup.cfg
7173

7274
- name: Print python version
7375
run: |
7476
python --version
7577
76-
- name: Install Python 3.8 packages
78+
- name: Install Python dependencies
7779
run: |
7880
python -m pip install --upgrade pip
79-
pip install -r etc/requirements.pip.txt
81+
pip install .
82+
pip install ".[lint]"
8083
8184
- name: Run black
8285
run: |
@@ -93,3 +96,106 @@ jobs:
9396
run: |
9497
pylint --jobs=2 --errors-only --exit-zero ./modflowapi
9598
99+
autotest_extensions:
100+
name: modflowapi extensions autotests
101+
needs: lint
102+
runs-on: ${{ matrix.os }}
103+
strategy:
104+
fail-fast: false
105+
matrix:
106+
os: [ ubuntu-latest, macos-latest, windows-latest ]
107+
python-version: [ 3.8, 3.9, "3.10"]
108+
defaults:
109+
run:
110+
shell: bash
111+
112+
steps:
113+
# check out repo
114+
- name: Checkout repo
115+
uses: actions/checkout@v3
116+
117+
- name: Setup Python
118+
uses: actions/setup-python@v4
119+
with:
120+
python-version: ${{ matrix.python-version }}
121+
cache: 'pip'
122+
cache-dependency-path: setup.cfg
123+
124+
- name: Install Python dependencies
125+
if: runner.os != 'Windows'
126+
run: |
127+
pip install --upgrade pip
128+
pip install .
129+
pip install ".[test]"
130+
131+
- name: Install Python dependencies Windows
132+
if: runner.os == 'Windows'
133+
run: |
134+
python.exe -m pip install --upgrade pip
135+
python.exe -m pip install .
136+
python.exe -m pip install ".[test]"
137+
138+
- name: Install modflow executables
139+
uses: modflowpy/install-modflow-action@v1
140+
with:
141+
path: ~/work/modflowapi/modflowapi/autotest
142+
repo: modflow6-nightly-build
143+
144+
- name: Run autotests
145+
working-directory: ./autotest
146+
shell: bash -l {0}
147+
run: |
148+
# chmod a+x libmf6*
149+
pytest -n auto -m "not mf6"
150+
151+
autotest_mf6_examples:
152+
name: modflowapi mf6 examples autotests
153+
needs: lint
154+
runs-on: ${{ matrix.os }}
155+
strategy:
156+
fail-fast: false
157+
matrix:
158+
os: [ ubuntu-latest, macos-latest, windows-latest ]
159+
python-version: [ 3.8, 3.9, "3.10"]
160+
defaults:
161+
run:
162+
shell: bash
163+
164+
steps:
165+
# check out repo
166+
- name: Checkout repo
167+
uses: actions/checkout@v3
168+
169+
- name: Setup Python
170+
uses: actions/setup-python@v4
171+
with:
172+
python-version: ${{ matrix.python-version }}
173+
cache: 'pip'
174+
cache-dependency-path: setup.cfg
175+
176+
- name: Install Python dependencies
177+
if: runner.os != 'Windows'
178+
run: |
179+
pip install --upgrade pip
180+
pip install .
181+
pip install ".[test]"
182+
183+
- name: Install Python dependencies Windows
184+
if: runner.os == 'Windows'
185+
run: |
186+
python.exe -m pip install --upgrade pip
187+
python.exe -m pip install .
188+
python.exe -m pip install ".[test]"
189+
190+
- name: Install modflow executables
191+
uses: modflowpy/install-modflow-action@v1
192+
with:
193+
path: ~/work/modflowapi/modflowapi/autotest
194+
repo: modflow6-nightly-build
195+
196+
- name: Run autotests
197+
working-directory: ./autotest
198+
shell: bash -l {0}
199+
run: |
200+
# chmod a+x libmf6*
201+
pytest -n auto -m "mf6 and not extensions"

‎README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,8 @@
22

33
`modflowapi` is an extension to [xmipy](https://pypi.org/project/xmipy/) for the MODFLOW API.
44

5-
The `modflowapi` can be used to access functionality in the eXtended Model Interface (XMI) wrapper (XmiWrapper) and additional functionality specific to the MODFLOW API. Currently it is a joint development of the USGS and Deltares.
5+
The `modflowapi` can be used to access functionality in the eXtended Model Interface (XMI) wrapper (XmiWrapper)
6+
and additional functionality specific to the MODFLOW API. Currently it is a joint development of the USGS and Deltares.
67

78
`modflowapi` can be installed by running
89
```

‎autotest/__init__.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# init file for pytest

‎autotest/conftest.py

Lines changed: 138 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,138 @@
1+
from itertools import groupby
2+
from os import linesep
3+
from pathlib import Path
4+
from tempfile import gettempdir
5+
6+
import pytest
7+
from filelock import FileLock
8+
9+
10+
__mf6_examples = "mf6_examples"
11+
__mf6_examples_path = Path(gettempdir()) / __mf6_examples
12+
__mf6_examples_lock = FileLock(Path(gettempdir()) / f"{__mf6_examples}.lock")
13+
14+
15+
def get_mf6_examples_path() -> Path:
16+
pytest.importorskip("modflow_devtools")
17+
from modflow_devtools.download import download_and_unzip
18+
19+
# use file lock so mf6 distribution is downloaded once,
20+
# even when tests are run in parallel with pytest-xdist
21+
__mf6_examples_lock.acquire()
22+
try:
23+
if not __mf6_examples_path.is_dir():
24+
__mf6_examples_path.mkdir(exist_ok=True)
25+
download_and_unzip(
26+
url="https://github.com/MODFLOW-USGS/modflow6-examples/releases/download/current/modflow6-examples.zip",
27+
path=__mf6_examples_path,
28+
verbose=True,
29+
)
30+
return __mf6_examples_path
31+
finally:
32+
__mf6_examples_lock.release()
33+
34+
def is_nested(namfile) -> bool:
35+
p = Path(namfile)
36+
if not p.is_file() or not p.name.endswith(".nam"):
37+
raise ValueError(f"Expected a namfile path, got {p}")
38+
39+
return p.parent.parent.name != __mf6_examples
40+
41+
42+
def pytest_generate_tests(metafunc):
43+
# examples to skip:
44+
# - ex-gwtgwt-mt3dms-p10: https://github.com/MODFLOW-USGS/modflow6/pull/1008
45+
exclude = ["ex-gwt-gwtgwt-mt3dms-p10"]
46+
namfiles = [
47+
str(p)
48+
for p in get_mf6_examples_path().rglob("mfsim.nam")
49+
if not any(e in str(p) for e in exclude)
50+
]
51+
52+
# parametrization by model
53+
# - single namfile per test case
54+
# - no coupling (only first model in each simulation subdir is used)
55+
key = "mf6_example_namfile"
56+
if key in metafunc.fixturenames:
57+
metafunc.parametrize(key, sorted(namfiles))
58+
59+
# parametrization by simulation
60+
# - each test case gets an ordered list of 1+ namfiles
61+
# - models can be coupled (run in order provided, sharing workspace)
62+
key = "mf6_example_namfiles"
63+
if key in metafunc.fixturenames:
64+
simulations = []
65+
66+
def simulation_name_from_model_path(p):
67+
p = Path(p)
68+
return p.parent.parent.name if is_nested(p) else p.parent.name
69+
70+
for model_name, model_namfiles in groupby(
71+
namfiles, key=simulation_name_from_model_path
72+
):
73+
models = sorted(
74+
list(model_namfiles)
75+
) # sort in alphabetical order (gwf < gwt)
76+
simulations.append(models)
77+
print(
78+
f"Simulation {model_name} has {len(models)} model(s):\n"
79+
f"{linesep.join(model_namfiles)}"
80+
)
81+
82+
def simulation_name_from_model_namfiles(mnams):
83+
try:
84+
namfile = next(iter(mnams), None)
85+
except TypeError:
86+
namfile = None
87+
if namfile is None:
88+
pytest.skip("No namfiles (expected ordered collection)")
89+
namfile = Path(namfile)
90+
return (
91+
namfile.parent.parent if is_nested(namfile) else namfile.parent
92+
).name
93+
94+
metafunc.parametrize(
95+
key, simulations, ids=simulation_name_from_model_namfiles
96+
)
97+
98+
99+
@pytest.fixture(scope="function")
100+
def tmpdir(tmpdir_factory, request) -> Path:
101+
node = (
102+
request.node.name.replace("/", "_")
103+
.replace("\\", "_")
104+
.replace(":", "_")
105+
)
106+
temp = Path(tmpdir_factory.mktemp(node))
107+
yield Path(temp)
108+
109+
keep = request.config.getoption("--keep")
110+
if keep:
111+
copytree(temp, Path(keep) / temp.name)
112+
113+
keep_failed = request.config.getoption("--keep-failed")
114+
if keep_failed and request.node.rep_call.failed:
115+
copytree(temp, Path(keep_failed) / temp.name)
116+
117+
118+
def pytest_addoption(parser):
119+
parser.addoption(
120+
"-K",
121+
"--keep",
122+
action="store",
123+
default=None,
124+
help="Move the contents of temporary test directories to correspondingly named subdirectories at the given "
125+
"location after tests complete. This option can be used to exclude test results from automatic cleanup, "
126+
"e.g. for manual inspection. The provided path is created if it does not already exist. An error is "
127+
"thrown if any matching files already exist.",
128+
)
129+
130+
parser.addoption(
131+
"--keep-failed",
132+
action="store",
133+
default=None,
134+
help="Move the contents of temporary test directories to correspondingly named subdirectories at the given "
135+
"location if the test case fails. This option automatically saves the outputs of failed tests in the "
136+
"given location. The path is created if it doesn't already exist. An error is thrown if files with the "
137+
"same names already exist in the given location.",
138+
)

‎autotest/pytest.ini

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
[pytest]
2+
addopts = -ra
3+
python_files =
4+
test_*.py
5+
markers =
6+
mf6: tests for modflow 6 examples
7+
extensions: tests for modflowapi extensions

0 commit comments

Comments
 (0)