Skip to content

Release/2.0.0 #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 114 commits into from
Mar 19, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
114 commits
Select commit Hold shift + click to select a range
a5acf1e
Implement `DBEntry.get_sample` in IMASPy
maarten-ic Jul 9, 2024
fd49e02
Add docstrings for GetSliceParameters / GetSampleParameters
maarten-ic Jul 9, 2024
45aaaaa
Merge remote-tracking branch 'origin/develop' into feature/IMAS-3885-…
maarten-ic Nov 11, 2024
73268bb
Update tests for DD 4.0.0
maarten-ic Nov 11, 2024
bc9ea19
Update NCDBEntryImpl for get_sample and raise NotImplementedError
maarten-ic Nov 11, 2024
674460b
Set `begin_timerange_action` as available since AL core 5.4
maarten-ic Nov 11, 2024
0a47f94
Make the NC2IDS reader class-based
maarten-ic Nov 14, 2024
d59fcab
Add missing docstring
maarten-ic Nov 14, 2024
4d028e2
Merge remote-tracking branch 'origin/develop' into feature/IMAS-3885-…
maarten-ic Nov 15, 2024
ee385b7
Disable MDSplus backend tests for get_sample
maarten-ic Nov 15, 2024
d518a87
Pull request #239: Make the NC2IDS reader class-based
maarten-ic Nov 15, 2024
b007316
Add validation for the ids_properties.homogeneous time variable in ne…
maarten-ic Nov 13, 2024
f7be384
Additional validation checks and tests
maarten-ic Nov 15, 2024
e524646
Fix a bug with lazy loading multiple IDSs from the same HDF5 DBEntry
maarten-ic Nov 19, 2024
d47566b
Validate netCDF sparsity metadata
maarten-ic Nov 19, 2024
54d78d6
Add environment variable to disable netCDF file validation.
maarten-ic Nov 19, 2024
5ccae5d
Eliminate nc2ids function
maarten-ic Nov 19, 2024
afb8c29
Update `get_sample` docstring to clarify that the interpolation mode …
maarten-ic Nov 20, 2024
983a652
Pull request #241: Fix a bug with lazy loading multiple IDSs from the…
maarten-ic Nov 22, 2024
6cd1e44
Add `validate_nc` command to imaspy CLI
maarten-ic Nov 22, 2024
6ec21c7
Fix incorrect exception when using mode="x" for netCDF files
maarten-ic Nov 25, 2024
1f6c6fe
Close netCDF datasets when an exception is raised
maarten-ic Nov 25, 2024
b7d8963
Add unit tests for `nc_validate.py`
maarten-ic Nov 25, 2024
fc2cbf2
Additional documentation for the `imaspy validate_nc` command line tool
maarten-ic Nov 27, 2024
3c35fa9
Pull request #220: Implement `DBEntry.get_sample` in IMASPy
maarten-ic Dec 3, 2024
7c56b5f
Explicitly set `IDSDEF_PATH` when opening a DBEntry with the UDA backend
maarten-ic Dec 4, 2024
74d2e3a
Make prepare_data_dictionaries compatible with DD>4.0.0 (change in sc…
olivhoenen Dec 9, 2024
77fb044
Updating the license and readme
olivhoenen Dec 11, 2024
d80778f
Replace references to LICENSE.md to LICENSE.txt
maarten-ic Dec 12, 2024
bd098bb
Pull request #244: Add `validate_nc` command to imaspy CLI
maarten-ic Dec 12, 2024
d427972
Pull request #248: Replace references to LICENSE.md to LICENSE.txt
maarten-ic Dec 12, 2024
1968cde
Pull request #246: Make prepare_data_dictionaries compatible with DD>…
olivhoenen Dec 12, 2024
cef4667
Fix a bug with lazy loading
maarten-ic Dec 17, 2024
4beab9f
Add contributing guidelines
olivhoenen Jan 10, 2025
2eb385e
fixed readme.md for imas
gautamraj8044 Dec 13, 2024
133f78c
Apply suggestion on naming
olivhoenen Dec 17, 2024
6f871f5
Update README.md
gautamraj8044 Dec 17, 2024
693f035
Update CONTRIBUTING.md
maarten-ic Jan 20, 2025
b2913f6
renamed imaspy to imas
prasad-sawantdesai Jan 22, 2025
6e9d47f
added github actions and readthedocs conf
prasad-sawantdesai Jan 22, 2025
36a7d8b
configures setuptools_scm
prasad-sawantdesai Jan 22, 2025
42c3f91
removed dd_helpers_old
prasad-sawantdesai Jan 22, 2025
5b3d0fe
Modifications for compatibility with TORAX.
Nush395 Jan 20, 2025
c1f7a96
Modify tests to use fixture.
Nush395 Jan 20, 2025
abcaf3f
Update imaspy/test/test_ids_mixin.py
Nush395 Jan 21, 2025
aab7f66
Update imaspy/test/test_cli.py
Nush395 Jan 21, 2025
006580a
Add error message when attemtping to store complex number with netcdf…
Nush395 Jan 21, 2025
b2afe07
Formatting.
Nush395 Jan 21, 2025
816bbd4
Add tests for different versions of netcdf.
Nush395 Jan 21, 2025
2f7b591
Minor changes to tests.
Nush395 Jan 21, 2025
3591f17
Rename test.
Nush395 Jan 21, 2025
0d48b0b
Update numpy exception version change to 1.25.
Nush395 Jan 21, 2025
68c225d
Fix bug in skip logic of tests.
Nush395 Jan 21, 2025
4570e13
merged develop branch
prasad-sawantdesai Jan 22, 2025
00e1b38
fixed ci scripts and formatting
prasad-sawantdesai Jan 22, 2025
95d587b
Adding release notes for tag 1.2.0
olivhoenen Jan 22, 2025
835dccf
renaming of imas-python to IMAS-Python
prasad-sawantdesai Jan 23, 2025
f8ed355
fixed pytest and updated documentation references
prasad-sawantdesai Jan 23, 2025
07c2144
merged with develop
prasad-sawantdesai Jan 23, 2025
9d925a5
fixed readthedocs link
prasad-sawantdesai Jan 23, 2025
f53fb88
Apply suggestions from code review
prasad-sawantdesai Jan 23, 2025
438dc73
Few fixes in the documentation
olivhoenen Jan 24, 2025
c5b43c4
Merge pull request #14 from iterorganization/rename_imaspy_sources_to…
olivhoenen Jan 24, 2025
954bdb8
Publish package to pypi for tags only, and to testpypi for develop only
olivhoenen Jan 24, 2025
6186d46
fixed issue with imas-core dependency while installing pip install -e…
prasad-sawantdesai Feb 3, 2025
1d5b967
fixed create_model_ids_xml function
prasad-sawantdesai Feb 3, 2025
39c8db7
pass filename as string
prasad-sawantdesai Feb 3, 2025
cb3830a
Update imas/backends/imas_core/mdsplus_model.py
prasad-sawantdesai Feb 4, 2025
9a57d5c
Update imas/backends/imas_core/mdsplus_model.py
prasad-sawantdesai Feb 4, 2025
bc010ab
fixed review comments
prasad-sawantdesai Feb 4, 2025
1b4189d
Implement implicit DD version conversion in NetCDF backend
maarten-ic Feb 4, 2025
0c835a1
Improve performance of nc2ids by bypassing IDSPrimitive.value.setter …
maarten-ic Feb 4, 2025
7ab02c7
WIP: lazy loading for netCDF backend
maarten-ic Feb 4, 2025
e388694
keep ubuntu-22.04 because latest has Python 3.13 and saxonche is not …
prasad-sawantdesai Feb 5, 2025
eabbecf
Add tests for netcdf lazy loading and fix implementation bugs
maarten-ic Feb 5, 2025
962bc71
fixed Python version
prasad-sawantdesai Feb 5, 2025
a8a86d4
Merge pull request #16 from prasad-sawantdesai/fixes_imas_core_and_sa…
olivhoenen Feb 5, 2025
cfd9a56
Fix bug where untensorized data could be extracted as a 0D numpy arra…
maarten-ic Feb 5, 2025
1f5362e
Fix the thrown exception message when trying to load imas_core
olivhoenen Feb 6, 2025
a110d69
Add missing `lazy` argument to unit tests
maarten-ic Feb 7, 2025
ae9af2a
Merge branch 'develop' of https://github.com/iterorganization/imas-py…
maarten-ic Feb 7, 2025
62506e0
Update netcdf documentation
maarten-ic Feb 7, 2025
ee230cc
Merge pull request #17 from maarten-ic/feature/lazy-load-from-netcdf
maarten-ic Feb 10, 2025
02bcf9a
initial version
prasad-sawantdesai Feb 10, 2025
3587bc7
added matrix to run within different Python environment
prasad-sawantdesai Feb 10, 2025
430391d
fixed UTC issue in Python 3.8
prasad-sawantdesai Feb 11, 2025
6d95d00
timestamp converted to supported format for Python 3.8
prasad-sawantdesai Feb 11, 2025
36cebdf
Refactor iterators used in ids2nc and nc2ids
maarten-ic Mar 6, 2025
2429595
Refactor ids2nc, extract common tensorization logic in IDSTensorizer
maarten-ic Mar 7, 2025
09fb355
Implement `imas.util.to_xarray`
maarten-ic Mar 7, 2025
78b89f6
Add `xarray` as optional imas-python dependency
maarten-ic Mar 7, 2025
a56a5e7
Update xarray advanced course to mention imas.util.to_xarray
maarten-ic Mar 10, 2025
ce313de
Additional documentation for `imas.util.to_xarray`
maarten-ic Mar 10, 2025
dffeac7
Additional documentation and example for to_xarray
maarten-ic Mar 13, 2025
31b5385
Merge branch 'main' into release/2.0.0
olivhoenen Mar 17, 2025
49d87d4
Fixup merge conflicts
olivhoenen Mar 17, 2025
90e97b6
Fixup code after merge
olivhoenen Mar 17, 2025
2d93ab0
Updating changelog for release 2.0.0
olivhoenen Mar 17, 2025
f37b699
Fixup formatting in changelog
olivhoenen Mar 17, 2025
6151b37
removed pull_request event
prasad-sawantdesai Mar 17, 2025
062266a
added pull_request event with options
prasad-sawantdesai Mar 17, 2025
02c6387
Merge branch 'develop' into release/2.0.0
olivhoenen Mar 18, 2025
2e3b1b1
added workflow for sphinx doc generation and fixed docstrings
prasad-sawantdesai Mar 18, 2025
1d212a0
fix sphinx docstring issue
prasad-sawantdesai Mar 18, 2025
8f49f94
fixed formatting
prasad-sawantdesai Mar 18, 2025
1d0398e
fixed sphinx doc issue
prasad-sawantdesai Mar 18, 2025
125ba37
removed deactivate command from workflow
prasad-sawantdesai Mar 18, 2025
50eba56
Update imas/backends/netcdf/ids_tensorizer.py
prasad-sawantdesai Mar 18, 2025
aa18dcb
Update imas/backends/netcdf/nc2ids.py
prasad-sawantdesai Mar 18, 2025
84cb879
imas to imaspy for old version
prasad-sawantdesai Mar 18, 2025
cb64e55
Merge branch 'develop' into release/2.0.0
olivhoenen Mar 18, 2025
d81408c
Update the 5 min intro
olivhoenen Mar 18, 2025
8f7986d
Update documentation for release 2.0
olivhoenen Mar 19, 2025
0ecccbf
Update docs/source/changelog.rst
olivhoenen Mar 19, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .git_archival.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
node: $Format:%H$
node-date: $Format:%cI$
describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
2 changes: 1 addition & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
@@ -1 +1 @@
imaspy/_version.py export-subst
.git_archival.txt export-subst
34 changes: 34 additions & 0 deletions .github/workflows/linting.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: linting-and-code-formatting

on:
push:
pull_request:
types: [opened, synchronize, reopened]

jobs:
build:

runs-on: ubuntu-22.04

steps:
- name: Checkout IMAS-Python sources
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
# until saxonche is available in 3.13
# https://saxonica.plan.io/issues/6561
python-version: "<3.13"

- name: Display Python version
run: python -c "import sys; print(sys.version)"

- name: Install the code linting and formatting tools
run: pip install --upgrade 'black >=24,<25' flake8

- name: Check formatting of code with black
run: black --check imas

- name: Check linting with flake8
run: flake8 imas
74 changes: 74 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
name: build-wheel-and-publish-test-pypi

on:
push:
pull_request:
types: [opened, synchronize, reopened]

jobs:
build:
name: Build distribution
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python
uses: actions/setup-python@v5
with:
# until saxonche is available in 3.13
# https://saxonica.plan.io/issues/6561
python-version: "<3.13"
- name: Install pypa/build
run: >-
python3 -m pip install pip setuptools wheel build
- name: Build a binary wheel and a source tarball
run: python3 -m build .
- name: Store the distribution packages
uses: actions/upload-artifact@v4
with:
name: python-package-distributions
path: dist/

publish-to-pypi:
name: Publish IMAS-Python distribution to PyPI
if: startsWith(github.ref, 'refs/tags/') # only publish to PyPI on tag pushes
needs:
- build
runs-on: ubuntu-22.04
environment:
name: pypi
url: https://pypi.org/p/imas-python
permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing
steps:
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/
- name: Publish distribution to PyPI
uses: pypa/gh-action-pypi-publish@release/v1

publish-to-testpypi:
name: Publish IMAS-Python distribution to TestPyPI
if: github.ref=='refs/heads/develop' # only publish to TestPyPI on develop pushes
needs:
- build
runs-on: ubuntu-22.04
environment:
name: testpypi
url: https://test.pypi.org/p/imas-python
permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing
steps:
- name: Download all the dists
uses: actions/download-artifact@v4
with:
name: python-package-distributions
path: dist/
- name: Publish distribution to TestPyPI
uses: pypa/gh-action-pypi-publish@unstable/v1
with:
repository-url: https://test.pypi.org/legacy/
verbose: true
51 changes: 51 additions & 0 deletions .github/workflows/test_with_pytest.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
name: Test using pytest

on:
push:
pull_request:
types: [opened, synchronize, reopened]

jobs:
test:
runs-on: ubuntu-22.04
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11", "3.12"] # Test on multiple Python versions

steps:
- name: Checkout repository
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}

uses: actions/setup-python@v4
with:
# until saxonche is available in 3.13
# https://saxonica.plan.io/issues/6561
python-version: ${{ matrix.python-version }}
- name: Display Python version
run: python -c "import sys; print(sys.version)"

- name: Install dependencies
run: |
python -m venv venv
source venv/bin/activate
pip install --upgrade pip setuptools wheel
pip install .[test]

- name: Run tests
run: |
source venv/bin/activate
python -m pytest -n=auto --cov=imas --cov-report=term-missing --cov-report=xml:coverage.xml --cov-report=html:htmlcov --junit-xml=junit.xml

- name: Upload coverage report ${{ matrix.python-version }}
uses: actions/upload-artifact@v4
with:
name: coverage-report-${{ matrix.python-version }}
path: htmlcov

- name: Upload test report ${{ matrix.python-version }}
uses: actions/upload-artifact@v4
with:
name: test-report-${{ matrix.python-version }}
path: junit.xml
53 changes: 53 additions & 0 deletions .github/workflows/verify_sphinx_doc.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: verify-sphinx-doc-generation

on:
push:
pull_request:
types: [opened, synchronize, reopened]

jobs:
build-and-test:
runs-on: ubuntu-22.04

steps:
- name: Checkout IMAS-Python sources
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
with:
# until saxonche is available in 3.13
# https://saxonica.plan.io/issues/6561
python-version: "<3.13"

- name: Display Python version
run: python -c "import sys; print(sys.version)"


- name: Set up Python virtual environment
run: |
python -m venv venv
source venv/bin/activate

- name: Install build dependencies
run: |
pip install --upgrade pip setuptools wheel build

- name: Build package
run: |
rm -rf dist
python -m build .

- name: Install package and dependencies
run: |
pip install "$(readlink -f dist/*.whl)[docs,netcdf]"

- name: Debug dependencies
run: |
pip freeze

- name: Build Sphinx documentation
run: |
export SPHINXOPTS='-W -n --keep-going'
make -C docs clean html
13 changes: 4 additions & 9 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -91,21 +91,16 @@ ENV/
*.swo

# SCM setuptools
imaspy/version.py

# Saxon symlink or downloaded file
saxon*.jar
imas/_version.py

# IMAS DD
data-dictionary
access-layer
containers/arch/imaspy/
containers/arch/imas/
containers/arch/data-dictionary/
containers/arch/access-layer/
imaspy/assets/IDSDef.zip
imas/assets/IDSDef.zip

# IDS files
*.ids
# *.ids

# ASV folder
/.asv
23 changes: 23 additions & 0 deletions .readthedocs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
version: 2

build:
os: "ubuntu-22.04"
tools:
python: "3.11"
jobs:
post_checkout:
- git fetch --unshallow || true

python:
install:
- method: pip
path: .
extra_requirements:
- docs
- netcdf
- h5py

sphinx:
builder: html
configuration: docs/source/conf.py
fail_on_warning: false
20 changes: 10 additions & 10 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,36 @@
# Contributing guidelines

We welcome any kind of contribution to `imas-python`,
We welcome any kind of contribution to `IMAS-Python`,
from a simple comment, a question or even a full fledged pull
request.
Please first make sure you read and follow the
[Code of Conduct](CODE_OF_CONDUCT.md).

## You think you found a bug in the code, or have a question in its use
1. use the [issue search](https://github.com/iterorganization/imas-python/issues)
1. use the [issue search](https://github.com/iterorganization/IMAS-Python/issues)
to check if someone already created a similar issue;
3. if not, make a **new issue** to describe your problem or question.
2. if not, make a **new issue** to describe your problem or question.
In the case of a bug suspiscion, please try to give all the relevant
information to allow reproducing the error or identifying
its root cause (version of the imas-python, OS and relevant
its root cause (version of the IMAS-Python, OS and relevant
dependencies, snippet of code);
4. apply relevant labels to the issue.
3. apply relevant labels to the issue.

## You want to make or ask some change to the code
1. use the [issue search](https://github.com/iterorganization/imas-python/issues)
1. use the [issue search](https://github.com/iterorganization/IMAS-Python/issues)
to check if someone already proposed a similar idea/change;
3. if not, create a **new issue** to describe what change you would like to see
2. if not, create a **new issue** to describe what change you would like to see
implemented and specify it if you intend to work on it yourself or if some help
will be needed;
4. wait until some kind of consensus is reached about your idea being relevant,
3. wait until some kind of consensus is reached about your idea being relevant,
at which time the issue will be assigned (to you or someone else who can work on
this topic);
5. if you do the development yourself, fork the repository to your own Github
4. if you do the development yourself, fork the repository to your own Github
profile and create your own feature branch off of the latest develop commit.
Make sure to regularly sync your branch with the latest commits from `develop`
(find instructions
[here](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork));
6. when your development is ready, create a pull request (find instructions
5. when your development is ready, create a pull request (find instructions
[here](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request-from-a-fork)).


Expand Down
12 changes: 6 additions & 6 deletions MANIFEST.in
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
include imaspy/assets/IDSDef.zip
include imaspy/assets/IDSDef2MDSpreTree.xsl
include imaspy/assets/ITER_134173_106_equilibrium.ids
include imaspy/assets/ITER_134173_106_core_profiles.ids
include imaspy/assets/equilibrium.ids
include imaspy/assets/core_profiles.ids
include imas/assets/IDSDef.zip
include imas/assets/IDSDef2MDSpreTree.xsl
include imas/assets/ITER_134173_106_equilibrium.ids
include imas/assets/ITER_134173_106_core_profiles.ids
include imas/assets/equilibrium.ids
include imas/assets/core_profiles.ids
20 changes: 13 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# IMAS-Python

IMAS-Python is a pure-python library to handle arbitrarily nested data structures.
It is designed for, but not necessarily bound to, interacting with Interface
Data Structures (IDSs) as defined by the Integrated Modelling & Analysis Suite (IMAS)
IMAS-Python (formerly known as IMASPy its for versions < 2) is a pure-python library to handle
arbitrarily nested data structures. It is designed for, but not necessarily bound to, interacting
with Interface Data Structures (IDSs) as defined by the Integrated Modelling & Analysis Suite (IMAS)
Data Model.


Expand All @@ -11,7 +11,7 @@ Data Model.
Install steps are described in the documentation generated from `/docs/source/installing.rst`.

Documentation is autogenerated from the source using [Sphinx](http://sphinx-doc.org/)
and can be found at the [ITER sharepoint](https://sharepoint.iter.org/departments/POP/CM/IMDesign/Code%20Documentation/IMAS-doc/index.html)
and can be found at the [readthedocs](https://imas-python.readthedocs.io/en/latest/)

The documentation can be manually generated by installing sphinx and running:

Expand All @@ -28,19 +28,25 @@ factory = imas.IDSFactory()
equilibrium = factory.equilibrium()
print(equilibrium)

equilibrium.ids_properties.homogeneous_time = imas.ids_defs.IDS_TIME_MODE_HETEROGENEOUS
equilibrium.ids_properties.homogeneous_time = imas.ids_defs.IDS_TIME_MODE_HOMOGENEOUS
equilibrium.ids_properties.comment = "testing"
equilibrium.time = [0.01]

with imas.DBEntry("imas:hdf5?path=./testdb","w") as dbentry:
with imas.DBEntry("imas:hdf5?path=./testdb", "w") as dbentry:
dbentry.put(equilibrium)

# or without imas_core dependency
with imas.DBEntry("./test.nc", "w") as dbentry:
dbentry.put(equilibrium)

```

A quick 5 minutes introduction is available in the documentation generated from `/docs/sources/intro.rst`.


## Legal

IMAS-Python is Copyright 2020-2024 ITER Organization, Copyright 2020-2023 Karel Lucas van de
IMAS-Python is Copyright 2020-2025 ITER Organization, Copyright 2020-2023 Karel Lucas van de
Plassche <karelvandeplassche@gmail.com>, Copyright 2020-2022 Daan van Vugt <dvanvugt@ignitioncomputing.com>,
and Copyright 2020 Dutch Institute for Fundamental Energy Research <info@differ.nl>.
It is licensed under [LGPL 3.0](LICENSE.txt).
Expand Down
Loading
Loading