Release/2.0.0 by olivhoenen · Pull Request #25 · iterorganization/IMAS-Python

olivhoenen · 2025-03-17T08:32:36Z

It's time to prepare the next release (first one after renaming hence the bump in major version).

Todo

rebase on top of main
update changelog
ensure all tests pass on bamboo (w.r.t imas-core)
check if documentation builds successfully (i.e. without errors/warnings)
check if version of built package is set correctly

…IMASPy-get_sample

In preparation for validating the NC data

…IMASPy-get_sample

Feature not yet implemented, see IMAS-5593

Merge in IMAS/imaspy from feature/refactor-nc2ids to develop * commit 'd59fcabc29d5691004649bcf48016255010d7fa7': Add missing docstring Make the NC2IDS reader class-based

…tCDF IDSs.

Ensure lazy contexts belonging to a different IDS are always closed. See IMAS-5603 for more details.

…has no effect on the `ids.time` vector. See also https://git.iter.org/projects/IMAS/repos/al-matlab/pull-requests/29/overview?commentId=48957

… same HDF5 DBEntry Merge in IMAS/imaspy from bugfix/IMAS-5603 to develop * commit 'e5246464d588069af3f0f25e5a0e00d41d7fd4ef': Fix a bug with lazy loading multiple IDSs from the same HDF5 DBEntry

Also fix a couple of bugs when opening invalid netCDF files and raise a proper exception.

Merge in IMAS/imaspy from feature/IMAS-3885-IMASPy-get_sample to develop * commit 'afb8c292c6527f03354ac35fa43dfaebd8fb4ef3': Update `get_sample` docstring to clarify that the interpolation mode has no effect on the `ids.time` vector. Disable MDSplus backend tests for get_sample Set `begin_timerange_action` as available since AL core 5.4 Update NCDBEntryImpl for get_sample and raise NotImplementedError Update tests for DD 4.0.0 Add docstrings for GetSliceParameters / GetSampleParameters Implement `DBEntry.get_sample` in IMASPy

…hemas layout)

Merge in IMAS/imaspy from feature/cli_validate_nc to develop * commit 'fc2cbf20b3cf1c549761214109d3d89b4ee34091': Additional documentation for the `imaspy validate_nc` command line tool Add unit tests for `nc_validate.py` Close netCDF datasets when an exception is raised Fix incorrect exception when using mode="x" for netCDF files Add `validate_nc` command to imaspy CLI

olivhoenen · 2025-03-17T14:12:41Z

--> https://imas-python.readthedocs.io/en/release-2.0.0

maarten-ic · 2025-03-17T15:22:17Z

Looks good!

Some small things people may run into:

pytest imas --mini --ascii (on the installing page) will fail when people don't have imas-core installed. Would be good to add a note in the text about this.
The changelog for v1.1 mentions that imas-core can be installed with pip install 'imas[imas-core] @ git+ssh://git@github.com/iterorganization/imas-core.git'. I don't have access to this git repository but expect that it doesn't work -- even when you do have access to the repository: the pip command tries to install a package called imas with extra dependencies imas-core.
The docs have carefully removed almost any reference to IMASPy. Maybe we could be more explicit about the rename somewhere, so that https://imas-python.readthedocs.io/en/latest/ pops up in the search results when Googling (or Bing-ing, or DuckDuckGo-ing) "imaspy"?
5-minute introduction starts with printing the version, which is reported as 1.0.0 in the docs. Could we update that to 2.0.0?
Should the 5-minute introduction be updated to use DBEntry URIs, and show an example for netCDF files?

prasad-sawantdesai · 2025-03-17T16:05:56Z

Just enabled ITER CI for IMAS-Python GitHub Repo. I found following warnings in sphinx doc generation

What's new in IMAS-Python 1.2.0
--------------------------
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/changelog.rst:12: WARNING: py:func reference target not found: imaspy.DBEntry.get_sample
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.imas_core.al_context.LazyALArrayStructChildContext.rst:30:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.imas_core.al_context.LazyALArrayStructChildContext.get_child
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.imas_core.al_context.LazyALArrayStructContext.rst:31:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.imas_core.al_context.LazyALArrayStructContext.get_child
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.imas_core.al_context.LazyALContext.rst:30:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.imas_core.al_context.LazyALContext.get_child
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.netcdf.ids2nc.IDS2NC.rst:33:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.netcdf.ids2nc.IDS2NC.tensorize
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.netcdf.ids_tensorizer.IDSTensorizer.rst:29:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.netcdf.ids_tensorizer.IDSTensorizer.tensorize
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.netcdf.nc2ids.LazyArrayStructContext.rst:22:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.netcdf.nc2ids.LazyArrayStructContext.get_child
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.netcdf.nc2ids.LazyArrayStructContext.rst:22:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.netcdf.nc2ids.LazyArrayStructContext.get_context
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.netcdf.nc2ids.LazyArrayStructContext.rst:22:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.netcdf.nc2ids.LazyArrayStructContext.iterate_to_index
/opt/bamboo-agent-home/xml-data/build-dir/IC-IG4-BDD/docs/source/generated/imas.backends.netcdf.nc2ids.LazyContext.rst:20:<autosummary>:1: WARNING: py:obj reference target not found: imas.backends.netcdf.nc2ids.LazyContext.get_child
make: *** [Makefile:23: html] Error 1

prasad-sawantdesai · 2025-03-18T14:23:45Z

---> ensure all tests pass on bamboo (w.r.t imas-core) passed
https://ci.iter.org/browse/IC-IG4-TWA5-4

prasad-sawantdesai · 2025-03-18T14:26:26Z

----> check if documentation builds successfully (i.e. without errors/warnings)
https://github.com/iterorganization/IMAS-Python/actions/runs/13924507015

prasad-sawantdesai · 2025-03-18T14:58:46Z

------> check if version of built package is set correctly

Creating distribution package 
(myenv) [sawantp1@sdcc-login02 dist]$ ll
total 37216
-rw-r--r--. 1 sawantp1 sawantp1 29372938 Mar 18 15:43 imas_python-1.2.1.dev94-py3-none-any.whl
-rw-r--r--. 1 sawantp1 sawantp1 29720207 Mar 18 15:42 imas_python-1.2.1.dev94.tar.gz


With Tarball
(myenv) [sawantp1@sdcc-login02 IMAS-Python-release-2.0.0]$ python
Python 3.11.5 (main, Feb  9 2024, 19:54:29) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import imas
15:55:17 CRITICAL Could not import 'imas_core': No module named 'imas_core'. Some functionality is not available. @imas_interface.py:34
>>> imas.__version__
'1.2.1.dev94'
>>>

Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>

olivhoenen · 2025-03-18T18:18:00Z

Looks good!

Some small things people may run into:

pytest imas --mini --ascii (on the installing page) will fail when people don't have imas-core installed. Would be good to add a note in the text about this.

The changelog for v1.1 mentions that imas-core can be installed with pip install 'imas[imas-core] @ git+ssh://git@github.com/iterorganization/imas-core.git'. I don't have access to this git repository but expect that it doesn't work -- even when you do have access to the repository: the pip command tries to install a package called imas with extra dependencies imas-core.

The docs have carefully removed almost any reference to IMASPy. Maybe we could be more explicit about the rename somewhere, so that https://imas-python.readthedocs.io/en/latest/ pops up in the search results when Googling (or Bing-ing, or DuckDuckGo-ing) "imaspy"?

5-minute introduction starts with printing the version, which is reported as 1.0.0 in the docs. Could we update that to 2.0.0?

Should the 5-minute introduction be updated to use DBEntry URIs, and show an example for netCDF files?

I'm updating the 5 min intro, few other stuff to modify w.r.t DD4.
By the way @maarten-ic, the behavior for this

>>> core_profiles.time = 1-1j
TypeError: can't convert complex to float

now complains about scalar to array conversion (while I could have assign [1-1j])

>>> core_profiles.time = 1-1j
19:15:10 INFO     Assigning incorrect type 'complex128' to <IDSNumericArray (IDS:core_profiles, time, FLT_1D)>
ValueError: Trying to assign a 0D value to <IDSNumericArray (IDS:core_profiles, time, FLT_1D)>

--> expected?

maarten-ic · 2025-03-19T07:38:47Z

I'm updating the 5 min intro, few other stuff to modify w.r.t DD4. By the way @maarten-ic, the behavior for this
>>> core_profiles.time = 1-1j
TypeError: can't convert complex to float
now complains about scalar to array conversion (while I could have assign [1-1j])
>>> core_profiles.time = 1-1j
19:15:10 INFO     Assigning incorrect type 'complex128' to <IDSNumericArray (IDS:core_profiles, time, FLT_1D)>
ValueError: Trying to assign a 0D value to <IDSNumericArray (IDS:core_profiles, time, FLT_1D)>
--> expected?

Mmm, yes, that was changed a while ago indeed. IMAS-Python prints an INFO statement and numpy does give a warning (once) when assigning a complex array to a FLT_1D:

>>> cp.time = [1-1j]
08:26:46 INFO     Assigning incorrect type 'complex128' to <IDSNumericArray (IDS:core_profiles, time, empty FLT_1D)>, attempting automatic conversion. @ids_primitive.py:483
/home/maarten/projects/iter-python/imaspy_12/imaspy/ids_primitive.py:484: ComplexWarning: Casting complex values to real discards the imaginary part
  value = np.asarray(
>>> cp.time
<IDSNumericArray (IDS:core_profiles, time, FLT_1D)>
numpy.ndarray([1.])
>>> cp.time = [1-1j]
08:27:03 INFO     Assigning incorrect type 'complex128' to <IDSNumericArray (IDS:core_profiles, time, FLT_1D)>
numpy.ndarray([1.]), attempting automatic conversion. @ids_primitive.py:483

I think this behavior is fine.

docs/source/intro.rst

maarten-ic

One minor comment, nothing else from my side at this moment.

docs/source/changelog.rst

Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>

maarten-ic and others added 30 commits July 9, 2024 13:55

Implement DBEntry.get_sample in IMASPy

a5acf1e

Add docstrings for GetSliceParameters / GetSampleParameters

fd49e02

Merge remote-tracking branch 'origin/develop' into feature/IMAS-3885-…

45aaaaa

…IMASPy-get_sample

Update tests for DD 4.0.0

73268bb

Update NCDBEntryImpl for get_sample and raise NotImplementedError

bc9ea19

Set begin_timerange_action as available since AL core 5.4

674460b

Make the NC2IDS reader class-based

0a47f94

In preparation for validating the NC data

Add missing docstring

d59fcab

Merge remote-tracking branch 'origin/develop' into feature/IMAS-3885-…

4d028e2

…IMASPy-get_sample

Disable MDSplus backend tests for get_sample

ee385b7

Feature not yet implemented, see IMAS-5593

Pull request #239: Make the NC2IDS reader class-based

d518a87

Merge in IMAS/imaspy from feature/refactor-nc2ids to develop * commit 'd59fcabc29d5691004649bcf48016255010d7fa7': Add missing docstring Make the NC2IDS reader class-based

Add validation for the ids_properties.homogeneous time variable in ne…

b007316

…tCDF IDSs.

Additional validation checks and tests

f7be384

Fix a bug with lazy loading multiple IDSs from the same HDF5 DBEntry

e524646

Ensure lazy contexts belonging to a different IDS are always closed. See IMAS-5603 for more details.

Validate netCDF sparsity metadata

d47566b

Add environment variable to disable netCDF file validation.

54d78d6

Eliminate nc2ids function

5ccae5d

Update get_sample docstring to clarify that the interpolation mode …

afb8c29

…has no effect on the `ids.time` vector. See also https://git.iter.org/projects/IMAS/repos/al-matlab/pull-requests/29/overview?commentId=48957

Pull request #241: Fix a bug with lazy loading multiple IDSs from the…

983a652

… same HDF5 DBEntry Merge in IMAS/imaspy from bugfix/IMAS-5603 to develop * commit 'e5246464d588069af3f0f25e5a0e00d41d7fd4ef': Fix a bug with lazy loading multiple IDSs from the same HDF5 DBEntry

Add validate_nc command to imaspy CLI

6cd1e44

Also fix a couple of bugs when opening invalid netCDF files and raise a proper exception.

Fix incorrect exception when using mode="x" for netCDF files

6ec21c7

Close netCDF datasets when an exception is raised

1f6c6fe

Add unit tests for nc_validate.py

b7d8963

Additional documentation for the imaspy validate_nc command line tool

fc2cbf2

Explicitly set IDSDEF_PATH when opening a DBEntry with the UDA backend

7c56b5f

Make prepare_data_dictionaries compatible with DD>4.0.0 (change in sc…

74d2e3a

…hemas layout)

Updating the license and readme

77fb044

Replace references to LICENSE.md to LICENSE.txt

d80778f

Fixup formatting in changelog

f37b699

olivhoenen unassigned maarten-ic Mar 17, 2025

prasad-sawantdesai and others added 3 commits March 18, 2025 08:39

removed pull_request event

6151b37

added pull_request event with options

062266a

Merge branch 'develop' into release/2.0.0

02c6387

prasad-sawantdesai and others added 9 commits March 18, 2025 17:39

added workflow for sphinx doc generation and fixed docstrings

2e3b1b1

fix sphinx docstring issue

1d212a0

fixed formatting

8f49f94

fixed sphinx doc issue

1d0398e

removed deactivate command from workflow

125ba37

Update imas/backends/netcdf/ids_tensorizer.py

50eba56

Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>

Update imas/backends/netcdf/nc2ids.py

aa18dcb

Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>

imas to imaspy for old version

84cb879

Merge branch 'develop' into release/2.0.0

cb64e55

Update the 5 min intro

d81408c

maarten-ic reviewed Mar 19, 2025

View reviewed changes

docs/source/intro.rst Outdated Show resolved Hide resolved

Update documentation for release 2.0

8f7986d

maarten-ic approved these changes Mar 19, 2025

View reviewed changes

docs/source/changelog.rst Outdated Show resolved Hide resolved

Update docs/source/changelog.rst

0ecccbf

Co-authored-by: Maarten Sebregts <110895564+maarten-ic@users.noreply.github.com>

olivhoenen temporarily deployed to pypi March 19, 2025 12:41 — with GitHub Actions Inactive

olivhoenen merged commit 27d7cd6 into main Mar 19, 2025
30 checks passed

olivhoenen deleted the release/2.0.0 branch March 21, 2025 07:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release/2.0.0#25

Release/2.0.0#25
olivhoenen merged 114 commits intomainfrom
release/2.0.0

olivhoenen commented Mar 17, 2025 •

edited by prasad-sawantdesai

Loading

Uh oh!

olivhoenen commented Mar 17, 2025

Uh oh!

maarten-ic commented Mar 17, 2025

Uh oh!

prasad-sawantdesai commented Mar 17, 2025

Uh oh!

prasad-sawantdesai commented Mar 18, 2025

Uh oh!

prasad-sawantdesai commented Mar 18, 2025

Uh oh!

prasad-sawantdesai commented Mar 18, 2025

Uh oh!

olivhoenen commented Mar 18, 2025

Uh oh!

maarten-ic commented Mar 19, 2025

Uh oh!

Uh oh!

maarten-ic left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

olivhoenen commented Mar 17, 2025 • edited by prasad-sawantdesai Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Todo

Uh oh!

olivhoenen commented Mar 17, 2025

Uh oh!

maarten-ic commented Mar 17, 2025

Uh oh!

prasad-sawantdesai commented Mar 17, 2025

Uh oh!

prasad-sawantdesai commented Mar 18, 2025

Uh oh!

prasad-sawantdesai commented Mar 18, 2025

Uh oh!

prasad-sawantdesai commented Mar 18, 2025

Uh oh!

olivhoenen commented Mar 18, 2025

Uh oh!

maarten-ic commented Mar 19, 2025

Uh oh!

Uh oh!

maarten-ic left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

olivhoenen commented Mar 17, 2025 •

edited by prasad-sawantdesai

Loading