Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for writing nested dictionaries to HDF5 #91

Merged
merged 3 commits into from
Mar 6, 2024

Conversation

drewoldag
Copy link
Collaborator

@drewoldag drewoldag commented Mar 5, 2024

Initial attempt to add support for nested dictionaries with numpy array values.

Problem & Solution Description (including issue #)

The goal is to be able to write out to an hdf5 file a structure that mirrors a nested dictionary where the leaf values are all numpy arrays. The current approach introduces a bit of recursion into the writeDictToHdf5 file. Still need to write a good handful of tests.

The specific use case here should support the following:

def write_dict(filename, ensemble_dict, **kwargs):
        # ensemble_dict has the form { <str> : <qp.Ensemble> }
        # output_tables has the form { <str> : <tabularized qp.Ensemble> }
        # where a tabularized qp.Ensemble is a dictionary of dicts with leaf values that are numpy arrays.

        output_tables = {}
        for key, val in ensemble_dict.items():
            # check that val is a qp.Ensemble
            if not isinstance(val, Ensemble):
                raise ValueError("All values in ensemble_dict must be qp.Ensemble")

            output_tables[key] = val.build_tables()
        io.writeDictsToHdf5(output_tables, filename, **kwargs)

Code Quality

  • My code follows the code style of this project
  • I have written unit tests or justified all instances of #pragma: no cover; in the case of a bugfix, a new test that breaks as a result of the bug has been added
  • My code contains relevant comments and necessary documentation for future maintainers; the change is reflected in applicable demos/tutorials (with output cleared!) and added/updated docstrings use the NumPy docstring format
  • Any breaking changes, described above, are accompanied by backwards compatibility and deprecation warnings

Copy link

codecov bot commented Mar 5, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (4dadddb) to head (8475829).

Additional details and impacted files
@@            Coverage Diff            @@
##              main       #91   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            8         8           
  Lines          674       683    +9     
=========================================
+ Hits           674       683    +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@drewoldag drewoldag requested a review from eacharles March 5, 2024 21:46
@drewoldag drewoldag changed the title WIP - Initial attempt to add support for nested dictionaries Add support for writing nested dictionaries to HDF5 Mar 5, 2024
Copy link
Collaborator

@eacharles eacharles left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks

@drewoldag drewoldag merged commit 779098d into main Mar 6, 2024
8 checks passed
@drewoldag drewoldag deleted the awo/nested_dict_support branch March 6, 2024 23:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants