Skip to content

Added partial support for compact datasets.#107

Merged
valeriupredoi merged 3 commits intoNCAS-CMS:mainfrom
zequihg50:main
Oct 2, 2025
Merged

Added partial support for compact datasets.#107
valeriupredoi merged 3 commits intoNCAS-CMS:mainfrom
zequihg50:main

Conversation

@zequihg50
Copy link
Contributor

Description

Added partial support for compact datasets (currently NotImplementedError is raised). This supports the data layout messages v3 and v4, v1 and v2 still pending but I can work out this in the near future. GitHub does not allow to attach a real file but I have included tests for a custom compact variable file.

Checklist

  • This pull request has a descriptive title and labels
  • This pull request has a minimal description (most was discussed in the issue, but a two-liner description is still desirable)
  • Unit tests have been added (if codecov test fails)
  • Any changed dependencies have been added or removed correctly (if need be)
  • If you are working on the documentation, please ensure the current build passes
  • All tests pass

@codecov
Copy link

codecov bot commented Sep 30, 2025

Codecov Report

❌ Patch coverage is 60.71429% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.73%. Comparing base (fefabba) to head (df0f8ec).
⚠️ Report is 190 commits behind head on main.

Files with missing lines Patch % Lines
pyfive/h5d.py 60.71% 6 Missing and 5 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #107      +/-   ##
==========================================
+ Coverage   72.58%   72.73%   +0.14%     
==========================================
  Files          11       11              
  Lines        2499     2531      +32     
  Branches      379      389      +10     
==========================================
+ Hits         1814     1841      +27     
- Misses        583      584       +1     
- Partials      102      106       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Collaborator

@bnlawrence bnlawrence left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is great, thanks @zequihg50! Do you think you could just move your compact support into dedicated private functions? I think that would be cleaner.

pyfive/h5d.py Outdated
match self.layout_class: # noqa
case 0: #compact storage
raise NotImplementedError("Compact Storage")
self.data = None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be more consistent with the rest of the code if we pushed the compact handling into a method called from here.

pyfive/h5d.py Outdated
match self.layout_class: # noqa
case 0: #compact storage
raise NotImplementedError("Compact Storage")
if self.data is None:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similarly, this functionality would be better in a method.

@kmuehlbauer
Copy link
Collaborator

kmuehlbauer commented Oct 2, 2025

@zequihg50 Thanks for these additions, could come in handy in h5netcdf, too.

Would it make sense to adapt the testing scheme to current state of #108? That way we do not need to ingest new data files into git.

@zequihg50
Copy link
Contributor Author

Hi @kmuehlbauer, I have refactored the test to meet the new testing scheme. Please feel free to make any changes if that doesn't fully comply.

@kmuehlbauer
Copy link
Collaborator

@zequihg50 + 💯 for containing within one test file.

@valeriupredoi
Copy link
Collaborator

many thanks @zequihg50 and cheers to @bnlawrence and @kmuehlbauer for reviewing! @bnlawrence do you mind approving your stale review, please? 🍻

@valeriupredoi valeriupredoi merged commit ce51787 into NCAS-CMS:main Oct 2, 2025
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants