Skip to content

Sledgehammer pass at refactoring units #1552

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 30 commits into from
Mar 26, 2025
Merged

Sledgehammer pass at refactoring units #1552

merged 30 commits into from
Mar 26, 2025

Conversation

thorbjoernl
Copy link
Collaborator

@thorbjoernl thorbjoernl commented Mar 20, 2025

Change Summary

A very rough attempt at collecting units related code in a new units submodule. For now this is mainly focused on moving existing code, not improving the code (I'll leave that to a second PR to make the reviews manageable). That being said, I've adressed a couple of low hanging fruits:

  • Simplify the implementation of comparison operators in TsType using Python's @total_ordering decorator.
  • Simplify sort_ts_types, get_lowest_resolution, get_highest_resolution using Python's built in sorted().
  • Using Python's calendar.isleap() to avoid Y2.1k problems :P
  • Remove _concprcp_units_helpers.py since it isn't used (?). Most functions have raised NotImplementedErrors for ~3 years.
  • Implement a MolecularMass class to represent molar mass.
  • Move imports out of functions as then it will fail at runtime when that code is encountered instead of failing at import.

Overview of moved things

  • pyaerocom/units_helpers.py -> pyaerocom/units/units_helpers.py
  • Units related code in pyaercom/helpers.py split between pyaercom/units/datetime/utils.py and pyaerocom/units/helpers.py
  • pyaerocom/time_config.py -> pyaerocom/units/datetime/_time_config.py
  • pyaerocom/molmasses.py -> pyaerocom/units/molecularmass.py
  • Most other touched files are just changes in imports.

Related issue number

related to #1512

Checklist

  • Start with a draft-PR
  • The PR title is a good summary of the changes
  • PR is set to AeroTools and a tentative milestone
  • Documentation reflects the changes where applicable
  • Tests for the changes exist where applicable
  • Tests pass locally
  • Tests pass on CI
  • At least 1 reviewer is selected
  • Make PR ready to review

@thorbjoernl thorbjoernl added this to the m2025-04 milestone Mar 20, 2025
Copy link

codecov bot commented Mar 20, 2025

Codecov Report

Attention: Patch coverage is 85.39326% with 52 lines in your changes missing coverage. Please review.

Project coverage is 77.47%. Comparing base (044e991) to head (4bacfe8).
Report is 41 commits behind head on main-dev.

Files with missing lines Patch % Lines
pyaerocom/units/datetime/utils.py 68.61% 43 Missing ⚠️
pyaerocom/units/molecular_mass.py 93.13% 7 Missing ⚠️
pyaerocom/aux_var_helpers.py 75.00% 1 Missing ⚠️
pyaerocom/colocation/uemep/colocator.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           main-dev    #1552      +/-   ##
============================================
- Coverage     78.11%   77.47%   -0.64%     
============================================
  Files           145      141       -4     
  Lines         21579    20852     -727     
============================================
- Hits          16856    16155     -701     
+ Misses         4723     4697      -26     
Flag Coverage Δ
unittests 77.47% <85.39%> (-0.64%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@thorbjoernl thorbjoernl self-assigned this Mar 20, 2025
@thorbjoernl thorbjoernl marked this pull request as ready for review March 24, 2025 13:37
from pyaerocom.helpers import (
get_lowest_resolution,
isnumeric,
make_datetime_index,
to_pandas_timestamp,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, surprising indeed that the import works

@@ -23,11 +23,10 @@
VariableDefinitionError,
VariableNotFoundError,
)
from pyaerocom.units.datetime import cftime_to_datetime64, datetime2str
from pyaerocom.helpers import (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one here too

from pyaerocom.helpers import (
get_lowest_resolution,
start_stop,
to_datestring_YYYYMMDD,
to_pandas_timestamp,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here too

@@ -7,8 +7,8 @@
from pyaerocom import const
from pyaerocom.aeroval.modelentry import ModelEntry
from pyaerocom.griddeddata import GriddedData
from pyaerocom.units.datetime import get_highest_resolution
from pyaerocom.helpers import (
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here is the import, sorry, I deleted a comment which was in the wrong file

def __radd__(self, other) -> MolecularMass:
return self.__add__(other)

def __sub__(self, other) -> MolecularMass:
if isinstance(other, MolecularMass):
Copy link
Collaborator

@charlienegri charlienegri Mar 25, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to improve coverage would it be possible to add some dummy testing here, something like

def test_molecular_mass___sub___and___mul__():
    mass = MolecularMass("H2")
    mass2 = MolecularMass("O")
    assert mass2.__sub__(mass).mass == 13.983519999999999
    assert mass.__mul__(mass2) == 32.252870472

?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coverage aside, I have no other comments otherwise, so I'll approve

return str(self.val)


def sort_ts_types(ts_types: list[str | TsType]) -> list[str]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Out of scope for this PR but it would be nice if all occurrences of time series types in pyaerocom were instances of TsType

@thorbjoernl thorbjoernl merged commit 084f209 into main-dev Mar 26, 2025
6 checks passed
@thorbjoernl thorbjoernl deleted the units branch March 26, 2025 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants