Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor dmod.modeldata serialization and deserialization types #257

Merged

Conversation

aaraney
Copy link
Member

@aaraney aaraney commented Jan 26, 2023

This PR refactors dmod.modeldata's dmod.core.serializable.Serializable subtypes into compliance with the changes introduced in #239. These changes should not affect downstream usage of dmod.modeldata Serializable subtypes.

For broader context, implications, and examples regarding changes that affect the Serializable subtype, see #239.

Changes

  • Serialization, deserialization, validation, and type coercion are now mainly handled by pydantic. All subtypes of dmod.core.serializable.Serializable are affected by this change.
  • pydantic added as dmod.modeldata dependency
  • missing dependency, GitPython, added to dmod.modeldata's setup.py

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal (no scope creep)
  • Code can be automatically merged (no conflicts)
  • Code follows project standards (link if applicable)
  • Passes all existing automated tests
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • Visually tested in supported browsers and devices (see checklist below 👇)
  • Project documentation has been updated (including the "Unreleased" section of the CHANGELOG)
  • Reviewers requested with the Reviewers tool ➡️

@aaraney
Copy link
Member Author

aaraney commented Feb 7, 2023

In writing some tests to accompany this PR I stumbled into a question. Is the below the desired behavior, @robertbartel?

from dmod.modeldata.hydrofabric.partition import Partition, PartitionConfig

catchment_ids = ["1", "2", "3"]
partition_id = 0

o1 = Partition(
        partition_id=partition_id,
        catchment_ids=catchment_ids,

        nexus_ids=["2"],
        remote_up_nexuses=["1"],
        remote_down_nexuses=["3"],
        )

o2 = Partition(
        partition_id=partition_id,
        catchment_ids=catchment_ids,

        nexus_ids=["2222"],
        remote_up_nexuses=["1111"],
        remote_down_nexuses=["3333"]
        )

same_catchment_id_and_partition_id = PartitionConfig(partitions=[o1, o2])

assert len(same_catchment_id_and_partition_id.partitions) ==  1

reasons why dict is overridden here:
pydantic will serialize from inner types outward, serializing each type
as a dictionary, list, or primitive and replacing its previous type with
the new "serialized" type. Consequently, this means hashable container
types like tuples and frozensets that contain values that "serialize" to
a non-hashable type (non-primitive, in this case) will raise a
`TypeError: unhashable type: 'dict'`. In the case of PartitionConfig,
FronzenSet[Partition] "serializes" inner Partition types as dictionaries
which are not hashable. To get around this, we will momentarily swap the
`partitions` field for a non-hashable container type, serialize using
`.dict()`, and swap back in the original `partitions` container.
aaraney added a commit to aaraney/DMOD that referenced this pull request Feb 10, 2023
aaraney added a commit to aaraney/DMOD that referenced this pull request Feb 10, 2023
robertbartel pushed a commit to robertbartel/DMOD that referenced this pull request Feb 16, 2023
robertbartel pushed a commit to robertbartel/DMOD that referenced this pull request Feb 16, 2023
@robertbartel robertbartel merged commit 4e20d27 into NOAA-OWP:pydantic-serializable Feb 22, 2023
@aaraney
Copy link
Member Author

aaraney commented Feb 22, 2023

Thanks, @robertbartel!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants