Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects #675

Merged
merged 5 commits into from
Oct 5, 2022

Conversation

rouille
Copy link
Collaborator

@rouille rouille commented Sep 2, 2022

Pull Request doc

Purpose

Test grid equality for round trip conversion Grid -> Network -> Grid.

What the code is doing

  • Define __class__ property so FromPyPSA instances have type Grid
  • Check grid equality and generate some prints

Testing

Tests are failing due to differences in the two grids in the test_import_exported_network. This is illustrated when running pytest powersimdata/input/converter/tests/test_pypsa_to_grid.py:

        print("Compare bus values for common columns")
>       assert_frame_equal(ref.bus[ref.bus.columns], test.bus[ref.bus.columns])
E       AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="type") are different
E       
E       Attribute "dtype" are different
E       [left]:  int64
E       [right]: object

And:

Difference in bus columns
Index(['type', 'Pd', 'Qd', 'Gs', 'Bs', 'zone_id', 'Vm', 'Va', 'baseKV',
       'loss_zone', 'Vmax', 'Vmin', 'lam_P', 'lam_Q', 'mu_Vmax', 'mu_Vmin',
       'interconnect', 'lat', 'lon'],
      dtype='object')
Index(['bus_id', 'type', 'Pd', 'Qd', 'Gs', 'Bs', 'zone_id', 'Vm', 'Va',
       'baseKV', 'loss_zone', 'Vmax', 'Vmin', 'lam_P', 'lam_Q', 'mu_Vmax',
       'mu_Vmin', 'interconnect', 'lat', 'lon', 'includes_pypsa_shunt'],
      dtype='object')
Difference in branch columns
Index(['from_bus_id', 'to_bus_id', 'r', 'x', 'b', 'rateA', 'rateB', 'rateC',
       'ratio', 'angle', 'status', 'angmin', 'angmax', 'Pf', 'Qf', 'Pt', 'Qt',
       'mu_Sf', 'mu_St', 'mu_angmin', 'mu_angmax', 'branch_device_type',
       'interconnect', 'from_zone_id', 'to_zone_id', 'from_zone_name',
       'to_zone_name', 'from_lat', 'from_lon', 'to_lat', 'to_lon'],
      dtype='object')
Index(['from_bus_id', 'to_bus_id', 'r', 'x', 'b', 'rateA', 'rateB', 'rateC',
       'ratio', 'angle', 'status', 'angmin', 'angmax', 'Pf', 'Qf', 'Pt', 'Qt',
       'mu_Sf', 'mu_St', 'mu_angmin', 'mu_angmax', 'branch_device_type',
       'interconnect', 'from_zone_id', 'to_zone_id', 'from_zone_name',
       'to_zone_name', 'from_lat', 'from_lon', 'to_lat', 'to_lon',
       'branch_device_type', 'branch_device_type', 'branch_device_type'],
      dtype='object')

There are likely other issues in the converter(s).

Where to look

N/A

Usage Example/Visuals

N/A

Time estimate

N/A

@rouille rouille self-assigned this Sep 2, 2022
@rouille
Copy link
Collaborator Author

rouille commented Sep 2, 2022

@FabianHofmann, this could help testing grid equality

@rouille
Copy link
Collaborator Author

rouille commented Sep 10, 2022

The roundtrip conversion has been fixed after merging #678.

@rouille rouille added the bug Something isn't working label Sep 10, 2022
@rouille rouille marked this pull request as ready for review September 10, 2022 00:19
@rouille
Copy link
Collaborator Author

rouille commented Sep 11, 2022

@FabianHofmann, I added storage devices to the reference grid and the test (grid equality after roundtrip conversion) is failing.

@FabianHofmann
Copy link
Contributor

@rouille thanks for the update, I'll fix that asap

@rouille rouille changed the title Grid equality Implement roundtrip conversion Sep 20, 2022
@rouille rouille changed the title Implement roundtrip conversion Enable roundtrip conversion Sep 20, 2022
@rouille
Copy link
Collaborator Author

rouille commented Sep 20, 2022

@rouille thanks for the update, I'll fix that asap

Thanks. The round trip conversion including storage has been fixed after merging #685. @bainan and @jenhagg, this is now ready for full review

Copy link
Collaborator

@BainanXia BainanXia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good. I left a couple of comments that might out of the scope of this PR. Also, as discussed elsewhere, we will refactor the grid equality definition here to incorporate grid comparisons to different extents as we do in the tests here.

for k, v in defaults.items():
storage[k] = storage[k].fillna(v) if k in storage else v

storage["p_nom"] = storage.get("Pmax")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We will need to refactor the p_nom assignment after the implementation of "subsystem" logic for hydro/PHS.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. It might be even more complicated. @FabianHofmann, will we need to add hydro and PHS generators located in the plant data frame of the Grid object to the StorageUnit component of the Network object? Or they will simply be added to the Generator component?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a good question, I'd prefer to add them to StorageUnit (assuming this is where they originally come from). But I prefer to tackle that in a separate PR.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good.

@rouille rouille force-pushed the ben/grideq branch 2 times, most recently from ce898ff to 7840e02 Compare September 29, 2022 23:36
@rouille rouille changed the title Enable roundtrip conversion Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects Oct 5, 2022
@rouille rouille merged commit 648edd5 into ben/import Oct 5, 2022
@rouille rouille deleted the ben/grideq branch October 5, 2022 17:34
rouille added a commit that referenced this pull request Oct 25, 2022
Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects
rouille added a commit that referenced this pull request Oct 25, 2022
Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects
rouille added a commit that referenced this pull request Oct 27, 2022
Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects
rouille added a commit that referenced this pull request Nov 5, 2022
Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects
rouille added a commit that referenced this pull request Nov 10, 2022
Enable roundtrip conversion PyPSA Network <--> PowerSimData Grid objects
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants