Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allocate generation_fuel_eia923 table data to generators #785

Merged
merged 23 commits into from
Dec 11, 2020

Conversation

cmgosnell
Copy link
Member

@cmgosnell cmgosnell commented Oct 5, 2020

This PR attempts to implement a solution to allocation the data from the generation_feul_eia923 table reported at the plant/fuel/prime_mover level to the plant/gen level. The bulk of this happens inside of pudl.analysis.allocate_net_gen.py and is integrated into pudl.analysis.mcoe and pudl.output.pudltabl

Closes Issue #777.

@codecov
Copy link

codecov bot commented Oct 5, 2020

Codecov Report

Merging #785 (8581618) into sprint27 (5a8a0d7) will increase coverage by 1.14%.
The diff coverage is 97.62%.

Impacted file tree graph

@@             Coverage Diff              @@
##           sprint27     #785      +/-   ##
============================================
+ Coverage     71.03%   72.18%   +1.14%     
============================================
  Files            42       43       +1     
  Lines          4957     5071     +114     
============================================
+ Hits           3521     3660     +139     
+ Misses         1436     1411      -25     
Impacted Files Coverage Δ
src/pudl/analysis/allocate_net_gen.py 97.00% <97.00%> (ø)
src/pudl/analysis/mcoe.py 95.00% <100.00%> (+0.13%) ⬆️
src/pudl/output/eia860.py 100.00% <100.00%> (ø)
src/pudl/output/eia923.py 98.76% <100.00%> (ø)
src/pudl/output/pudltabl.py 62.10% <100.00%> (+3.65%) ⬆️
src/pudl/helpers.py 92.04% <0.00%> (+0.35%) ⬆️
src/pudl/workspace/datastore.py 61.05% <0.00%> (+12.63%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5a8a0d7...8581618. Read the comment docs.

@cmgosnell cmgosnell changed the base branch from sprint25 to ferc1-eia-2019 October 16, 2020 14:59
@cmgosnell cmgosnell changed the title WIP: explore net generation diff Allocate generation_feul_eia923 table data to generators Oct 16, 2020
@zaneselvans zaneselvans changed the title Allocate generation_feul_eia923 table data to generators Allocate generation_fuel_eia923 table data to generators Oct 16, 2020
@zaneselvans zaneselvans changed the base branch from ferc1-eia-2019 to sprint26 October 19, 2020 23:07
@zaneselvans zaneselvans added this to the PUDL Sprint 27 milestone Nov 3, 2020
@zaneselvans zaneselvans self-assigned this Nov 3, 2020
this is a little premature considering the fact that we don't have these 
years rn...
@zaneselvans zaneselvans changed the base branch from sprint26 to sprint27 November 5, 2020 22:34
@cmgosnell cmgosnell removed this from the PUDL Sprint 27 milestone Nov 18, 2020
@cmgosnell cmgosnell added this to the Sprint 28 milestone Nov 18, 2020
Comment on lines 285 to 287
gen = pudl_out.gen_eia923()
if pudl_out.fill_net_gen:
gen = pudl_out.gen_allocated()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it make sense to move this conditional logic into a helper function in pudl_out that would check the status of fill_net_gen and return the corresponding dataframe?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that is a good idea!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hey @rousik I integrated this logic into pudl_out as you suggested! I did a little bit of additional column management such that the columns in output of pudl_out.gen_eia923() is the same whether fill_net_gen is True or False. let me know what you think!

self._dfs['hr_by_unit'] = pudl.analysis.mcoe.heat_rate_by_unit(
self)
self._dfs['hr_by_unit'] = (
pudl.analysis.mcoe.heat_rate_by_unit(self)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Passing self to some other method sounds like object-oriented design gone wrong. I see that this is not a problem introduced by this PR so I do not expect solution for it here but we should probably consider refactoring the code and eliminate this pattern in the future.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yes, the output/analysis layer needs a big overhaul. They were made originally in the very early days. I have a few ideas for this, but it'd be great to get your input on that when the time comes!

@cmgosnell cmgosnell merged commit 63fc9ca into sprint27 Dec 11, 2020
@cmgosnell cmgosnell deleted the net_gen_fix branch December 11, 2020 23:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Associate net generation from the generation_fuel_eia923 table w/ generators
3 participants