Fix all plants ferc1 #1656
Fix all plants ferc1 #1656
Conversation
…h syntax of other ferc1 plant tables
…test (just so it's alphabetical)
… table, change opex_total - opex_fuel to opex_total - opex_fuel.fillna(0). A lot of the opex_fuel rows are NA, but they really mean zero because they are renewable. When you try and subtract NA from a float you get NA even if the original value is not NA. This means that all the renewables have NA opex_nonfuel (O&M Cost) which is not true! This fixes that.
…ble because opex_total apparently doesn't actually include opex_maintenance. Adding this makes it match the official BHE data we're working with with RMI.
Codecov Report
@@ Coverage Diff @@
## dev #1656 +/- ##
=======================================
+ Coverage 83.8% 84.8% +0.9%
=======================================
Files 65 65
Lines 7270 7849 +579
=======================================
+ Hits 6099 6663 +564
- Misses 1171 1186 +15
Continue to review full report at Codecov.
|
src/pudl/output/ferc1.py
Outdated
| @@ -59,7 +59,7 @@ def plants_steam_ferc1(pudl_engine): | |||
| .assign( | |||
| capacity_factor=lambda x: x.net_generation_mwh / (8760 * x.capacity_mw), | |||
| opex_fuel_per_mwh=lambda x: x.opex_fuel / x.net_generation_mwh, | |||
| opex_nonfuel=lambda x: x.opex_production_total - x.opex_fuel, | |||
| opex_nonfuel=lambda x: x.opex_production_total - x.opex_fuel.fillna(0), | |||
There was a problem hiding this comment.
are there instances in which we think the fuel cost could actually be non-zero and the opex_production_total does actually include it even though opex_fuel is null?
It might be nice to do a little checking of this opex_production_total before we make this assumption.
There was a problem hiding this comment.
Ok, for the steam table only 0.12% of the opex_production_total values don't add up to the sum of all other opex_CATEGORY columns. Most of these look like errors where the opex_production_total value is very low, in some cases single digits when it obviously shouldn't be.....this is probably something we should flag or fix down the road....but anyways these 36 bad values should probably not use the production_total value and rather the actual sum total (calculated). All the bad values' production_total values are lower than the sum total of opex_ columns so it's unlikely that they would include additional, "hidden" fuel values where opex_fuel is null. (Also there are only 4 bad rows where opex_fuel is null)
Given how exactly most of the opex values sum to equal the opex_production_total value regardless of whether there is a value for opex_fuel, I think that it's unlikely/other values would have to be reported wrong in order for opex_fuel to be null but a non-zero fuel amount was also included in the production_total value.
There was a problem hiding this comment.
Aren't we supposed to remove calculable columns in one of the ETL steps?
There was a problem hiding this comment.
this is great news! thanks for investigating! that # of records are definitely ignore-able imho!
…is change in the transform module as well as the output module where there are references to the original opex_total column
…dict for the small gens table
…es for RMI purposes to opex_total_nonfuel so it isn't confused with the opex components that sum up to opex_total.
|
I just pushed some code that changes the erroneously labeled |
This PR changes a few things related to the ferc1 output table with all the plants:
all_plants_ferc1toplants_all_ferc1to match the syntax of the other ferc1 plant tablesopex_nonfuelcolumn for all of the plant tables so that the aggregateplants_all_ferc1table has an accurate Total O&M metric. Compared these values against RMI official data for validation.We will need to update any references to the
all_plants_ferc1table in other repos -- I already did this for the rmi repo in a branch calledfix-all-plants-ferc-ref.