Skip to content

Conversation

@josiahjohnston
Copy link
Contributor

Add GENS_BY_ENERGY_SOURCE set for faster/easier indexing.
Apply the "construction dictionary" pattern to the 'GENS_*' sets which might be making testing slightly faster.
Update the set initialization in dispatch.py to match Pyomo documentation of supported API (initialize argument instead of rule). rule works for now, but using published API should be more future proof.

Apply the "construction dictionary" pattern to the 'GENS_*' sets which might be making testing slightly faster.
Update the set initialization in dispatch.py to match Pyomo documentation of supported API (`initialize` argument instead of `rule`). `rule` works for now, but using published API should be more future proof.
@josiahjohnston josiahjohnston requested a review from mfripp June 8, 2019 01:25
- remove unneeded leading underscores from various element names
- use fuels from FUELS_FOR_GEN (possibly multiple values) instead of 
gen_energy_source (which may just say "multiple")
Copy link
Member

@mfripp mfripp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fuel-based generators may generally have multiple fuels. So I updated the code to pull all fuels for each fuel-based generator into the GENS_BY_ENERGY_SOURCE list. We should add an example case with one or more multi-fuel generators to catch this type of regression. It's on my to-do list, but I'm happy for someone else to do it!

I also removed leading underscores from a lot of element names in this file. I find them a bit fussy and distracting, and they seem to imply that we're doing something special and secret when we're really just implementing straightforward behavior. Leading underscores on m._Component_dict also break the parallelism with m.Component (and the pattern set by Pyomo itself), and there's no need to identify them as behind-the-scenes elements, since we delete them before they can be used for anything. There's also no need for underscores on helper functions defined within define_components(), since they are out of scope for any other code anyway.

I have done some work on a more general approach to building these aggregation dictionaries, but it's not ready to roll out yet. The most promising way I found was to have a helper function of our own that automatically generates an initialization rule with a closure that holds the aggregation dictionary. I got worried that the closure would interfere with pickling the model when we save it or when PySP sends it over the network, but that doesn't actually seem to be the case. So maybe I will implement this at some point, since it's such a common problem and our current code to handle it is verbose and repetitive.

I will merge this pull request into the 2.0.3 branch soon.

@mfripp
Copy link
Member

mfripp commented Jun 14, 2019

Forgot to mention:

I think if not d is considered more Pythonic than if len(d) == 0. I would argue that the intent is less clear to new users, but nevertheless it better fits Python conventions and the expectations of more experienced users, so I went with it. It may also be slightly faster to check "falsiness" of a dictionary than to compare its length to 0, but I doubt it makes much difference.

Also, we can generally do del m.component instead of delattr(m, 'component').

@mfripp mfripp changed the base branch from master to 2.0.3 June 14, 2019 21:54
@mfripp mfripp changed the base branch from 2.0.3 to master June 14, 2019 21:55
@mfripp mfripp changed the base branch from master to 2.0.3 June 14, 2019 22:01
@mfripp mfripp changed the base branch from 2.0.3 to master June 14, 2019 22:03
@mfripp mfripp changed the base branch from master to 2.0.3 June 14, 2019 22:29
@mfripp mfripp merged commit 16d4086 into 2.0.3 Jun 14, 2019
@mfripp mfripp deleted the set_updates branch June 14, 2019 22:34
@josiahjohnston
Copy link
Contributor Author

I like shorter statements that are more pythonic. Generalizing this might be nice, especially if it helps with readability & consistency. Not sure how broadly we'd want to deploy the general form, since the timing benefits seem negligible for a lot of sets, and using a custom solution has a cost of readability/cognitive load. Do you have a sense of which set initializations are bottlenecks in your runs? In the long term, it might be nice to package this sort of augmentation into Pyomo pull requests.

In other languages with strict enforcement of private variables & methods, leading underscores can indicate special/secret behavior. Python doesn't enforce private vars or methods, but treats leading underscores as a helpful suggestion that the method or variable in question is meant for internal use (not part of a stable public interface) ..encoded into PEP-8. When I see those underscores in libraries that I'm working with, I'll try to avoid code against them because their implementation or existence might change in the next release.

For practical purposes, dropping leading underscores from this method should be fine. A method defined inside of another method is another way of indicating internal use. An underscore on these dictionary seems a little better to me, since that dictionary is just a side-effect of our optimization. But it isn't that big of deal to me either way, especially since that variable only exists during initialization - probably just for the duration of one for loop.

staadecker pushed a commit that referenced this pull request Jan 28, 2023
Refactoring and simplification of postprocess
staadecker pushed a commit that referenced this pull request Jan 28, 2023
Refactoring and simplification of postprocess
staadecker pushed a commit that referenced this pull request Jan 29, 2023
Refactoring and simplification of postprocess
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants