-
Notifications
You must be signed in to change notification settings - Fork 90
Minor updates to sets for functionality, speed and API conformity #110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Apply the "construction dictionary" pattern to the 'GENS_*' sets which might be making testing slightly faster. Update the set initialization in dispatch.py to match Pyomo documentation of supported API (`initialize` argument instead of `rule`). `rule` works for now, but using published API should be more future proof.
- remove unneeded leading underscores from various element names - use fuels from FUELS_FOR_GEN (possibly multiple values) instead of gen_energy_source (which may just say "multiple")
mfripp
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fuel-based generators may generally have multiple fuels. So I updated the code to pull all fuels for each fuel-based generator into the GENS_BY_ENERGY_SOURCE list. We should add an example case with one or more multi-fuel generators to catch this type of regression. It's on my to-do list, but I'm happy for someone else to do it!
I also removed leading underscores from a lot of element names in this file. I find them a bit fussy and distracting, and they seem to imply that we're doing something special and secret when we're really just implementing straightforward behavior. Leading underscores on m._Component_dict also break the parallelism with m.Component (and the pattern set by Pyomo itself), and there's no need to identify them as behind-the-scenes elements, since we delete them before they can be used for anything. There's also no need for underscores on helper functions defined within define_components(), since they are out of scope for any other code anyway.
I have done some work on a more general approach to building these aggregation dictionaries, but it's not ready to roll out yet. The most promising way I found was to have a helper function of our own that automatically generates an initialization rule with a closure that holds the aggregation dictionary. I got worried that the closure would interfere with pickling the model when we save it or when PySP sends it over the network, but that doesn't actually seem to be the case. So maybe I will implement this at some point, since it's such a common problem and our current code to handle it is verbose and repetitive.
I will merge this pull request into the 2.0.3 branch soon.
|
Forgot to mention: I think Also, we can generally do |
|
I like shorter statements that are more pythonic. Generalizing this might be nice, especially if it helps with readability & consistency. Not sure how broadly we'd want to deploy the general form, since the timing benefits seem negligible for a lot of sets, and using a custom solution has a cost of readability/cognitive load. Do you have a sense of which set initializations are bottlenecks in your runs? In the long term, it might be nice to package this sort of augmentation into Pyomo pull requests. In other languages with strict enforcement of private variables & methods, leading underscores can indicate special/secret behavior. Python doesn't enforce private vars or methods, but treats leading underscores as a helpful suggestion that the method or variable in question is meant for internal use (not part of a stable public interface) ..encoded into PEP-8. When I see those underscores in libraries that I'm working with, I'll try to avoid code against them because their implementation or existence might change in the next release. For practical purposes, dropping leading underscores from this method should be fine. A method defined inside of another method is another way of indicating internal use. An underscore on these dictionary seems a little better to me, since that dictionary is just a side-effect of our optimization. But it isn't that big of deal to me either way, especially since that variable only exists during initialization - probably just for the duration of one for loop. |
Refactoring and simplification of postprocess
Refactoring and simplification of postprocess
Refactoring and simplification of postprocess
Add GENS_BY_ENERGY_SOURCE set for faster/easier indexing.
Apply the "construction dictionary" pattern to the 'GENS_*' sets which might be making testing slightly faster.
Update the set initialization in dispatch.py to match Pyomo documentation of supported API (
initializeargument instead ofrule).ruleworks for now, but using published API should be more future proof.