-
Notifications
You must be signed in to change notification settings - Fork 12
v2: Clarify model initialization #645
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Clarify that the parameter table is applied before initial assignments are executed.
matthiaskoenig
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if this clarifies things.
In my opinion, InitialAssignments must be applied when the model is loaded, i.e., before any changes to the model are made. This is how I understand SBML and SED-ML handle InitialAssignments.
All InitialAssignments are applied when the model loads.
Many tools eliminate InitialAssignments when loading a model because they apply the changes and then leave the model in an initialized state. Roadrunner does something similar.
To override initial assignments, the new assignment must be encoded in the parameter table.
In summary, we have to clarify this. However, my interpretation is different; InitialAssignments must be applied first.
|
To clarify. The changes in the parameter table are basically EventAssignments (and are handled as such in the timecourse definition). EventAssignments are applied after InitialAssignments. See SBML Specification for simulation at t=0 (https://raw.githubusercontent.com/combine-org/combine-specifications/main/specifications/files/sbml.level-3.version-2.core.release-2.pdf, p29 L10):
|
|
Yes, you are right. That was too hasty. My main problem was that so far, it was not mentioned at all how parameter table parameters are treated. Changing a parameter for the first period via the condition table should have the same effect as changing it through the parameter table. It can be phrased in a way that the parameter table implicitly defines a default condition that is applied to the first period of every experiment. |
|
Hm, I think we'd a problem there if we consider the parameter table parameters part of the first condition to be applied: Assume we have a species
This way, Thus, we need to include an additional initialization step:
However, this leaves a subtle difference between parameter table changes and condition table changes that we didn't have in v1 and that might confuse users. |
|
|
||
| .. _v2_initialization: | ||
|
|
||
| Initialization and parameter application |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This statement from the experiment table could be included/moved here
time will override any initial time specified in the model, except in the case of time = -ìnf, in which case the model-specified time will be used (or 0, if the model does not explicitly specify an initial time).
|
I would support applying changes from the parameter table before model initialisation. To my understanding, Applying the parameter table before initialisation has a clear benefit: it enables the use of
|
|
I see parameter table parameter values as changing the model definition before any simulation is initialized. This way:
|
I fully agree with that. That's similar to the discussion in #624. However, there the conclusion was that condition table changes should be treated as event assignments, including the changes at the initial timepoint, and that moving all initialization to the condition table was fine. In PEtab v1, changing some entity through the condition table had the same effect as changing it through the parameter table. I would greatly prefer keeping it that way. I'd find it very confusing if setting A0=0 in the parameter table affected my initial assignments, but specifying the same in the condition table for all experiments would be ignored in my initial assignments. So I'd say parameter table and condition table changes should both be applied before initial assignments, or both after, but not mixed. |
Fair point. The above example also applies to setting where you want to map different initial condition parameters, which also wouldn't work with the current spec. Sounds we either need revisit that whole topic again or have a big massive warning sign because this is something user, including me, will get wrong. |
|
Rereading that whole thread, I would say let's go with the big fat warning signs, i.e. check whether any parameter/condition table assignments appear in the initial assignments. |
Although I'm not so much looking forward to go over all test cases again, my conclusion would have been the opposite 🙈 I can live with either, though. Since this is quite a central aspect, it would be great to get some additional opinions. |
It's not only what occurs explicitly in initial assignments, one also needs to check the implicit dependencies. One additional point to consider here: This decision will have a significant impact on the effect of changing a compartment size via the parameter or the condition table. If model initialization happens before applying any PEtab overrides (i.e., PEtab overrides are interpreted as event assignments), then this will scale the concentrations and preserve the initial amounts. If model initialization happens after applying any PEtab overrides (i.e., PEtab changes for the first period override the initial values in the model), then this will preserve the initial concentrations (or the What is more helpful will depend on the exact model, but I guess, many users would expect to see the latter, which is the same as if they would change the compartment size directly in the model. So, either we will have some inconsistency in the treatment of the first period vs later periods of an experiment, or we will have some inconsistency regarding changing initial values through the model vs through PEtab. |
I'd say this depends on how you define
Hm, in gradient-based optimization, we usually don't really care about |
Additional implications if the changes of the first period are interpreted as event assignments instead of initial assignments: EDIT: okay, on second thought, this is maybe not that critical. One just has to remember that the concentration in the condition table needs to be specified as |
I voted for condition changes before the initial condition there 😀 There was a lot of back and forth but again here I would say both condition changes and parameter table values are applied before the initial condition.
I agree with this. As specifying the initial condition is not directly in the PEtab scope (only experiment-specific changes to the initial condition), we should support users specifying the initial condition in their model -- therefore changes from PEtab parameters and conditions should be applied before the initial condition, such that the initial condition can be dependent on them.
It would be problematic but to clarify, I see now that I was describing the situation that you already described above: does the PEtab user need to write in their manuscript that: "The parameter was estimated, except for when the model's initial condition was evaluated, where it took the value defined in the model." ? For this reason, I don't agree with the "SBML event" perspective for the parameter table values and experiment condition changes. I think the "pre-initial condition" perspective is more intuitive for users and has no unexpected consequences that would require warnings, except for those in PEtab v1 already. Treating condition changes as SBML events with highest priority is fine for the intermediate timepoints, but not for the initial timepoint due to the model's initial condition. |
Agreed (at least for the first period).
|
|
I think we have a couple of interconnected issues here. I'll try to summarize my understanding of the different concerns and potential solutions brought up so far. Main problem: The current specs ( We want to address that while achieving (most of) the following objectives: Objective 1: The parameter table values must be usable when evaluating targetValues of the condition table. Otherwise, it won't be possible to estimate any initial conditions in a experiment-specific manner. Objective 2: Changing some parameter globally through the parameter table should have the same effect as changing it for every experiment through the condition table, at least for the cases where we set a parameter to some constant value (which is the only thing supported in the parameter table anyways). That was the initial purpose of the condition table in PEtab v1, and I think that was good. Now in PEtab v2, this is a bit more complex, because we don't just define initial conditions, but also changes during a simulation and we allow for more complex expressions. Objective 3: Ideally, condition table changes have the same effect independent of when they are applied during an experiment. Objective 4: Compartment size changes should preserve the amounts of contained species. (Not so essential for me personally, but I think this was agreed upon earlier. I don't think it was ever clarified whether this should also hold for parameter table changes.) Objective 5: Be able to make use of initial assignments contained in the model while using PEtab condition specific overrides to avoid completely moving them to the condition table. I think these objectives are mutually exclusive (happy to be proven wrong) and the weighting will probably be very subjective. For me, currently, Objective 1 >> 2 >> 5 ~= 3 > 4. What can we do? Option 1: One way to achieve Objective 1 would be introducing an additional initialization step after model initialization but before experiment-initialization (first initialize the model without any PEtab changes, then apply the parameter table changes, then apply the condition table changes). Option 2a: Another way to achieve Objective 1 would be applying parameter table changes before model initialization. This way, the parameter table changes would mirror SBML initial assignment. However, this conflicts with Objective 2, because condition table changes are currently interpreted as event assignments. Changing a compartment size through an initial assignment may have a different effect than changing it through an event. It covers Objective 3, though. It covers Objective 4 for the condition table, and covers Objective 5 for the parameter table. To resolve the issue in Option 2a, we have at least the following options: Option 2b: Apply the parameter table changes before model initialization and always treat the first period of each experiment as initial assignments instead of event assignments. This generally makes sense to me, but it means that a given condition has different implications depending on when it is applied during a timecourse (first vs. subsequent period), and thus, interferes with Objective 3. That's the cost of conflating initial assignments and event assignments in the condition table. It does not cover Objective 4 for the condition table for the first period of an experiment, but it covers Objective 5. Option 2c: Apply the parameter table changes before model initialization and treat all condition table changes as initial assignments. This covers Objectives 1-3,5, but is incompatible with Objective 4. For all of the above options, the condition table changes could be converted to SBML events for simulation, they would just look different. Looking forward to hear other opinions / solutions. |
|
Thanks for clarifying these aspects. I think Option 2c is the best option, for the objectives you described as well as some additional reasons:
|
I agree with Dilan here. Only thing I would add is that I am not sure whether we want to always satisfy objective 4. I wouldn't interpret condition table assignments in the first period as changes to volumes, but rather initialisations. Thus I would also be okay with option 2b even though objective 3 generally makes sense for me. |
|
Just to clarify that I understand this correctly, with option 2c, if an initial assignment or assignment rule / algebraic rule existed for a symbol in conflict with initializations from parameter / condition table, those would be dropped in preference for the new values that would be added as initial assignment? Also as for objective 4, based on the |
The original initial assignment would be dropped, yes. An assignment/algebraic rule target would be illegal, since we voted against "structural" alterations.
Right. |
|
Thanks for the feedback so far. Based on that, Option 2c seems to be the way forward. I will shortly update the pull request accordingly. Any additional thoughts are still welcome. While it sounds complicated, it think, this will be the behavior that most users would expect intuitively. |
| Target value expressions for the first period must not refer to model | ||
| entities that are not also listed in the parameter table. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possible to also allow model entities listed in the first period condition changes? This would mean that the order of application of changes now matters.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In principle, yes, as long as there aren't any circular dependencies. However, this would be inconsistent with later periods, where the respective symbols evaluate to their value at the end of the previous condition, not to the newly assigned value.
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Done. Please let me know if this is in line with your expectations. |
Update tests according to changed initialization semantics (PEtab-dev/PEtab#645).
After PEtab-dev/PEtab#645, any condition that is used as a first condition cannot refer to any symbols other then the parameters listed in the parameter table, or `time'. Closes PEtab-dev#424.
| 1. Pre-initialization | ||
|
|
||
| 1. Parameters values for parameters that occur in the parameter table are | ||
| applied to the uninitialized model. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Clarify what uninitialized means here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This will probably be model-format-dependent. I can add "pre-initial-assignment-execution" as an example for SBML models. Keeping SBML specifics out of the PEtab specs didn't work too well anyways... 🙈
|
@matthiaskoenig @fbergmann @sebapersson Does this work for you? |
sebapersson
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good to me, thanks for fixing!
matthiaskoenig
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reads good to me, have to test how easy this is to implement with roadrunner (especially setting the parameters before applying the initial assignments).
After PEtab-dev/PEtab#645, any condition that is used as a first condition cannot refer to any symbols other then the parameters listed in the parameter table, or `time'. Closes PEtab-dev#424.
After PEtab-dev/PEtab#645, any condition that is used as a first condition cannot refer to any symbols other then the parameters listed in the parameter table, or `time'. Closes PEtab-dev#424.
After PEtab-dev/PEtab#645, any condition that is used as a first condition cannot refer to any symbols other then the parameters listed in the parameter table, or `time'. Closes #424.
…tics (#443) Due to PEtab-dev/PEtab#645 ... * ... the first period of a PEtab experiment has to be applied before initial assignments are evaluated. That means, the changes have to be implemented as initial assignments instead of event assignments at the initial timepoint (or any pre-existing initial assignments would have to be included in the event assignment). * ... for any subsequent periods, the event assignments need to be modified. Compartment-size changes in PEtab no longer follow the SBML event assignment semantics. That means, we need event assignments for all concentration-based species inside such a compartment to preserve concentrations instead of amounts. Closes #452. --------- Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com>
Update tests according to changed initialization semantics (PEtab-dev/PEtab#645) and add some extra observables and measurement timepoints to simplify debugging.
Clarify that the parameter table is applied before initial assignments are executed and change the way how concentration and compartment changes are handled.