Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

partial compile for facts #4760

Open
sanderr opened this issue Aug 31, 2022 · 4 comments
Open

partial compile for facts #4760

sanderr opened this issue Aug 31, 2022 · 4 comments
Labels
design Issues with a design aspect

Comments

@sanderr
Copy link
Contributor

sanderr commented Aug 31, 2022

When a fact comes in we currently trigger a full compile. In case facts are used in conjunction with lsm (or far less likely, with a custom partial compile tool/extension), this could be a big impediment. It would be better if we could retrigger partial compiles for each partial that had an unknown for this fact.

@sanderr sanderr added discussion There is a discussion happening on this issue design Issues with a design aspect and removed discussion There is a discussion happening on this issue labels Aug 31, 2022
@wouterdb
Copy link
Contributor

Central problems

  1. interaction core/lsm
  2. attributing and unknown to a slice

Interaction core/lsm

  1. I would add a listener/interceptor to the param service to allow LSM to modify the compile request before it is fired, with all unknowns as input, the basic compile config as input and one or more compiles as output.
    1. alternatively, this can be added to listener/interceptor the compile service as well, if we add a field to the compile to indicate its reason in a machine friendly way.

attributing and unknown to a slice

  1. In the exporter, I would extend the Unknown with the ResourceSet its resource lives in
  2. Add this additional field in the DB

Overall

  1. When a fact is updated, the listener in LSM is called
  2. the listener selects the appropriate slice(s) and modifies the compile request (or splits, if multiple slices are involved)

@sanderr
Copy link
Contributor Author

sanderr commented Jun 29, 2023

In the exporter, I would extend the Unknown with the ResourceSet its resource lives in

"its resource" -> afaik we propagate Unknown through the model, not tracking where it originates (we didn't care up till now). So linking an Unknown in a resource attribute to a fact could be more difficult than just an exporter after-thought.

@sanderr
Copy link
Contributor Author

sanderr commented Jun 29, 2023

(Attributing an unknown to a slice shouldn't be too difficult, attributing an unknown to a fact could be more so.)

@sanderr
Copy link
Contributor Author

sanderr commented Jun 29, 2023

alternatively, this can be added to listener/interceptor the compile service as well

I prefer the other option: adding it to the param service. Third option would be to make this an optional capability of an extension, where the param service has to ask the extension manager if it needs to delegate. I still prefer your first proposal though.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
design Issues with a design aspect
Projects
None yet
Development

No branches or pull requests

2 participants