Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DF] Separate computation graph from dataset it should run on #15070

Open
martamaja10 opened this issue Mar 27, 2024 · 0 comments
Open

[DF] Separate computation graph from dataset it should run on #15070

martamaja10 opened this issue Mar 27, 2024 · 0 comments

Comments

@martamaja10
Copy link
Contributor

Explain what you would like to see improved and how.

Moving from JIRA: ROOT-9438

Advanced users (e.g. TMVA internals) might want to define a computation graph once (e.g. by jitting) and then apply it to several different data-sets or several subsets of a dataset in sequence, without having to re-generate the computation graph each time.
The introduction of a new type of head node, a "computation manager", which can attach to different "loop managers", would be one relatively simple way to allow this.
Common usage of RDataFrame would remain the same, with the loop manager using a computation manager to "speak" to the computation graph under the hood. Explicit usage of the computation manager node would allow to attach to different loop managers.
At least the schema of the dataset (or the subset that one expects to read) should probably still be known a priori, to allow RDataFrame to error out early in case of misspelled column name, inexistent branch name and so on.
To allow the reusage of the same computation graph, actions should not be deleted at the end of the event loop as it is currently done but kept around for the next run.

ROOT version

all

Installation method

all

Operating system

all

Additional context

No response

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants