-
Notifications
You must be signed in to change notification settings - Fork 92
feat: Umpire csv stats output #3622
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Conversation
…mory stats (umpire) with more flexibility
… not useful on any other sub-class
…iloOutput as it is not useful on any other sub-class
errorReporter( isNewlyOpened ? | ||
"Output stream failed to open.\nPossible reasons: File doesn't exist / permissions / locking issue." : | ||
"Output stream is in invalid state for writing.\nPossible reasons: Stream closed / buffer corruption / previous writing failed." ); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since good() check the state of stream, why don't we check this condition every time (headerToStream & dataToStream)?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't understand, as headerToStream()
is based on toStream()
, isn't good()
verified correctly?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What I mean is why we can't get rid of bool isNewlyOpened
and it will simplify the whole process.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is your suggestion to extract this code from toStream()
directly in the TableCSVFormatter
methods?
If not, can you show a short code snippet? I don't visualize what you mean.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I got it, isNewlyOpened
complexifies the code, and is not accurate in some scenarios. I'll remove it and edit the message.
…rcentage computation
…curate in some situations + was complexifying the code
Ok, I think the PR is now ready, it will need a rebaseline because of the |
This PR aims at adding a new
MemoryStats
component to add inOutputs
to control where the Umpire report will be output (log by default, CSV optionally).It also fixes the percentage value computation (which did not match what the description was telling, it was computed only for rank 0 rather than for all ranks).
Here is a generated CSV:
The same CSV aligned for readability:
Here is the new table output, which may be clearer:
This PR also moves the
parallelThreads
attribute fromOutputBase
toSiloOutput
because it was only used there. It didn't felt right to me to have a visible & documented attribute that had no effect, if I understood what it does correctly.