Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[metacling] Optimize TCling*Info interfaces #3616

Conversation

vgvassilev
Copy link
Member

Currently, if ::Name() interface is called we pretty print the Decl name. This is suboptimal because it causes many memory allocations for something which is essentially immutable.

This PR introduces step-by-step working cache if ::Name() was called. It reduces the temporary memory allocations by 12 Mb in standard ROOT and 130Mb in -Druntime_cxxmodules=On cache. The benchmarking test was provided by @pcanal in #3012.

It is important to reduce the temporary allocations because they can contribute to increasing of the peak memory usage of ROOT.

@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from d42a2ad to 5576e44 Compare March 29, 2019 13:21
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@vgvassilev vgvassilev changed the title Meta cling optimize TCling*Info interfaces [metacling] Optimize TCling*Info interfaces Mar 29, 2019
@phsft-bot
Copy link
Collaborator

Build failed on windows10/default.
See console output.

@phsft-bot
Copy link
Collaborator

Build failed on ROOT-performance-centos7-multicore/default.
See console output.

Errors:

  • 87/1881 Test [Math] Avoid redefinition of VECCORE_ENABLE_VC macro #677: tutorial-roostats-CreateExampleFile ...............................................................***Failed Error regular expression found in output. Regex=[: error:] 3.43 sec
  • 839/1881 Test Fix for testArithmeticCpu from TMVA #512: tutorial-histfactory-example ......................................................................***Failed Error regular expression found in output. Regex=[: error:] 5.52 sec
  • 906/1881 Test Fix buffer overflow. #651: tutorial-roofit-rf513_wsfactory_tools .............................................................***Failed Error regular expression found in output. Regex=[: error:] 1.08 sec
  • 935/1881 Test Fixing an O(N^2) scaling problem caused by TTree::Draw() #650: tutorial-roofit-rf512_wsfactory_oper ..............................................................***Failed Error regular expression found in output. Regex=[: error:] 5.02 sec

Failing tests:

And 313 more

@dpiparo
Copy link
Member

dpiparo commented Mar 31, 2019

@phsft-bot build again please.

@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@phsft-bot
Copy link
Collaborator

Build failed on ROOT-performance-centos7-multicore/default.
See console output.

Errors:

  • 86/1881 Test [Math] Avoid redefinition of VECCORE_ENABLE_VC macro #677: tutorial-roostats-CreateExampleFile ...............................................................***Failed Error regular expression found in output. Regex=[: error:] 3.48 sec
  • 835/1881 Test Fix for testArithmeticCpu from TMVA #512: tutorial-histfactory-example ......................................................................***Failed Error regular expression found in output. Regex=[: error:] 5.15 sec
  • 903/1881 Test Fix buffer overflow. #651: tutorial-roofit-rf513_wsfactory_tools .............................................................***Failed Error regular expression found in output. Regex=[: error:] 1.03 sec
  • 931/1881 Test Fixing an O(N^2) scaling problem caused by TTree::Draw() #650: tutorial-roofit-rf512_wsfactory_oper ..............................................................***Failed Error regular expression found in output. Regex=[: error:] 4.64 sec

Failing tests:

And 313 more

@phsft-bot
Copy link
Collaborator

Build failed on windows10/default.
See console output.

@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from 5576e44 to 91f2ecc Compare April 1, 2019 07:17
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@phsft-bot
Copy link
Collaborator

Build failed on ROOT-performance-centos7-multicore/default.
See console output.

Errors:

  • 82/1881 Test [Math] Avoid redefinition of VECCORE_ENABLE_VC macro #677: tutorial-roostats-CreateExampleFile ...............................................................***Failed Error regular expression found in output. Regex=[: error:] 3.36 sec
  • 854/1881 Test Fix for testArithmeticCpu from TMVA #512: tutorial-histfactory-example ......................................................................***Failed Error regular expression found in output. Regex=[: error:] 7.20 sec
  • 905/1881 Test Fix buffer overflow. #651: tutorial-roofit-rf513_wsfactory_tools .............................................................***Failed Error regular expression found in output. Regex=[: error:] 1.13 sec
  • 933/1881 Test Fixing an O(N^2) scaling problem caused by TTree::Draw() #650: tutorial-roofit-rf512_wsfactory_oper ..............................................................***Failed Error regular expression found in output. Regex=[: error:] 4.71 sec

Failing tests:

And 37 more

…ied.

We cannot merge both codepaths because root.exe should always run with modules
if -Druntime_cxxmodules is specified, however, rootcling enables modules only
if -cxxmodule flag is passed.

This patch fixes asserts when ROOT is built in -Druntime_cxxmodules=On but
no -cxxmodule flag is specified (in tree/test for instance).
@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from 91f2ecc to 9e85377 Compare April 1, 2019 10:24
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@root-project root-project deleted a comment from phsft-bot Apr 2, 2019
@root-project root-project deleted a comment from phsft-bot Apr 2, 2019
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@phsft-bot
Copy link
Collaborator

Build failed on windows10/default.
See console output.

@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from 72d9c68 to 53d2202 Compare April 2, 2019 20:30
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@phsft-bot
Copy link
Collaborator

Build failed on windows10/default.
See console output.

@vgvassilev
Copy link
Member Author

Looks like the windows failure is unrelated.

@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from 53d2202 to ae330ea Compare April 2, 2019 21:31
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@vgvassilev
Copy link
Member Author

@pcanal, this is ready for a review.

@phsft-bot
Copy link
Collaborator

Build failed on windows10/default.
See console output.

const clang::Decl *GetDecl() const { return fDecl; } // Underlying representation without Double32_t
void SetDecl(const clang::Decl* D) {
// FIXME: We should track down all set and potentially avoid them.
fDecl = D;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this invalidate the name caches?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, good point.

return fNameCache.c_str();

const clang::TypedefNameDecl *td = llvm::cast<clang::TypedefNameDecl>(fDecl);
const clang::ASTContext &ctxt = GetDecl()->getASTContext();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a different between the value of fDecl and GetDecl() here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, this class does not override GetDecl and the base class implementation returns fDecl.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If they are the same, can we use one of the two in both line (otherwise the future reader will be puzzled :) )

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed.

@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from ae330ea to d2112c6 Compare April 3, 2019 06:58
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, mac1014/cxx17, windows10/default
How to customize builds

@phsft-bot
Copy link
Collaborator

Build failed on windows10/default.
See console output.

void SetDecl(const clang::Decl* D) {
// FIXME: We should track down all sets and potentially avoid them.
fDecl = D;
fNameCache = {}; // invalidate the cache.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that 'better' than fNameCache.clear();

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clear should be faster as it does not make a copy I think. Quickbench shows it is 5 times faster on some implementations...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool. I also find it semantically clearer :)

This patch refactors the code by moving the common parts in a base class. It
implements proper caching if a TClingDataMemberInfo::Name is called multiple
times.

The correct caching saves approximately 100 Mb of temporary allocations in
runtime_modules but also saves 10Mb on some workflows in standard ROOT.
The base class caching reduces the temporary memory allocations.
This patch removes the redundant norm context method arguments.
@vgvassilev vgvassilev force-pushed the meta-cling-optimize-tclingdatamemberinof branch from d2112c6 to 19f1a75 Compare April 4, 2019 19:58
@phsft-bot
Copy link
Collaborator

Starting build on ROOT-performance-centos7-multicore/default, ROOT-fedora27/noimt, ROOT-fedora29/python3, ROOT-ubuntu16/rtcxxmod, ROOT-ubuntu18.04-i386/cxx14, mac1014/cxx17, windows10/default
How to customize builds

@phsft-bot
Copy link
Collaborator

Build failed on ROOT-ubuntu18.04-i386/cxx14.
See console output.

Errors:

  • /mnt/build/workspace/root-pullrequests-build/build/include/VecCore/Backend/Vc.h:21:19: error: ‘VcScalarT’ does not name a type
  • /mnt/build/workspace/root-pullrequests-build/build/include/VecCore/Backend/Vc.h:22:19: error: ‘VcScalar’ does not name a type

@vgvassilev vgvassilev merged commit a7e4d08 into root-project:master Apr 5, 2019
@vgvassilev vgvassilev deleted the meta-cling-optimize-tclingdatamemberinof branch April 5, 2019 05:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants