[docu] Add new ROOT build option: documentation-building

[ferdymercury]

This Pull request:

Changes or fixes:

This allows to build the documentation as part of the normal ROOT build and sets the basis for integrating it into the GitHub CI. And to test new PullRequests so that they do not break the existing documentation nor do add new warnings to it.

You can still build the documentation as an external project, like is done now.

Status

Draft, still work to do on the notebook / tutorial part.

History

This supersedes the old PR #9966 that I just closed.

The goals were:

• Migrate old Makefile to modern CMake. See legacy Makefiles and Configures #9090
• Allow building the documentation from a git source repository with read-only permissions. I.e. try to not pollute the sources within the process. This follows the Make to CMake migration philosophy. See [doc] Building Doxygen documentation pollutes source directory #8947
• As done by ALICE, use a 'dynamic' Doxyfile declaration, that only specifies what needs to be changed with respect to the default one. This makes easier the maintenance, as you do not need to constantly update the Doxyfile when a new doxygen version arises, and prevents warnings when running in older doxygen versions. This will hopefully contribute to next point:
• Automated documentation test for new PR #9953 in combination with https://github.com/ammaraskar/gcc-problem-matcher
• Allow in the future for automatic meta-documentation of the CMake flag system. See Doxygen documentation of CMake flags #8999
• Potentially add a flag in the main ROOT CMakeLists.txt, to enable the building of the documentation via a normal "add_subdirectory()", so that one does not need to build as a separate process.
• Make the doxygen documentation thread-safe
• Don't let doxygen search for "input" files in the output directory (e.g. it looks now for Mathjax output files that one just "generated")

Checklist:

• tested changes locally
• We would need to update https://github.com/root-project/web/blob/main/for_developers/doxygen/index.md
• Update rootspi scripts to build docu in CI accordingly
• tutorial generation does not work yet as "out of source". To-do...
• require min doxygen version 1.9.4
• pre-check that output directory is empty ?
• if GVIZ not found, disable dot
• Stamp file generation to only recompile when changes
• Remove unneeded unnecessary files Images.dat etc ? (or prevent that they appear in root source)
• Make filter.cxx threadsafe but giving unique names to ImageSizes.dat .root etc
• set NUM_PROC_THREADS
• listLibs pass ROOT_CMD as variable
• Understand why libs.C does not find classes such as TBrowserTimer in libCore.so
• check why there are stdout.dat and NumberOfImages not being removed
• Consider enabling https://www.doxygen.nl/manual/config.html#cfg_clang_assisted_parsing
• Fix strip_from_path in root-tag ROOT tag file contains hard coded paths #10059
• Check compress [doxy] qch and tag as compressed files #10066
• Upgrade mathjax [doxy] upgrade to Mathjax3 #10067

[github-actions]

Test Results

    11 files      11 suites   2d 4h 48m 45s ⏱️
 2 635 tests  2 635 ✅ 0 💤 0 ❌
27 419 runs  27 419 ✅ 0 💤 0 ❌

Results for commit a9ba8e7.

♻️ This comment has been updated with latest results.

[couet]

@ferdymercury is this new mechanism, using cmake, ready to replace the current Makefile technique? if yes how to test it?

[ferdymercury]

@ferdymercury is this new mechanism, using cmake, ready to replace the current Makefile technique? if yes how to test it?

It's almost ready. Just missing the notebook and tutorial part.

To test it, just clone my branch.

Then:

Compile ROOT with CMake flag -Ddocu=ON -DDOCU_LOCATION=/tmp/docoutput -DDOCU_THREADS=8 -DDOCU_LOGFILE=/tmp/warnings.txt -DDOXYGEN_EXECUTABLE=/opt/doxygen/build/bin/doxygen -DDOCU_INPUT="./mainpage.md;../../core"

You probably also need to disable WebCanvas in your .rootrc

Note, only specify DOCU_INPUT if you want to build a small part of the docu, to be quicker. Otherwise, it will get the default, which is to build all folders.

[couet]

I will try when ready

[ferdymercury]

I will try when ready

I'd say it's ready to try now.

[couet]

@ferdymercury
I took the branch from #15314 . I issued the command suggested in you previous comment:

cmake ../root-new-doc -Ddocu=ON -DDOCU_LOCATION=/tmp/docoutput -DDOCU_THREADS=8 -DDOCU_LOGFILE=/tmp/warnings.txt -DDOXYGEN_EXECUTABLE=/Users/couet/bin/doxygen -DDOCU_INPUT="./mainpage.md;../../core"

and I get:

CMake Error at documentation/doxygen/CMakeLists.txt:58 (STRING):
  STRING sub-command REGEX, mode REPLACE needs at least 6 arguments total to
  command.

[jolly-chen]

@ferdymercury This seems to be caused by gROOT->GetGitBranch\(\) failing at

root/documentation/doxygen/CMakeLists.txt

Line 46 in eb2d443

execute_process(COMMAND ${ROOT_root_CMD} -l -b -q -e gROOT->GetGitBranch\(\) OUTPUT_VARIABLE ROOT_GIT_VERSION OUTPUT_STRIP_TRAILING_WHITESPACE)

because there is no root.exe yet when building doxygen as a step of the ROOT build.

@couet Can you try building without documentation first (-Ddocu=off), run source bin/thisroot.sh, then build again with -Ddocu=on)? This should work.

[ferdymercury]

@ferdymercury I took the branch from #15314 . I issued the command suggested in you previous comment:

cmake ../root-new-doc -Ddocu=ON -DDOCU_LOCATION=/tmp/docoutput -DDOCU_THREADS=8 -DDOCU_LOGFILE=/tmp/warnings.txt -DDOXYGEN_EXECUTABLE=/Users/couet/bin/doxygen -DDOCU_INPUT="./mainpage.md;../../core"

and I get:

CMake Error at documentation/doxygen/CMakeLists.txt:58 (STRING):
  STRING sub-command REGEX, mode REPLACE needs at least 6 arguments total to
  command.

Thanks for the review! This might be a platform-dependent issue, it works well in Ubuntu. Or it might be that you are using a tarball instead of git repository?

So the conflicting line is:

STRING(REGEX REPLACE "^\n\\(const char \\*\\) " "" ROOT_GIT_VERSION ${ROOT_GIT_VERSION})

which is called after:

execute_process(COMMAND ${ROOT_root_CMD} -l -b -q -e gROOT->GetGitBranch() OUTPUT_VARIABLE ROOT_GIT_VERSION OUTPUT_STRIP_TRAILING_WHITESPACE)

Could you post here the output of ROOT_GIT_VERSION before calling REPLACE ? So that I know what to fix.

[ferdymercury]

@jolly-chen ohh I see what you mean.

Maybe it's better then to add a dependency on root.exe for this ?

[jolly-chen]

@ferdymercury Yes I think this would be fixed with this todo

root/documentation/doxygen/CMakeLists.txt

Line 22 in eb2d443

# TODO: https://stackoverflow.com/questions/16408060/how-do-i-add-a-dependency-on-a-script-to-a-target-in-cmake

[ferdymercury]

@couet Can you retry now after applying this commit ?

a720d79

[couet]

I'll try and let you know

[couet]

I am a bit lost now .. this branch has conflicts ... how to get again the latest version ... ? there is this PR and this one: #15314 that's very confusing

[ferdymercury]

I am a bit lost now .. this branch has conflicts ... how to get again the latest version ... there is this PR and this one #15314 that's very confusing

I've resolved the conflicts now. Try with this one, the other PR is just for later on.

[couet]

So, I cloned the ROOT repo in the folder docgmake. Then in that cloned folder I did gh pr checkout 15160 to get this branch. Then cd ..; mkdir docgmake-bin; cd docgmake-bin. And in that bin folder I issued the command:

cmake ../docgmake -Ddocu=ON -DDOCU_LOCATION=~/rootdoc -DDOCU_THREADS=8 -DDOCU_LOGFILE=/tmp/warnings.txt -DDOXYGEN_EXECUTABLE=/Users/couet/bin/doxygen -DDOCU_INPUT="./mainpage.md;../../core"

then make -j8

which gives me:

...
[100%] Built target modules_idx
[100%] Generating tutorials/hsimple.root

Processing hsimple.C...
hsimple   : Real Time =   0.23 seconds Cpu Time =   0.08 seconds
(TFile *) 0x7f77c78bd300
[100%] Built target hsimple
/bin/sh: -DROOT_COMMAND=\"ROOT_root_CMD-NOTFOUND\": command not found
make[2]: *** [documentation/doxygen/CMakeFiles/Preparation] Error 127
make[1]: *** [documentation/doxygen/CMakeFiles/Preparation.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Built target PreparationPyz
make: *** [all] Error 2

So it seems the gmake command was wrong.

[jolly-chen]

@couet can you try again with this commit da41596?

(sorry accidentally closed the PR)

[couet]

I am rebuilding. By the way, why do we need to specify DOXYGEN_EXECUTABLE ? If a valid oxygen version is in $PATH we should not be obliged to specify this variable.