[Flang] Add Sphinx man page and html support for Flang #141882

snarang181 · 2025-05-29T00:26:20Z

This patch refactors the Flang documentation CMake and Sphinx configuration to address build issues.

CMake changes:

Moves the gen_rst_file_from_td() call out of the HTML-only block so that both docs-flang-html and docs-flang-man builds depend on the generated FlangCommandLineReference.rst file.

conf.py changes:

Introduces myst_parser dependency as a required Markdown parser for both HTML and man builds.
Introduces the correct source_suffix mapping for both .rst and .md files.
Populates the man_pages configuration so the main index page generates a flang(1)man page.

Fixes #141757

github-actions · 2025-05-29T00:26:41Z

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

snarang181 · 2025-05-29T00:31:15Z

@pawosm-arm -- Sorry to tag you explicitly, I do not have access to formally request reviews using the Reviewers section.

pawosm-arm · 2025-05-29T07:37:33Z

@pawosm-arm -- Sorry to tag you explicitly, I do not have access to formally request reviews using the Reviewers section.

No worries, I will look into it today.

github-actions · 2025-05-29T07:40:47Z

✅ With the latest revision this PR passed the Python code formatter.

pawosm-arm · 2025-05-29T09:17:55Z

I think more people need to look into it. AFAIR the plan was that flang does not use the .rst files for documentation, it was supposed to rely on the markdown files entirely.

snarang181 · 2025-05-29T11:07:55Z

I think more people need to look into it. AFAIR the plan was that flang does not use the .rst files for documentation, it was supposed to rely on the markdown files entirely.

@pawosm-arm, thanks for looking into it. I got rid of the .rst files. When I added the explicit source suffix, it seemed like the man-page build needed the parser for the .md files. With the myst_parser library not being imported, I was hitting parsing errors of the .md files, so I explicitly changed the requirement. With that and adding placeholder missing .md files, I can generate a man doc at ${BUILD_DIR}/tools/flang/docs/man/flang.1

kiranchandramohan · 2025-05-29T11:11:04Z

ninja flang-doc might be generating files like FIRLangRef.md. Would adding these files manually affect that?
Note that the HTML files generated from these are displayed currently in the following pages:
https://flang.llvm.org/docs/
https://flang.llvm.org/docs/FIRLangRef.html
https://flang.llvm.org/docs/FlangCommandLineReference.html

snarang181 · 2025-05-29T11:13:28Z

ninja flang-doc might be generating files like FIRLangRef.md. Would adding these files manually affect that?

Note that the HTML files generated from these are displayed currently in the following pages:

https://flang.llvm.org/docs/

https://flang.llvm.org/docs/FIRLangRef.html

https://flang.llvm.org/docs/FlangCommandLineReference.html

It did not seem like the build process got affected locally for me. Is there any other way of testing this out?

kiranchandramohan · 2025-05-29T11:31:49Z

Can you enable -DLLVM_ENABLE_SPHINX=ON -DSPHINX_WARNINGS_AS_ERRORS=OFF in the build options and try ninja docs-flang-html?
The patch that added FIRLangRef.md to docs is https://reviews.llvm.org/D128650

snarang181 · 2025-05-29T12:13:09Z

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

kiranchandramohan · 2025-05-29T12:25:44Z

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

snarang181 · 2025-05-29T12:40:26Z

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

As I mentioned the man page produced at ${BUILD_DIR}/tools/flang/docs/man/flang.1 looks like the one produced by tablegen. The FIRLangRef.md looks to be completely overwritten in the build directory.
For the other placeholder file, FlangCommandLineReference.md, there is no fallout either.

kiranchandramohan · 2025-05-29T12:51:34Z

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

As I mentioned the man page produced at ${BUILD_DIR}/tools/flang/docs/man/flang.1 looks like the one produced by tablegen. The FIRLangRef.md looks to be completely overwritten in the build directory. For the other placeholder file, FlangCommandLineReference.md, there is no fallout either.

Can we do something similar to https://reviews.llvm.org/D128650 to avoid needing dummy FIRLangRef.md and FlangCommandLineReference?

snarang181 · 2025-05-29T13:03:38Z

https://reviews.llvm.org/D128650

@kiranchandramohan -- this runs for me successfully and does not interfere with the placeholder file. The final FIRLangRef.md that is produced in the build artifacts has been generated by tablegen.

What about the man pages and its contents. Is it the empty file or the one produced by tablegen?

As I mentioned the man page produced at ${BUILD_DIR}/tools/flang/docs/man/flang.1 looks like the one produced by tablegen. The FIRLangRef.md looks to be completely overwritten in the build directory. For the other placeholder file, FlangCommandLineReference.md, there is no fallout either.

Can we do something similar to https://reviews.llvm.org/D128650 to avoid needing dummy FIRLangRef.md and FlangCommandLineReference?

@kiranchandramohan, I removed the placeholder .md files and the man pages get generated now. Just a warning of the form
WARNING: toctree contains reference to nonexisting document 'FIRLangRef'
WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference'
This is probably OK?

tarunprabhu · 2025-05-29T14:31:49Z

@kiranchandramohan, I removed the placeholder .md files and the man pages get generated now. Just a warning of the form WARNING: toctree contains reference to nonexisting document 'FIRLangRef' WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference' This is probably OK?

It might not be ok. If the buildkite fails, as it does here, it will probably break buildbots too. I am not sure what is causing this though.

snarang181 · 2025-05-29T14:35:10Z

@kiranchandramohan, I removed the placeholder .md files and the man pages get generated now. Just a warning of the form WARNING: toctree contains reference to nonexisting document 'FIRLangRef' WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference' This is probably OK?

It might not be ok. If the buildkite fails, as it does here, it will probably break buildbots too. I am not sure what is causing this though.

Looking further into it.

snarang181 · 2025-05-30T02:04:10Z

@kiranchandramohan @tarunprabhu, what do you guys think about modifying index.md in the following manner ? This will load things on demand and we should not be hitting the nonexistent document errors.

```{eval}
import os
if os.path.exists('FlangCommandLineReference.rst'):
    print('FlangCommandLineReference')

kiranchandramohan · 2025-05-30T09:42:13Z

@kiranchandramohan @tarunprabhu, what do you guys think about modifying index.md in the following manner ? This will load things on demand and we should not be hitting the nonexistent document errors.
```{eval}
import os
if os.path.exists('FlangCommandLineReference.rst'):
    print('FlangCommandLineReference')

I would prefer something like https://reviews.llvm.org/D128650 where the generated md/rst files are copied by the docs-flang-man target before it hits the code that triggers the nonexistent document warning/errors.

snarang181 · 2025-05-30T18:53:07Z

Hi @kiranchandramohan @tarunprabhu, with my latest commit the html target is built without any warnings. When I trigger the man build via ninja docs-flang-man,
I see

WARNING: toctree contains reference to nonexisting document 'FIRLangRef'
WARNING: toctree contains reference to nonexisting document 'FlangCommandLineReference'

If we build with -DSPHINX_WARNINGS_AS_ERRORS=OFF, this will lead to the man page successfully being built. If the flag is not set, we will hit a fatal error and fail the man build. Should we add this as a TODO and open another issue for that? I think this patch is tailored for the missing man_config issue and man not being built at all. Let me know what you guys think.

snarang181 · 2025-05-31T00:53:02Z

@kiranchandramohan, @tarunprabhu, @pawosm-arm
Hi guys, I think this PR is ready for review. We can successfully build both HTML and man targets with these changes. Let me know how it looks.
Thanks.

Modify man page doc

kiranchandramohan · 2025-06-04T13:50:31Z

I'd accept this, but the others must agree with the introduction of the rst files next to md files.

I will defer to @kiranchandramohan on the introduction of .rst files.

I am OK if it can be put in a sub-directory like clang (docs/CommandGuide).

snarang181 · 2025-06-04T13:52:44Z

I'd accept this, but the others must agree with the introduction of the rst files next to md files.

I will defer to @kiranchandramohan on the introduction of .rst files.

I am OK if it can be put in a sub-directory like clang (docs/CommandGuide).

Just to be clear, you mean putting index.rst in something like flang/docs/CommandGuide?

snarang181 · 2025-06-04T15:12:29Z

I am OK if it can be put in a sub-directory like clang (docs/CommandGuide).

@kiranchandramohan, moved. index.rst to docs/CommandGuide

kiranchandramohan

LGTM. Thanks for all the changes. Please wait for @tarunprabhu or @pawosm-arm

pawosm-arm · 2025-06-04T16:37:58Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:

WARNING: multiple files found for the document "index": index.md, index.rst

snarang181 · 2025-06-04T16:40:04Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:
WARNING: multiple files found for the document "index": index.md, index.rst

Are you building man or HTML?

pawosm-arm · 2025-06-04T16:48:14Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:
WARNING: multiple files found for the document "index": index.md, index.rst
Are you building man or HTML?

This is a simple change I've added to the build script (I presume this is how it will be used by the others too):

    -DLLVM_BUILD_DOCS=ON
    -DLLVM_ENABLE_SPHINX=ON

This results in the following lines being generated in the CMakeCache.txt file:

//Generate build targets for the Flang docs.
FLANG_INCLUDE_DOCS:BOOL=ON

//HTML documentation install directory for flang
FLANG_INSTALL_SPHINX_HTML_DIR:STRING=share/doc/LLVM/flang/html

//Output standalone HTML files
SPHINX_OUTPUT_HTML:BOOL=ON

//Output man pages
SPHINX_OUTPUT_MAN:BOOL=ON

//When building documentation treat warnings as errors
SPHINX_WARNINGS_AS_ERRORS:BOOL=ON

From this you can read I'm building both.

snarang181 · 2025-06-04T16:53:48Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:
WARNING: multiple files found for the document "index": index.md, index.rst
Are you building man or HTML?
This is a simple change I've added to the build script (I presume this is how it will be used by the others too):
    -DLLVM_BUILD_DOCS=ON
    -DLLVM_ENABLE_SPHINX=ON
This results in the following lines being generated in the CMakeCache.txt file:
//Generate build targets for the Flang docs.
FLANG_INCLUDE_DOCS:BOOL=ON

//HTML documentation install directory for flang
FLANG_INSTALL_SPHINX_HTML_DIR:STRING=share/doc/LLVM/flang/html

//Output standalone HTML files
SPHINX_OUTPUT_HTML:BOOL=ON

//Output man pages
SPHINX_OUTPUT_MAN:BOOL=ON

//When building documentation treat warnings as errors
SPHINX_WARNINGS_AS_ERRORS:BOOL=ON
From this you can read I'm building both.

So I have both enabled too according to my local CMakeCache.txt and I am able to build both targets. This error was seen before, just as a sanity check, do you have a clean build?

pawosm-arm · 2025-06-04T16:59:20Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:
WARNING: multiple files found for the document "index": index.md, index.rst
Are you building man or HTML?
This is a simple change I've added to the build script (I presume this is how it will be used by the others too):
    -DLLVM_BUILD_DOCS=ON
    -DLLVM_ENABLE_SPHINX=ON
This results in the following lines being generated in the CMakeCache.txt file:
//Generate build targets for the Flang docs.
FLANG_INCLUDE_DOCS:BOOL=ON

//HTML documentation install directory for flang
FLANG_INSTALL_SPHINX_HTML_DIR:STRING=share/doc/LLVM/flang/html

//Output standalone HTML files
SPHINX_OUTPUT_HTML:BOOL=ON

//Output man pages
SPHINX_OUTPUT_MAN:BOOL=ON

//When building documentation treat warnings as errors
SPHINX_WARNINGS_AS_ERRORS:BOOL=ON
From this you can read I'm building both.
So I have both enabled too according to my local CMakeCache.txt and I am able to build both targets. This error was seen before, just as a sanity check, do you have a clean build?

I did a clean build after cherry-picking your recent commits to this PR over the same base commit (to which I'm doing git-reset each time you do a force-push). I don't know if these messages were seen before, as the build was passing before those most recent cherry-picks so not much has changed really.

snarang181 · 2025-06-04T17:27:10Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:
WARNING: multiple files found for the document "index": index.md, index.rst
Are you building man or HTML?
This is a simple change I've added to the build script (I presume this is how it will be used by the others too):
    -DLLVM_BUILD_DOCS=ON
    -DLLVM_ENABLE_SPHINX=ON
This results in the following lines being generated in the CMakeCache.txt file:
//Generate build targets for the Flang docs.
FLANG_INCLUDE_DOCS:BOOL=ON

//HTML documentation install directory for flang
FLANG_INSTALL_SPHINX_HTML_DIR:STRING=share/doc/LLVM/flang/html

//Output standalone HTML files
SPHINX_OUTPUT_HTML:BOOL=ON

//Output man pages
SPHINX_OUTPUT_MAN:BOOL=ON

//When building documentation treat warnings as errors
SPHINX_WARNINGS_AS_ERRORS:BOOL=ON
From this you can read I'm building both.
So I have both enabled too according to my local CMakeCache.txt and I am able to build both targets. This error was seen before, just as a sanity check, do you have a clean build?
I did a clean build after cherry-picking your recent commits to this PR over the same base commit (to which I'm doing git-reset each time you do a force-push). I don't know if these messages were seen before, as the build was passing before those most recent cherry-picks so not much has changed really.

No, you are right, I hadn't done the builds back to back, just independently. I have a fix, pushing it.

snarang181 · 2025-06-04T17:31:38Z

@pawosm-arm, can you test it out?

snarang181 · 2025-06-04T17:45:41Z

I'm trying to build it, but it fails, I presume due to warnings being treated as errors as I can't see any actual errors in the log. Does it build for you?:
WARNING: multiple files found for the document "index": index.md, index.rst
Are you building man or HTML?
This is a simple change I've added to the build script (I presume this is how it will be used by the others too):
    -DLLVM_BUILD_DOCS=ON
    -DLLVM_ENABLE_SPHINX=ON
This results in the following lines being generated in the CMakeCache.txt file:
//Generate build targets for the Flang docs.
FLANG_INCLUDE_DOCS:BOOL=ON

//HTML documentation install directory for flang
FLANG_INSTALL_SPHINX_HTML_DIR:STRING=share/doc/LLVM/flang/html

//Output standalone HTML files
SPHINX_OUTPUT_HTML:BOOL=ON

//Output man pages
SPHINX_OUTPUT_MAN:BOOL=ON

//When building documentation treat warnings as errors
SPHINX_WARNINGS_AS_ERRORS:BOOL=ON
From this you can read I'm building both.
@pawosm-arm
What was going on: When we did a ninja docs-flang-man followed by ninja docs-flang-html, there were two refs to an index file -- one with a .md and another with an .rst. This was mainly because we had the same CMAKE_BINARY_DIR/Source dir. So I have separated the Source directories for both the builds to have independent workflows (this can be removed of course once we unify the index file).

pawosm-arm · 2025-06-04T18:06:33Z

@pawosm-arm, can you test it out?

Yes I'll trigger another build today.

pawosm-arm · 2025-06-04T21:10:54Z

@pawosm-arm, can you test it out?

Yes I'll trigger another build today.

It looks much better now. Although the Introduction section looks weird (the way it starts going directly to listing the command line options), but I tend to agree that it's a good for the starters and the proper refinement should happened later and should wait for the contributors to pick it up. For now it looks as such:

FLANG(1)                                Flang                               FLANG(1)

NAME
       flang - Flang Documentation (In Progress)

       NOTE:
          This man page is under development.

       For full documentation, please see the online HTML docs:

       https://flang.llvm.org/docs/

                                           ----

FLANG COMMAND LINE REFERENCE
   Flang command line argument reference
       o Introduction

       o Actions

       o Compilation options

         o Preprocessor options

           o Include path management

           o Dumping preprocessor state

         o Diagnostic options

         o Target-independent compilation options

           o Common Offloading options

           o HIP options

         o Target-dependent compilation options

           o AARCH64

           o X86

           o LoongArch

         o Optimization level

         o Debug information generation

           o Kind and level of debug information

             o Debug level

       o Linker options

   Introduction
       -B<prefix>

       Search $prefix$file for executables, libraries, and data files. If $prefix is
       a directory, search $prefix/$file

       -Qunused-arguments

       Don't emit warning for unused driver arguments

       --config=<file>, --config <arg>
 Manual page flang(1) line 1 (press h for help or q to quit)

snarang181 · 2025-06-04T21:14:48Z

Whoever has write-access, please merge this.

pawosm-arm · 2025-06-04T21:25:05Z

Whoever has writeaccess, please merge this.

Done

github-actions · 2025-06-04T21:25:05Z

@snarang181 Congratulations on having your first Pull Request (PR) merged into the LLVM Project!

Your changes will be combined with recent changes from other authors, then tested by our build bots. If there is a problem with a build, you may receive a report in an email or a comment on this PR.

Please check whether problems have been caused by your change specifically, as the builds can include changes from many authors. It is not uncommon for your change to be included in a build that fails due to someone else's changes, or infrastructure issues.

How to do this, and the rest of the post-merge process, is covered in detail here.

If your change does cause a problem, it may be reverted, or you can revert it yourself. This is a normal part of LLVM development. You can fix your changes and open a new PR to merge them again.

If you don't get any reports, no action is required from you. Your changes are working as expected, well done!

This commit removes the disablement of generating the flang docs (man pages). It was disabled as it had been failing. We had to wait until a fix for generating flang manpages has landed upstream. See llvm/llvm-project#141882

llvmbot added the flang Flang issues not falling into any other category label May 29, 2025

snarang181 changed the title ~~[Flang][Docs] Add Sphinx man page support for Flang~~ [Flang] Add Sphinx man page support for Flang May 29, 2025

pawosm-arm requested review from Meinersbur, kiranchandramohan and naromero77 May 29, 2025 09:21

kiranchandramohan requested review from tarunprabhu and pawosm-arm May 29, 2025 10:06

snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from 4f24ea6 to bee779e Compare May 30, 2025 16:27

snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from 1d69ea5 to cd886e1 Compare May 31, 2025 00:08

snarang181 changed the title ~~[Flang] Add Sphinx man page support for Flang~~ [Flang] Add Sphinx man page and html support for Flang May 31, 2025

Remove redundant command in CMake

737c725

Modify man page doc

snarang181 force-pushed the snarang181/flang-manpage-build-fix branch from a33bc37 to 737c725 Compare June 4, 2025 11:13

snarang181 added 3 commits June 4, 2025 10:10

Move index.rst to docs/CommandGuide

0f256db

Adjust path to CommandLineReference file

149036a

relative path change in removing of CommandLineGuide

ef6fa37

kiranchandramohan approved these changes Jun 4, 2025

View reviewed changes

snarang181 requested a review from tarunprabhu June 4, 2025 15:46

Samarth Narang added 2 commits June 4, 2025 10:13

Separate out build directories for man and html builds

0c24894

Modify build script to use HTML routes

b1ef915

Explicitly copy generated rst file for HTML build

4eee2e8

pawosm-arm approved these changes Jun 4, 2025

View reviewed changes

pawosm-arm merged commit 47171ac into llvm:main Jun 4, 2025
10 checks passed

pawosm-arm mentioned this pull request Jun 6, 2025

[ATfL] build.sh modification to generate flang/armflang manpages arm/arm-toolchain#363

Merged

[Flang] Add Sphinx man page and html support for Flang #141882

[Flang] Add Sphinx man page and html support for Flang #141882

Conversation

snarang181 commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

pawosm-arm commented May 29, 2025

Uh oh!

github-actions bot commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pawosm-arm commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

kiranchandramohan commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

kiranchandramohan commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

kiranchandramohan commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

kiranchandramohan commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

tarunprabhu commented May 29, 2025

Uh oh!

snarang181 commented May 29, 2025

Uh oh!

snarang181 commented May 30, 2025

Uh oh!

kiranchandramohan commented May 30, 2025

Uh oh!

snarang181 commented May 30, 2025

Uh oh!

snarang181 commented May 31, 2025

Uh oh!

kiranchandramohan commented Jun 4, 2025

Uh oh!

snarang181 commented Jun 4, 2025

Uh oh!

snarang181 commented Jun 4, 2025

Uh oh!

kiranchandramohan left a comment

Choose a reason for hiding this comment

Uh oh!

pawosm-arm commented Jun 4, 2025

Uh oh!

snarang181 commented Jun 4, 2025

Uh oh!

pawosm-arm commented Jun 4, 2025

Uh oh!

snarang181 commented Jun 4, 2025

Uh oh!

pawosm-arm commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

snarang181 commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

snarang181 commented Jun 4, 2025

Uh oh!

snarang181 commented Jun 4, 2025

Uh oh!

pawosm-arm commented Jun 4, 2025

Uh oh!

pawosm-arm commented Jun 4, 2025

Uh oh!

snarang181 commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

snarang181 commented May 29, 2025 •

edited

Loading

github-actions bot commented May 29, 2025 •

edited

Loading

pawosm-arm commented Jun 4, 2025 •

edited

Loading

snarang181 commented Jun 4, 2025 •

edited

Loading

snarang181 commented Jun 4, 2025 •

edited

Loading