Skip to content

Separate macro definitions and add cppm files #293

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
May 17, 2025

Conversation

SidneyCogdill
Copy link
Contributor

@SidneyCogdill SidneyCogdill commented May 11, 2025

  • Moved "proxy.h" to <proxy/proxy.h>, to match the installed location (Oh the chaotic amount of changed files). Now that both the installed version (vcpkg/conan) and the FetchContent_Declare / add_subdirectory has the same include hierarchy for consistency.
  • Added .cppm files for modules (similar to vulkan.cppm)
  • Extracted the convenient macros into a separate file (Usable with modules).
  • Added a basic test case for C++20 modules
  • Added docs to mention C++20 modules support, including an example.

I'm still undecided whether to ship two separate proxy / proxy_interface modules, or to merge both into proxy (causing implementation details to be exported in proxy module). There's now only one proxy module.

Fixes #295.

and ignore this thx

For historical reference, in previous commits the module support is shipped with msft_proxy target:

For installed version (e.g. the one package manager does), import proxy should work sort of work after find_package(proxy) and target_link_libraries as the cppm files are exported by the target. (I haven't tested it yet.) Consumer target must set to C++20 mode. C++23/26 mode causes the following compile time error for Clang 21:

[build] error: C++23 was disabled in AST file 'CMakeFiles/msft_proxy@synth_02a900fb16f7.dir/f7d7a83617b3.bmi' but is currently enabled
[build] error: module file CMakeFiles/msft_proxy@synth_02a900fb16f7.dir/f7d7a83617b3.bmi cannot be loaded due to a configuration mismatch with the current compilation [-Wmodule-file-config-mismatch]

@SidneyCogdill
Copy link
Contributor Author

SidneyCogdill commented May 11, 2025

The module works when it's manually declared. It doesn't work reliably when exported by CMake install. CMake seems to be doing more than just making all the CXX_MODULES files available to the current target (like how traditional headers work) when compiling (perhaps to share BMIs across different targets?) but that causes issues when current target has different C++ standard.

VulkanHpp provided this helper CMake script for modules: ...

I think I can make import proxy; work reliably using an alternate approach. By manually placing the helper script within the proxyConfig.cmake, rather than going through the CMake export process (as a workaround), until CMake export works as expected.

The reason modules reliably worked when manually declared is because manually declared targets are affected by CMAKE_CXX_STANDARD, therefore it gets the same C++ standard between the library and the consumer. imported targets on the other hand doesn't seem to be affected by that global variable.

Shipping the module in a separate CMake target with a different (preferably the highest currently supported) C++ standard doesn't work well either: It places a much stricter requirement on the consumer compiler version. Too old and it will not work (for example older Clang which doesn't recognize -std=c++26 option).

The best way to provide modules support today seems to be providing only the .cppm files without binding them to the msft_proxy target. Similar to VulkanHpp, where the instruction to manually declare a module target is provided in the README.md. This way the user can manually override the C++ standard version for the proxy module. This is until CMake properly implements forward compatibility.

@SidneyCogdill
Copy link
Contributor Author

It seems I've implemented the module support correctly. It's just that CMake hasn't implemented forward compatibility support for C++ modules yet: https://gitlab.kitware.com/cmake/cmake/-/issues/25909#note_1525432 .

For now the exported module requires that the consumer has lower standard mode than proxy. For example if I change proxy itself to build in C++26 mode (and then install it), then the consumer can freely use C++20/23/26 mode.

SidneyCogdill added 2 commits May 11, 2025 12:42
and provide workarounds in the docs.
Copy link
Collaborator

@mingxwa mingxwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the PR! The direction looks promising. Except for the inline comments, I don't feel the existing build system is robust enough, specifically

  • The CI pipeline did not pass. We should ensure this change won't break existing code. However, I manually tested with the latest MSVC with CMake, and the build passed.
  • Integration with other modules could be challenging. For example, if a user's build system prefers import std;, the includes in the proxy module may generate ODR violation at some point. It could be even more challenging if a user uses the fmt library with their module interface. We should ensure these scenarios are covered by unit tests.

@mingxwa mingxwa requested review from tian-lt and guominrui May 11, 2025 15:13
1. Removed details directory
2. Use 2 space indention
3. `.ixx` suffix for module
4. Merged `proxy_interface` into `proxy`
@SidneyCogdill
Copy link
Contributor Author

Integration with other modules could be challenging. For example, if a user's build system prefers import std;, the includes in the proxy module may generate ODR violation at some point. It could be even more challenging if a user uses the fmt library with their module interface. We should ensure these scenarios are covered by unit tests.

for stdlib it shouldn't be a problem (they implemented the extern "C++" workaround). Mixing import std; and #include should work as long as the user properly #include-ed stdlib in global fragment (#include then import works; the reverse way seems to not work reliably).

for fmt they have the FMT_ATTACH_TO_GLOBAL_MODULE macro to enable the extern "C++" block which also mitigates the issue: https://github.com/fmtlib/fmt/blob/7af94e55979ecaa0357ee52e0ad41ed8b050fc14/src/fmt.cc#L107

But yeah, I haven't written unit tests for it yet because import std; is still experimental in CMake and getting import fmt; to work seems to be not as straightforward as VulkanHpp (clearly documented in README).

@SidneyCogdill
Copy link
Contributor Author

SidneyCogdill commented May 12, 2025

The CI pipeline did not pass. We should ensure this change won't break existing code.

I think the CIs are failing on non-Windows systems because CMake doesn't have module support for Unix Makefile. Could they be changed to Ninja instead?

Alternatively I can disable the module CMake target by default so that CI will succeed (but I'm not sure what to do with the doc page that provides modules example).

SidneyCogdill added 2 commits May 12, 2025 00:41
and change some "proxy.h" to <proxy/proxy.h>
@SidneyCogdill
Copy link
Contributor Author

GCC 14.2 works with the PR in my local environment. Except GCC seems to required #include <tuple> for std::ignore (used by the PRO_DEF_ macro)? https://en.cppreference.com/w/cpp/utility/tuple/ignore says it's part of <utility>.

@SidneyCogdill
Copy link
Contributor Author

I've changed the GitHub Actions to use -GNinja. Let's give it a try (and fix whatever is failing). I think older compilers might have problem with modules support.

... if it's consumed as subdirectory.
@mingxwa
Copy link
Collaborator

mingxwa commented May 12, 2025

I've changed the GitHub Actions to use -GNinja. Let's give it a try (and fix whatever is failing). I think older compilers might have problem with modules support.

I think we should either detect unsupported compilers in CMakeLists.txt (like freestanding tests) or specifying a flag through the command line in the yaml files.

@SidneyCogdill
Copy link
Contributor Author

The module test isn't hard to address (use a variable to control enabling/disabling the test case). The C++20 module example codes extracted from docs are a different story. Currently it's extracted with Python script and all sources are globbed. I don't think it can support declaring an example as "only enabled when XXX variable is defined". Changing them to not use glob would work though.

@mingxwa
Copy link
Collaborator

mingxwa commented May 12, 2025

The C++20 module example codes extracted from docs are a different story. Currently it's extracted with Python script and all sources are globbed.

I filed #296 to allow exclusions. Please help review

The doc test is now capable of handling multiple
sources in a doc, with template support for CMake
target definition.

The C++ module example is also updated to take
advantage of the new doc test generator.

GitHub Actions is tweaked:

- The compatibility CI now has reversed order,
sorting from latest to oldest.
- Known-incompatible compilers now have module
disabled.
@SidneyCogdill
Copy link
Contributor Author

SidneyCogdill commented May 12, 2025

The extract_example_code_from_docs.py is updated.

With an actual markdown parser, it's now capable of resolving multiple code blocks within the ## Example section and adding them to the same CMake target. I've tried to tame the regular expression but it fails shortly with complex logic, hence the new parser.

You can also use the XXX.cmake.in template file to customize CMakeLists.txt output (check the cpp20_modules_support.cmake.in file in the commit). This makes it a much more general solution than the file exclusion mechanism.

Also much of the complexity of the doc test generator is now moved into the Python script. (IMO Python is much more readable and maintainable than CMake when the control flow gets complex)

SidneyCogdill added 3 commits May 12, 2025 08:08
SidneyCogdill added 2 commits May 12, 2025 09:33
I forgot to actually use it to decide if module
target should be declared. How careless.
This time I locally tested it rather than assuming it will work.
And it compiles with -D=PROXY_BUILDING_WITH_MODULE=FALSE.
Copy link
Collaborator

@mingxwa mingxwa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your effort to make the proxy module pass CI build! This journey was really impressive.

1. Reverted `extract_example_code_from_docs.py`.
2. Added "Module definition:" and "Client:" to cpp20_modules_support.md,
to exclude it from the doc test generator.
3. Minor formatting issues (trailing EOL, empty lines ...) are fixed.
@SidneyCogdill SidneyCogdill force-pushed the feature/cpp20-modules branch from f957093 to 09ea107 Compare May 15, 2025 03:42
@SidneyCogdill
Copy link
Contributor Author

Note: do not merge yet. I'm testing locally if it works as expected for all the toolchain combinations I have access to.

mingxwa

This comment was marked as outdated.

@SidneyCogdill
Copy link
Contributor Author

Testing locally with CPM uncovered this problem: apparently CPM prevents the variables set by subdirectory projects from propagating to parent project. On the other hand CPM automatically sets a proxy_SOURCE_DIR variable instead which can be used to locate the include directory.

Note this issue doesn't affect FetchContent_Declare.

The issue is addressed in 62f6c4d .

include(cmake/CPM.cmake)
CPMAddPackage(
  NAME proxy
  GIT_TAG 62f6c4d1370c76a2f6a00cb16babc0092beee019
  GIT_REPOSITORY https://github.com/SidneyCogdill/proxy.git
)

Since with CPM there's no downstream packager, the microsoft/proxy repository itself is the library vendor, I think it will be a good idea to add a CI test (at a later time), to make sure that consuming with CPM actually works.

@mingxwa mingxwa merged commit 8ac85f6 into microsoft:main May 17, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Typo in the README
2 participants