diff --git a/3394_annotations/Makefile b/3394_annotations/Makefile new file mode 100644 index 0000000..cc60407 --- /dev/null +++ b/3394_annotations/Makefile @@ -0,0 +1,2 @@ +p3394r0.html : annotations.md +include ../md/mpark-wg21.mk diff --git a/3394_annotations/annotations.md b/3394_annotations/annotations.md index c105a2f..aa29fae 100644 --- a/3394_annotations/annotations.md +++ b/3394_annotations/annotations.md @@ -1,25 +1,24 @@ --- title: Annotations for Reflection tag: constexpr +document: P3394R0 +date: today +audience: EWG +hackmd: true +author: + - name: Wyatt Childers + email: + - name: Dan Katz + email: + - name: Barry Revzin + email: + - name: Daveed Vandevoorde + email: --- -Annotations for Reflection -=== +# Introduction -**Document number**: P3394R0 [\[Latest\]](https://wg21.link/p3394) [\[Status\]](https://wg21.link/p3394/status) -**Date**: 2024-09-14 -**Authors**: - - Wyatt Childers (wcc@edg.com) - - Dan Katz (dkatz85@bloomberg.net) - - Barry Revzin (barry.revzin@gmail.com) - - Daveed Vandevoorde (daveed@edg.com) -**Audience**: Evolution - - - -## Introduction - -Ever since writing [P1240R2](https://wg21.link/p1240r2) ("Scalable Reflection in C++")], but more so since [P2996R5](https://wg21.link/p2996r5) ("Reflection for C++26"), we have been requested to add a capability to annotate declarations in a way that reflection can observe. For example, Jeremy Ong presented compelling arguments in a post to the SG7 reflector: https://lists.isocpp.org/sg7/2023/10/0450.php. Corentin Jabot also noticed the need while P1240 was evolving and wrote [P1887R0](https://wg21.link/p1887) ("Reflection on Attributes"), which proposes syntax not entirely unlike what we present here. +Ever since writing [@P1240R0] ("Scalable Reflection in C++"), but more so since [@P2996R0] ("Reflection for C++26"), we have been requested to add a capability to annotate declarations in a way that reflection can observe. For example, Jeremy Ong presented compelling arguments in a post to the SG7 reflector: https://lists.isocpp.org/sg7/2023/10/0450.php. Corentin Jabot also noticed the need while P1240 was evolving and wrote [@P1887R0] ("Reflection on Attributes"), which proposes syntax not entirely unlike what we present here. In early versions of P2996 (and P1240 before that), a workaround was to encode properties in the template arguments of alias template specializations: @@ -40,19 +39,19 @@ There are problems with this approach, unfortunately: * It doesn't work for all contexts where we might want to annotate declarations (e.g., enumerators). * It doesn't directly express the intent. -* It turns out that providing access to aliases used in the declaration of reflected entities raises difficult questions and P2996 is therefore likely going to drop the ability to access that information (with the intention of resurrecting the capability with a later paqper once specification and implementation challenges have been addressed). +* It turns out that providing access to aliases used in the declaration of reflected entities raises difficult questions and P2996 is therefore likely going to drop the ability to access that information (with the intention of resurrecting the capability with a later paper once specification and implementation challenges have been addressed). In this paper, we propose simple mechanisms that more directly support the ability to annotate C++ constructs. -## Motivating Examples +# Motivating Examples We'll start with a few motivating examples for the feature. We'll describe the details of the feature in the subsequent section. These examples are inspired from libraries in other programming languages that provide some mechanism to annotate declarations. -### Command-Line Argument Parsing +## Command-Line Argument Parsing -Rust's [clap](https://docs.rs/clap/latest/clap/) library provides a way to add annotations to declarations to help drive how the parser is declared. We can now [do the same](https://godbolt.org/z/GsYKWjdeh): +Rust's [clap](https://docs.rs/clap/latest/clap/) library provides a way to add annotations to declarations to help drive how the parser is declared. We can now [do the same](https://godbolt.org/z/dM3erErrz): ```cpp struct Args { @@ -152,7 +151,7 @@ namespace clap { Overall, a fairly concise implementation for an extremely user-friendly approach to command-line argument parsing. -### Test Parametrization +## Test Parametrization The pytest framework comes with a decorator to [parametrize](https://docs.pytest.org/en/7.1.x/how-to/parametrize.html) test functions. We can now do [the same](https://godbolt.org/z/vnT51585G): @@ -258,9 +257,9 @@ void invoke_all() { } ``` -### Serialization +## Serialization -Rust's [serde](https://serde.rs/) library is a framework for serialization and deserialization. It is easy enough with reflection to do member-wise serialization. But how do you opt into that? An annotation provides a cheap mechanism of [doing just that](https://godbolt.org/z/51qjha8he) (built on top of [Boost.Json](https://www.boost.org/doc/libs/1_85_0/libs/json/doc/html/index.html)): +Rust's [serde](https://serde.rs/) library is a framework for serialization and deserialization. It is easy enough with reflection to do member-wise serialization. But how do you opt into that? An annotation provides a cheap mechanism of [doing just that](https://godbolt.org/z/cjP8EdTj3) (built on top of [Boost.Json](https://www.boost.org/doc/libs/1_85_0/libs/json/doc/html/index.html)): ```cpp struct [[=serde::derive]] Point { @@ -279,15 +278,15 @@ But opting in is just the first thing you might want to do with serialization. Y ```cpp struct [[=serde::derive]] Person { - [[=serde::rename("firstName")]] std::string first; - [[=serde::rename("lastName")]] std::string last; + [[=serde::rename("first name")]] std::string first; + [[=serde::rename("last name")]] std::string last; }; ``` Which leads to: ```cpp -// prints {"firstName":"Peter","lastName":"Dimov"} +// prints {"first name":"Peter","last name":"Dimov"} std::cout << boost::json::value_from(Person{.first="Peter", .last="Dimov"}); ``` @@ -306,8 +305,8 @@ namespace boost::json { void tag_invoke(value_from_tag const&, value& v, T const& t) { auto& obj = v.emplace_object(); template for (auto M : nonstatic_data_members_of(^^T)) { - constexpr auto field = annotation_of(M) - .transform([](serde::Rename r){ + constexpr auto field = annotation_of(M) + .transform([](serde::rename r){ return std::string_view(r.field); }) .value_or(identifier_of(M)); @@ -318,9 +317,9 @@ namespace boost::json { } ``` -You can imagine extending this out to support a wide variety of other serialization-specific attributes that shouldn't otherwise affect the C++ usage of the type. +You can imagine extending this out to support a wide variety of other serialization-specific attributes that shouldn't otherwise affect the C++ usage of the type. For instance, a [more complex approach](https://godbolt.org/z/jaKTe57Gf) additionally supports the `skip_serializing_if` annotation while first collecting all `serde` annotations into a struct. -## Proposal +# Proposal The core idea is that an *annotation* is a compile-time value that can be associated with a construct to which attributes can appertain. Annotation and attributes are somewhat related ideas, and we therefore propose a syntax for the former that builds on the existing syntax for the latter. @@ -336,7 +335,7 @@ Syntactically, an annotation is an attribute of the form `= expr` where `expr` i Currently, we require that an annotation has structural type because we're going to return annotations through `std::meta::info`, and currently all reflection values must be structural. -### Why not Attributes? +## Why not Attributes? Attributes are very close in spirit to annotations. So it made sense to piggy-back on the attribute syntax to add annotations. Existing attributes are designed with fairly open grammar and they can be ignored by implementations, which makes it difficult to connect them to user code. Given a declarations like: @@ -354,7 +353,7 @@ Originally, the plus sign (`+`) was considered (as in P1887), but it is not idea As such, this paper proposes annotations as distinct from attributes, introduced with a prefix `=`. -### Library Queries +## Library Queries We propose the following set of library functions to work with annotations: @@ -396,7 +395,7 @@ Of these, four can be directly implemented in terms of the unary `annotations_of And lastly, `annotate` provides the ability to programmatically add an annotation to a declaration. -### Additional Syntactic Constraints +## Additional Syntactic Constraints Annotations can be repeated: @@ -436,6 +435,56 @@ Instead, use: int select_footgun(int); ``` -### Implementation Experience +## Implementation Experience + +The core language feature and the basic query functions have been implemented in the EDG front end and in Bloomberg's P2996 Clang fork (with option `-freflection-latest`), both available on Compiler Explorer. + +## Other Directions To Go + +As evidenced in the motivating examples earlier, there is a lot of value in this proposal even in this simple form. However, there is more to consider when it comes to annotations. + +This proposal right now lets us unconditionally add an annotation to a type: + +```cpp +struct [[=X]] Always; +``` + +But it does not let us conditionally add an annotation to a type: + +```cpp +template +struct /* X only for some T */ Sometimes; +``` + +Or to really generalize annotations. For instance, in the clap example earlier, our example showed usage with `clap::Short` and `clap::Long`. What if somebody wants to compose these into their own annotation that attaches both `clap::Short` and `clap::Long` to a declaration? + +More broadly, there is clear value in having an annotation be able to be invoked by the declaration itself. Doing so allows the two uses above easily enough. The interesting question, though, is whether this callback (syntax to be determined) is invoked at the _beginning_ of the declaration or at the _end_ of the declaration. For annotations on classes, this would be before the class is complete or after the class is complete. Before completeness allows the class to observe the annotation during instantiation. After completeness allows the annotation callback to observe properties of the type. In some sense, Herb Sutter's [@P0707R4] ("Metaclass functions: Generative C++") was adding annotations on classes, invoked on class completeness, that allow mutation of the class. + +One concrete, simpler example. We can, with this proposal as-is, create a `Debug` annotation that a user can add to their type and a specialization of `std::formatter` for all types that have a `Debug` annotation [as follows](https://godbolt.org/z/bcYE7nY4s): + +```cpp +template struct Derive { }; +template inline constexpr Derive derive; + +inline constexpr struct{} Debug; + +template requires (has_annotation(^^T, derive)) +struct std::formatter { + // ... +}; + +struct [[=derive]] Point { + int x; + int y; +}; + +int main() { + auto p = Point{.x=1, .y=2}; + // prints p=Point{.x=1, .y=2} + std::println("p={}", p); +} +``` + +This *works*, but it's not really the ideal way of doing it. This could still run into potential issues with ambiguous specialization of `std::formatter`. Better would be to allow the `Debug` annotation to, at the point of completion of `Point`, inject an explicit specialization of `std::formatter`. This would rely both on the ability for the annotation to be called back and language support for such injection (see [@P3294R1] ("Code Injection with Token Sequences")). -The core language feature and the basic query functions have been implemented in the EDG front end and in Bloomberg's P2996 Clang fork (with option `-freflection-latest`), both available on Compiler Explorer. \ No newline at end of file +There are still open questions as to how to handle such callbacks. Does an annotation that gets called back merit different syntax from an annotation that doesn't? Can it mutate the entity that it is attached to? How do we name the potential callbacks? \ No newline at end of file diff --git a/3394_annotations/p3394r0.html b/3394_annotations/p3394r0.html index 2f68a45..93da231 100644 --- a/3394_annotations/p3394r0.html +++ b/3394_annotations/p3394r0.html @@ -1,493 +1,1275 @@ - - - + - - - - - - - - Annotations for Reflection - HackMD - - - - - - - - - - - - - - - - - - -

Annotations for Reflection

Document number: P3394R0 [Latest] [Status]
-Date: 2024-09-14
-Authors:
-- Wyatt Childers (wcc@edg.com)
-- Dan Katz (dkatz85@bloomberg.net)
-- Barry Revzin (barry.revzin@gmail.com)
-- Daveed Vandevoorde (daveed@edg.com)
-Audience: Evolution

Introduction

Ever since writing P1240R2 ("Scalable Reflection in C++")], but more so since P2996R5 ("Reflection for C++26"), we have been requested to add a capability to annotate declarations in a way that reflection can observe. For example, Jeremy Ong presented compelling arguments in a post to the SG7 reflector: https://lists.isocpp.org/sg7/2023/10/0450.php. Corentin Jabot also noticed the need while P1240 was evolving and wrote P1887R0 ("Reflection on Attributes"), which proposes syntax not entirely unlike what we present here.

In early versions of P2996 (and P1240 before that), a workaround was to encode properties in the template arguments of alias template specializations:

template <typename T, auto... Annotations>
-using Noted = T;
-
-struct C {
-    Noted<int, 1> a;
-    Noted<int*, some, thing> b;
-};
-

It was expected that something like type_of(^C::a) would produce a reflection of Noted<int, 1> and that can be taken apart with metafunctions like template_arguments_of — which both preserves the type as desired (a is still an int) and allows reflection queries to get at the desired annotations (the 1, some, and thing in this case).

There are problems with this approach, unfortunately:

    -
  • It doesn't work for all contexts where we might want to annotate declarations (e.g., enumerators).
    • -
  • It doesn't directly express the intent.
    • -
  • It turns out that providing access to aliases used in the declaration of reflected entities raises difficult questions and P2996 is therefore likely going to drop the ability to access that information (with the intention of resurrecting the capability with a later paper once specification and implementation challenges have been addressed).
    • -

In this paper, we propose simple mechanisms that more directly support the ability to annotate C++ constructs.

Motivating Examples

We'll start with a few motivating examples for the feature. We'll describe the details of the feature in the subsequent section.

These examples are inspired from libraries in other programming languages that provide some mechanism to annotate declarations.

Command-Line Argument Parsing

Rust's clap library provides a way to add annotations to declarations to help drive how the parser is declared. We can now do the same:

struct Args {
-    [[=clap::Help("Name of the person to greet")]]
-    [[=clap::Short, =clap::Long]]
-    std::string name;
-
-    [[=clap::Help("Number of times to greet")]]
-    [[=clap::Short, =clap::Long]]
-    int count = 1;
-};
-
-
-int main(int argc, char** argv) {
-    Args args = clap::parse<Args>(argc, argv);
-
-    for (int i = 0; i < args.count; ++i) {
-        std::cout << "Hello " << args.name << '\n';
-    }
-}
-

Here, we provide three types (Short, Long, and Help) which help define how these member variables are intended to be used on the command-line. This is implemented on top of Lyra.

When run:

$ demo -h
-USAGE:
-  demo [-?|-h|--help] [-n|--name <name>] [-c|--count <count>]
-
-Display usage information.
-
-OPTIONS, ARGUMENTS:
-  -?, -h, --help
-  -n, --name <name>       Name of the person to greet
-  -c, --count <count>     Number of times to greet
-
-$ demo -n wg21 --count 3
-Hello wg21
-Hello wg21
-Hello wg21
-

While Short and Long can take explicit values, by default they use the first letter and whole name of the member that they annotate.

The core of the implementation is that parse<Args> loops over all the non-static data members of Args, then finds all the clap-related annotations and invokes them:

template <typename Args>
-auto parse(int argc, char** argv) -> Args {
-    Args args;
-    auto cli = lyra::cli();
-
-    // ...
-
-    template for (constexpr info M : nonstatic_data_members_of(^^Args)) {
-        auto id = std::string id(identifier_of(mem));
-        auto opt = lyra::opt(args.[:M:], id);
-
-        template for (constexpr info A : annotations_of(M)) {
-            if constexpr (parent_of(type_of(A)) == ^^clap) {
-                // for those annotions that are in the clap namespace
-                // invoke them on our option
-                extract<[:type_of(A):]>(A).apply_annotation(opt, id);
-            }
-        }
-
-        cli.add_argument(opt);
-    }
-
-    // ...
-};
-

So, for instance, Short would be implemented like this:

namespace clap {
-    struct ShortArg {
-        // optional isn't structural yet but let's pretend
-        optional<char> value;
-
-        constexpr auto operator()(char c) const -> ShortArg {
-            return {.value=c};
-        };
-
-        auto apply_annotation(lyra::opt& opt, std::string_view id) const -> void {
-            char first = value.value_or(id[0]);
-            opt[std::string("-") + first];
-        }
-    };
-
-    inline constexpr auto Short = ShortArg();
-}
-

Overall, a fairly concise implementation for an extremely user-friendly approach to command-line argument parsing.

Test Parametrization

The pytest framework comes with a decorator to parametrize test functions. We can now do the same:

namespace N {
-    [[=parametrize({
-        Tuple{1, 1, 2},
-        Tuple{1, 2, 3}
-        })]]
-    void test_sum(int x, int y, int z) {
-        std::println("Called test_sum(x={}, y={}, z={})", x, y, z);
-    }
-
-    struct Fixture {
-        Fixture() {
-            std::println("setup fixture");
-        }
-
-        ~Fixture() {
-            std::println("teardown fixture");
-        }
-
-        [[=parametrize({Tuple{1}, Tuple{2}})]]
-        void test_one(int x) {
-            std::println("test one({})", x);
-        }
-
-        void test_two() {
-            std::println("test two");
-        }
-    };
-}
-
-int main() {
-    invoke_all<^^N>();
-}
-

When run, this prints:

Called test_sum(x=1, y=1, z=2)
-Called test_sum(x=1, y=2, z=3)
-setup fixture
-test one(1)
-teardown fixture
-setup fixture
-test one(2)
-teardown fixture
-setup fixture
-test two
-teardown fixture
-

Here, parametrize returns a value that is some specialization of Parametrize (which is basically an array of tuples, except that std::tuple isn't structural so the implementation rolls its own).

The rest of the implementation looks for all the free functions named test_* or nonstatic member functions of class types that start with test_* and invokes them once, or with each parameter, depending on the presence of the annotation. That looks like this:

consteval auto parametrization_of(std::meta::info M) -> std::meta::info {
-    for (auto a : annotations_of(M)) {
-        auto t = type_of(a);
-        if (has_template_arguments(t) and template_of(t) == ^^Parametrize) {
-            return a;
-        }
-    }
-    return std::meta::info();
-}
-
-template <std::meta::info M, class F>
-void invoke_single_test(F f) {
-    constexpr auto A = parametrization_of(M);
-
-    if constexpr (A != std::meta::info()) {
-        // if we are parametrized, pull out that value
-        // and for each tuple, invoke the function
-        // this is basically calling std::apply on an array of tuples
-        constexpr auto Params = extract<[:type_of(A):]>(A);
-        for (auto P : Params) {
-            P.apply(f);
-        }
-    } else {
-        f();
-    }
-}
+  
+  
+  
+  
+  Annotations for Reflection
+  
+  
+  
+  
+  

+  

 
-  consteval info annotate(info item,
-                          info value,
-                          source_location loc = source_location::current());
-}
-

is_annotation checks whether a particular reflection represents an annotation.

We provide two overloads of annotations_of to retrieve all the annotations on a particular item:

    -
  1. Returns all the annotations.
    2. -
  2. Returns all the annotations a such that type_of(a) == type.
    3. -

And a singular version, annotation_of<T>, that returns the annotation a such that dealias(type_of(a)) == ^^T. If no such annotation exists, returns nullopt. If more than one such annotation exists and all the values are not template-argument-equivalent, this call is not a constant expression.

And then two overloads of has_annotation that simply checks if a given annotation exists:

    -
  1. Checks if there exists an annotation a such that dealias(type_of(a)) == ^^T.
    2. -
  2. Checks if there exists an annotation a such that value_of(a) == reflect_value(value).
    3. -

Of these, four can be directly implemented in terms of the unary annotations_of(item), but we think they'll be common enough to merit inclusion.

And lastly, annotate provides the ability to programmatically add an annotation to a declaration.

Additional Syntactic Constraints

Annotations can be repeated:

[[=42, =42]] int x;
-static_assert(annotations_of(^x).size() == 2);
-

Annotations spread over multiple declarations of the same entity accumulate:

[[=42]] int f();
-[[=24]] int f();
-static_assert(annotations_of(^f).size() == 2);
-

Annotations follow appertainance rules like attributes, but shall not appear in the attribute-specifier-seq of a type-specifier-seq or an empty-declaration:

struct [[=0]] S {};  // Okay: Appertains to S.
-[[=42]] int f();     // Okay: Appertains to f().
-int f[[=0]] f();     // Ditto.
-int [[=24]] f();     // Error: Cannot appertain to int.
-[[=123]];            // Error: No applicable construct.
-

To avoid confusion, annotations are not permitted after an attribute-using-prefix. So this is an error:

[[using clang: amdgpu_waves_per_eu, =nick("weapon")]]
-int select_footgun(int);
-

Instead, use:

[[using clang: amdgpu_waves_per_eu]] [[=nick("weapon")]]
-int select_footgun(int);
-

Implementation Experience

The core language feature and the basic query functions have been implemented in the EDG front end and in Bloomberg's P2996 Clang fork (with option -freflection-latest), both available on Compiler Explorer.

- - + - diff --git a/md/pandoc.py b/md/pandoc.py index 6794d9a..1425ceb 100755 --- a/md/pandoc.py +++ b/md/pandoc.py @@ -97,7 +97,15 @@ def op(elem, doc): if isinstance(elem, pf.Code) and 'op' in elem.classes: return pf.RawInline(f'{elem.text}') +def std(elem, doc): + if not doc.get_metadata('hackmd', False): + return + + if isinstance(elem, pf.RawBlock) and not isinstance(elem.parent, pf.BlockQuote): + return pf.Div(pf.BlockQuote(elem), classes=["std"]) + + if __name__ == '__main__': # pf.run_filters([h1hr, bq, graphviz, mermaid, op]) - pf.run_filters([h1hr, bq, mermaid, op]) + pf.run_filters([h1hr, bq, std, mermaid, op]) # pf.run_filters([h1hr, bq, op])