Create Static Enough Metaprogramming proposal

[mraleph]

This moves content from #4271 into a markdown file in the repository to make discussion and revisions easier.

I have incorporated some of the feedback from discussions on the issue - but I continue to maintain focus on this as a toolchain feature. I have added some remarks that analyzer can't constant fold everything anyway because it does not have access to the compilation environment.

I would like to collect a few rounds of feedback and then rejuvenate the prototype implementation to get something experimental working across all platforms in the SDK so that we can have an idea of how well this could work in a real world.

[eernstg]

LGTM. (Very interesting indeed!)

I could suggest the addition of 'a' or 'the' in many locations, but decided that this wouldn't be the top priority at this time.

[jakemac53]

This definitely does drastically simplify the proposal - but it also makes it a lot less useful. Ultimately trying to accomplish this was the downfall of the original macros proposal though.

But, I think it is important to look at the original motivations for macros, and which of those use cases are covered by this.

Ultimately, I think you can end up with some fairly decent solutions for automatic encoding/decoding, equality, toString, etc. But not copyWith or constructors due to the signatures being dependent on the shape of the class (you could generate the bodies of these but that's only half the boilerplate).

Edit: I see you have an interesting copyWith idea using records, that is fairly reasonable actually.

[mraleph]

I have added a section trying to cover some of the use cases mentioned peaked from an old spreadsheet I managed to find.

I could not exactly figure out what some of those entail, could you help me a bit (look for TODO: unclear what this means. and if you have context please add it in the comment). I am interested about auto listenable and render accessors. I think union types (ala freezed) means more concise way of declaring sealed class hierarchies - which I think should just be its own feature (paired with primary constructors).

I think a number of use cases require property wrappers and/or ability to redirect methods - this should probably be its own feature. If property wrappers are available then @konst can be used to handle the boilerplate.

Finally some cases (e.g. proxies, functional widgets and reduced widget classes boilerplate) require ability to create class declarations. I think it is actually an okay feature to give via @konst reflection. As long as classes are anonymous and are otherwise not visible.

[jakemac53]

What is the baseline?

[mraleph]

I don't have numbers right now but I can try to revive the prototype and update this later.

[jakemac53]

This would be 🔥 (and help a lot with the reduction of boilerplate)

[mraleph]

Yeah, it would be useful in many places. Also it would solve a performance problem we occasionally encounter where inheritance or generics cause polymorphism and severely reduce performance.

[mraleph]

I have made some updates based on comments. PTAL.

[munificent]

I still have the same reservations as before, but landing this in the repo so we can discuss more sounds great. It's a very solid well-thought out proposal and the background information it provides is really helpful for any metaprogramming-related features we might do.

[osa1]

I'm confused about how @konst applies to statements and functions. Given this:

When @konst is applied to loop iteration variables it instructs the compiler to expand the loop at compile time by first computing the sequence of values for that iteration variable, then cloning the body for each value in order and substituting iteration variable with the corresponding constant.

(in particular, "expand the loop at compile time" suggests code generation based on constant values)

I would expect this function

void bar<@konst T>(@konst T v) {

}

to be monomorphised based on Ts passed. (and generate one more copy for non-const T)

Otherwise when I do things like this code in the JSON examples:

Map<String, Object?> toJson<@konst T>(T value) => {
    for (@konst final field in TypeInfo.of<T>().fields)
      if (!field.isStatic) field.name: field.getFrom(value),
  };

The loop cannot be unfold at compile time, because the unfolded loop needs to be different for each T.

But the text doesn't mention monomorphisation at all. Am I confused?

If this allows monomorphisation (and it has to, if I get it right), then that's a pretty big capability for the langauge that's also worth mentioning.

[osa1]

Is this a typo for "ahead-of-time-JIT"? (off -> of)

[osa1]

Reading this I was curious how the analyzer can be used for this. It would be helpful to have a link to a code generator written using analyzer.

[osa1]

I believe that this allows to considerably simplify the design and implementation of this feature

I think this is true for this language feature, but is it also true for the language and tooling in general? For example, AFAIU invoke would be Function.apply in runtime. This will require that some (maybe all?) of the dynamically typed programming features like Function.apply will have to stay, and they'll be an essential part of the language rather than old features we want to get rid of. (because they are slow and require metadata that adds to binary sizes)

[osa1]

Shouldn't the Object? here be TypeInfo<Object?> (or TypeInfo, TypeInfo<dynamic>)? Can the field types be anything else other than a TypeInfo?

[osa1]

I also find it confusing to see what function calls (or other syntax) is part of the generated code vs. compile-time evaluation. For example, if I change the toJson example from the proposal to this:

Map<String, Object?> toJson<@konst T>(T value) => {
    for (@konst final field in TypeInfo.of<T>().fields)
      if (!field.isStatic) processFieldName(field.name): processFieldValue(field.getFrom(value)),
  };

(processFieldName and processFieldValue are new)

Just by looking at this code (without seeing processFieldName and processFieldValue definitions) I can't tell whether these function calls are evaluated in compile time or runtime.

I wonder if we could make stages explicit. There are many languages that do this that we could look for inspiration. For example Common Lisp's backquote and comma, MetaOCaml's (.< expr >.) and escape (.~expr) make the stages explicit.