Add @standalone attribute for module constructors

[dkorpel]

No description provided.

[dlang-bot]

Thanks for your pull request and interest in making D better, @dkorpel! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

• My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
• My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
• I have provided a detailed rationale explaining my changes
• New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

---

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#15537"

[maxhaton]

Might be cleaner to return a pointer, the other is functions in DMD do this so its a common pattern to keep going.

[maxhaton]

Ugly naming convention. Unfortunate.

[maxhaton]

Seems worth a try out.

[ghost]

why not @standalone directly ?

[dkorpel]

So the name can't be out of sync with the attribute I guess. Really, I just mimicked how @mustuse did it

[maxhaton]

There will be more of these in future, can it be made into a reusable function?

[ghost]

to be frank, this is not a hot path. It's just that it's useless to add N kilobytes of code while the static data is already there.

[dkorpel]

can it be made into a reusable function

What part of the error would other attributes re-use?

[maxhaton]

That's up to your judgement - "I just mimicked" hinted at some repetition. If there is no overlap then don't, but it would be really good to make these types of logic declarative. I'll write a sketch hang on.

[dkorpel]

The 'repetition' is also using @%s and Id.udaStandalone.toChars() instead of @standalone, that's too small of a thing to extract to a function.

[ghost]

ditto

[WalterBright]

The @standalone attribute is only useful for static constructors. It is meaningless when applied to anything else. Shouldn't an attribute be something that applies to lots of things, instead of exactly one thing? Some other syntax should trigger this. Maybe in the spirit of pragma(crt_constructor) we could have pragma(acyclic_constructor) ?

[WalterBright]

P.S. the number of pre-defined attributes is getting a little large, and we're running out of space in the integer mask for them!

[dkorpel]

Shouldn't an attribute be something that applies to lots of things, instead of exactly one thing?

I don't see why that's a rule. override only applies to class methods, @selector only applies to Objective-C methods.
Internally, a pragma is an attribute (it extends AttribDeclaration). The differences that matter are:

• pragma's can be put inside the function body (PragmaStatement)
• unknown pragma's are an error unless compiling with -ignore, which few people do
• in contrast, core.attribute attributes can be made compatible with older compiler versions using static if (__VERSION__ < 2106) enum standalone; else import core.attribute : standalone;.

[dkorpel]

P.S. the number of pre-defined attributes is getting a little large, and we're running out of space in the integer mask for them!

Then you'd be happy to know that none of the attributes in core.attribute add a new bit mask :)

[ljmf00]

This is a very much appreciated feature to have. Weka struggles a lot with these, and we run our unittests with oncycle=print, which is not desirable. Most of it is fine, because they are, in fact, independent. This could also make the compiler avoid enforcing to reference the ModuleInfo, when its not needed.

The hacky solution "just create another module" is not a feasible thing for large codebases.

[maxhaton]

Based on our codebases I think I'd prefer an attribute than a pragma.

[WalterBright]

Pragma is already used for pragma(crt_constructor). Doesn't it seem weird to use a pragma for that unusual constructor, and an attribute for another unusual constructor? pragma(crt_constructor) is a pretty good signal that it is for something that is a bit outside of the language, which acyclic constructors certainly are.

[ljmf00]

which acyclic constructors certainly are.

It shouldn't be. It's just defining how it should behave. We should have this since the beginning

[WalterBright]

@ljmf00 The acyclic static constructor creates a correctness hole in the language. It should stand out with a different syntax, one like the crt_constructor which also creates a correctness hole. Using an attribute implies it is more central to the language.

[adamdruppe]

Attributes are the correct solution here. Please don't force it to be a pragma, that'd make it much harder to use in stable codebases.

[MrcSnm]

Hello, this is a nice feature and all, but there is a fundamental problem:
This could and should be done without any programmer intervention, i.e: no attribute or pragmas.
Think about people trying to learn D, and then they didn't read the entire spec (99% chance of that), they stumble on that error, can't fix it unless they ask (most people just go away, they don't even ask).

D should be all about making it work more by writing less, it is a well suited language for a beginner to learn and putting this attribute instead of really solving the problem and making it just works is just a step backwards.

If you take a look into the forums, people already calls that as attribute soup, and creating a brand new attribute is the wrong direction, Razvan even recently announced the unused import removal tool, which should be similar to what this @standalone attribute does, but automatically.

IMO, this is just more unnecessary friction in the language, more cluttering in symbols (yes, @standalone is not that uncommon word anyway).

[adamdruppe]

It'd be great if it was done automatically but that's not realistically going to happen. D is supposed to be a practical, real world language. This PR serves a practical, real world use case.

If the implementation were to ever automatically infer this, the attribute doesn't harm anything in any way while solving a real problem. Merge ASAP.

[dkorpel]

Using an attribute implies it is more central to the language.

It's a library attribute though (druntime/core/attribute), not a reserved language attribute. I could make it uglier by adding underscores, but historically, I don't think that refrained people from using __traits or __gshared.

[dkorpel]

D should be all about making it work more by writing less

I agree, and we considered this in the monthly meeting, but the complexity and fragility of inference of @standalone was not deemed worth it. Any better solution ideas are welcome though!

[dkorpel]

Merge ASAP

Debugging the failure will have to wait until after DConf. In the meantime, the bike shedding about the syntax may continue! 🙂

[ljmf00]

@ljmf00 The acyclic static constructor creates a correctness hole in the language. It should stand out with a different syntax, one like the crt_constructor which also creates a correctness hole. Using an attribute implies it is more central to the language.

Sure but its the same argument as the @trusted attribute. It also open holes, but sometimes is needed, if we dont want to degrade performance in some specific cases, as we do assumptions.

[adamdruppe]

This is still an important need for D, it slipped my mind for a month but I hit yet another case where it would be useful. What's the current situation?

[dkorpel]

I'll look into the test failure this week

[adamdruppe]

ping again

[adamdruppe]

importing the module is the correct behavior.

[RazvanN7]

This needs formal approval from @WalterBright .

[WalterBright]

@ryuukk 's comment about the syntax #15537 (comment) is persuasive. What is the compelling advantage to using the attribute, with its rather clumsy looking syntax and double underscore?

One of the objections to pragma's was resolved by @adamdruppe 's PR #15893

Doesn't:

pragma(standalone)

just look nice?

[dkorpel]

Whaaaa? All this time you've been arguing that @standalone looks too nice for an 'ugly wart' in the language, and therefore it should become a pragma. I thought renaming it to @__standalone would be a good compromise, but now it looks clumsy and you want it to look nice? I can rename it back.

[adamdruppe]

I support multiple years of D compilers in the ecosystem. This means I can only use a new feature if either 1) it can be tested and enabled selectively in the code itself or 2) has been in the language for over 3 years, such that the oldest supported compiler has it.

the pragma objection stands until 2027. It is good it is going to be fixed some day, but I can't go breaking my client's code until then, so if this is a pragma, it is useless to me for years, whereas the attribute can be committed to library code right now, heck, before the feature is even merged.

[MrcSnm]

@WalterBright The main reason why pragma is not being considered is the difficulty of compiler feature identification.
That means, people prefer it as an attribute since it can be checked on compilation time. For example, the inexistence of a featured condition such as static if(compilerHasStandalone) makes it harder.

The pragma problem is that one would need to static if the compiler versions for trying to detect that feature (and still be erratic, since, for example, extern(Objective-C) is supported in DMD from version X, while it is not even implemented on LDC and GDC)

Even if this was done right now, it would take some years for it being considered in various codebases since most don't want to keep updating their compilers all the time.

[WalterBright]

I have a mtg to go to, so will address all points later.

[WalterBright]

@dkorpel :

@__standalone is just ugly. Can I change my mind about that? Even worse, though, is the required import:

pragma(crt_constructor) void foo();
pragma(crt_destructor) void bar();

import core.attribute : __standalone;
@__standalone void xxx();

The more I look at it, the worse the inconsistency and aesthetics of xxx() look.

Another peculiar inconsistency of this is it being the only compiler recognized attribute that is not part of the grammar. I am hesitant to embark down that road.

@adamdruppe :

I didn't know you were committed to supporting 3 years worth of compilers, thanks for telling me. Even though your initiative to ignore unsupported pragmas has been merged, I understand it won't work with older compilers. But I do have a solution! Consider modules A and B which import each other. We want an acyclic constructor in A. This can be done with:

module C;
extern (C) void acyclic_constructor();
static this() { acyclic_constructor(); }

module A;
import C;
import B;
extern (C) void acyclic_constructor() { ... }

module B;
import A;
...

This should work on all versions of D.

@MrcSnm :

There is currently no __traits support for detecting things like constructors, pragma(crt_destructor), etc. If there is a need to detect such pragmas, another __trait could be added for the set of them.

[adamdruppe]

You can't do those extern(C) import tricks in mixin templates, which is the primary use case I have for this, because you don't control which module the functions end up in. They exist in random user modules, not something I write, and it is impossible to split a single mixed in declaration across two modules.

[adamdruppe]

There is currently no __traits support for detecting things like constructors, pragma(crt_destructor), etc. If there is a need to detect such pragmas, another __trait could be added for the set of them.

One of the beauties of using attributes is you don't need more special cases. The existing language supports all this stuff for them already - reflection, grouping, name conflict resolution, and more - the only magic thing about them is the compiler recognizes them. It is extremely beneficial to use this instead of adding more and more special cases for every little thing that comes up.

Another peculiar inconsistency of this is it being the only compiler recognized attribute that is not part of the grammar.

ldc and gdc have tons of compiler recognized attributes. Why would being part of the grammar be beneficial at all anyway?

I also don't think mustUse is in the grammar and even dmd supports that one.

[pbackus]

Another peculiar inconsistency of this is it being the only compiler recognized attribute that is not part of the grammar.

@mustuse is not part of the grammar.

[WalterBright]

I'll think about the template mixin case.

mustuse is indeed in dmd, though I wasn't involved with it.
https://dlang.org/spec/attribute.html#mustuse-attribute

If the __standalone attribute is ignored in older compilers, how do your mixin templates work around that?

[dkorpel]

though I wasn't involved with it.

You reviewed and accepted the DIP. Either way, core.attribute exists, is used by dmd, and works fine. There is no dangerous road to embark on.

[dkorpel]

The more I look at it, the worse the inconsistency and aesthetics of xxx() look.

I'll remove the underscores. The import is not ergonomic, but remember that this is a once-in-100KLOC feature, not something the user is supposed to reach to quickly (by your own account). As for consistency, @standalone is an attribute of static constructors, not regular functions like pragma(crt_constructor), so a more apt comparison is this:

import core.attribute : standalone;

shared static this()
{
    // normal sctor
}

@standalone @system 
shared static this()
{
    // standalone sctor
}

Which looks fine to me.

[adamdruppe]

If the __standalone attribute is ignored in older compilers, how do your mixin templates work around that?

This is the status quo, so there's no regression. But the situation is iffy:

1. In some cases, they simply don't notice the buggy behavior because the mixins I provide tend to be used in top-level modules that aren't mutually imported anyway. This works decently well for applications, but not really for libraries.

2. In other cases, the module cycle detection in druntime must be turned off. e.g. for using my android bindings library: https://github.com/adamdruppe/d_android/blob/master/source/hacks.d#L8 (note that with the attribute, I can static if(__traits(hasMember, core.attribute, "standalone")) /* no hack */ else { /* keep old hack */ } to transparently improve the situation for all users - reflecting on availability of features lets libraries do things like this! BTW static if(__VERSION__ > 2010 is a possible substitute for this, but __VERSION__ does not correctly cover uses where a feature is backported; it is often inaccurate for gdc's LTS users, so a direct detection of the feature with __traits(hasMember) - possible only with attributes, not with pragmas - is preferable for them to benefit too)

This is a global switch that is not suitable for libraries in general - it can have unpredictable side effects when combined with other code - but is the only option at this time to move forward with the use case, so we use it and hope for the best.

Having @standalone means these uses can finally be expanded with confidence in new compilers, while users of old compilers don't get the enhancement of expanded use, but also don't get the regression of their existing use being broken; they're no better AND no worse than they are now.

[WalterBright]

Ok, @dkorpel and @adamdruppe make a good case, thank you.

[adamdruppe]

Splendid, thank you, this is gonna solve a long annoyance.

[schveiguy]

Is there a reason why @standalone cannot be applied to thread-local static constructors? All it is doing is removing it from cycle detection, and there are still valid uses in the case of thread-local constructors.

[adamdruppe]

I just don't think there's an existing cycle-independent list for thread local ones, whereas there was for global ones so it just a factor of convenience.

[schveiguy]

No, both cycles are detected independently.

dmd/druntime/src/rt/minfo.d

Lines 517 to 518 in 03f1497

	if (!doSort(MIctor | MIdtor, _ctors) ||
	!doSort(MItlsctor | MItlsdtor, _tlsctors))

[adamdruppe]

Yes, but m.ictor already exists (the cycle-independent analog to m.ctor) but there is no m.itlsctor to pair with m.tlsctor (see runCtors and runTlsCtors a few lines below the function you linked).

So before (or simultaneously with) standalone can be added for tls ctors, some kind of itlsctor implementation also needs to be added.

[schveiguy]

Huh, I never knew about the ictor thing. What I am talking about is the MIstandalone flag (which is what I thought designated standalone modules).

See

dmd/druntime/src/rt/minfo.d

Line 478 in 03f1497

if (m.flags & MIstandalone)

Now I have no idea exactly what this PR is doing...

[schveiguy]

https://issues.dlang.org/show_bug.cgi?id=24425 Please comment if I'm off base here.

[adamdruppe]

I think that would work, indeed, that was one of my first ideas to implement this but when i looked at the code, ictor was already there so the plan shifted to just use that existing stuff. But yeah it is limited to shared static this so expanding to the others would need something new and putting them in a dummy module with the standalone flag indeed ought to work.