New "Acceptance Testing" section for RFCs?

[epage]

Some times it can be a challenge to get testers for an RFC implementation to get the feedback to know if its ready for stablization (see Mechanism for beta testing unstable features).

What if we had a new section "Acceptance Testing" where we queued up RFCs that are are at stage of needing general feedback, either as part of an intermediate design milestone (e.g. rust-lang/rfcs#3243) or in prep for stablization (e.g. rust-lang/cargo#8415 (comment))? They'd need to link out to a test plan (defining what you are looking, link to how to use the feature, link to how to use it with nightly, where to apply feedback, etc).

[jonhoo]

In the interest of continuing to leave a trail for things related to this effort, see also The case for a new release channel: testing, Survey of "High Impact" Features, RFC: Preview for Unstable Features, and ultimately this comment.

I love the idea of a "call for testing" in TWiR. My main concern continues to be the ability to get widespread testing when it requires nightly.

[U007D]

@epage Interesting thoughts. Would it help if the section were titled something more "call-to-action" oriented e.g. "Call for Testing"?
@jonhoo I read the links you provided (only most of the RFC: Preview for Unstable Features) + your linked comment. I have personal opinions on where we should land, but before I get to those, to address @epage's concern, it would be a shame to let perfect be the enemy of good.

To that end, a section with a call to action and clear instructions (or a link to clear instructions) that would evolve as the feature stabilization process does would allow us to move forward (if we elected to) quickly, while still preserving the option to evolve the stabilization process.

Definitely something to discuss with @nellshamrell and the rest of the TWiR editors, but if either of you have a draft of what you'd like to see the section look like please share it here, otherwise let us know. I'm happy bring up the topic for discussion with the other editors.

--

My own 2¢ on the topic is to start with Nightly, moving to having a mechanism to ungate the to-be-stablilized features individually and as a group. From there, move the capability to Beta, and we can learn about what issues there are with exercising the features using Beta vs feature-gated on Stable. At some point beyond this, if deemed appropriate, we could move the to-be-stabilized feature gates to stable (although I think the time-lag synchronizing with Stable could be a source of vocal objections).

Along with the technical implementation of the feature gates, key would be having clear instructions on how to exercise each of the to-be-stabilized features, as most people will be unfamiliar with the nuances of the feature and will not be inclined to read all the historical context to get up-to speed.

[epage]

Draft: comes before "Final Comment Period"

### Call For Testing

An important step for RFC implementation is for people to experiment with the implementation and give feedback, especially before stablization.  The following RFCs are at point where user testing is needed before moving forward:

Pre-Stablization
- [RFC 2906 Tracking Issue](https://github.com/rust-lang/cargo/issues/8415): Testing steps **LINKME**

[Muscraft]

Having a summary of the changes might be nice as well. I know for RFC 2906 we made changes from what the RFC called for and there is a summary because of that. The link to the tracking issue could be just a link to the most recent summary as well.

[nellshamrell]

Adding this section sounds good to me - and it's a great way to increase participation!

[Muscraft]

Unless there is more to discuss on how the future looks for this section I think #3249 closes this issue

[U007D]

Thanks everyone. The new section is in the template.

I am happy to fill out this section on an ongoing basis as a function of my FCP/RFC editor role--but if there is anyone on the TWiR editorial team who had their heart set on owning this section, I am flexible--just let me know!

I will leave this issue open a bit longer as we currently lack specific inclusion criteria for the line items to be included in this section. I will do a run-through for this week's TWiR-443 and will publish what I come up with here as a draft for feedback and adjustments. If what we come up with works for folks, I will then close this issue.

As always, thoughts/feedback/ideas welcome!

[U007D]

In working through this, I think the simplest way forward is to create a new label (e.g. call-for-testing) that authors and implementers can apply to their RFCs/tracking items, etc., as appropriate.

(For future consideration, I remain convinced that we will get better uptake when the line items come with specific testing instructions; this is something we can iterate on over time.)

I have added the two RFCs explicitly called out in this thread RFC 3243 and TI 8415 to this week's TWiR and will keep an eye out for feedback. @epage, please note I have omitted the "Pre-stabilization" and "Intermediate Design" categories at least initially, since there are only two items in this weeks TWiR. I am happy to discuss how and what we should include as we get a sense of volume and how this information will be used.

**Going forward, I propose building the section based on new results from this query after having added a call-for-testing label. The author would add any auxiliary data such as tracking issues, testing steps, etc. into the comments at that time and I'll pick those up for TWiR.

If you have suggestions on any of this please share, otherwise if this looks good I'll figure out how to create a new call-for-testing label and we will proceed from there.**

Edit: removed "is-open" criteria from proposed weekly query.

[Muscraft]

@U007D I don't know if you saw but I opened a PR for TI 8415 already in #3256

[U007D]

@Muscraft No, I hadn't seen that--thanks for the heads up. I will incorporate your data into my PR.

Update: Done.

[Muscraft]

**Going forward, I propose building the section based on new results from this query after having added a call-for-testing label. The author would add any auxiliary data such as tracking issues, testing steps, etc. into the comments at that time and I'll pick those up for TWiR.

That query would miss things like RFC 2906 as it is "closed". I think just looking for the label call-for-testing is a better idea. That way we do not worry about if it is open or closed, new or old. This also has the benefit that each RFC that is looking for testing would be in every TWiR until testers are no longer needed.