Description
This is based on feedback from @jonhoo during team brainstorming on Zulip. Made it an issue for visibility.
The problem
The existing labels used to call for participations do not adequately describe the complexity of involved in the progress of staring from an issue to a pull request and merging it. Contributors and maintainers often suffer from this. For instance,
#11786 and #11775 are cases I failed to tell them the consensus and complexity beforehand.
#9599 is a case that the level of design and implementation complexity were embedded in comments.
#11666 is an example that implementation might be relatively easy but need to reach a consensus first.
#11432 is an example that the contributors saw the triage and thought “hmm… does that mean feature get accepted?”
This old #2526 may need an RFC, but is probably more dying for a driver taking care of it for a long while.
Every contributor has different expectations when helping the project. Not everyone can be satisfied by completing E-easy tasks. Also, not everyone loves discussing a design back-and-forth with the risk of proposal rejection. It could save both maintainers and contributors' time if we have had a better labeling system around contributor participation.
What we have today
I said the contributor experience of each issue cannot be precisely described by existing labels. However you're not sold. Let's see what we have today.
We have three levels of participation experience. It's not easy to tell if it indicates the discussion is easy or the coding part is hard.
Two labels calling out for design works. We don't know how difficult to disambiguate the requirements into a proper design.
- https://github.com/rust-lang/cargo/labels/Needs%20design
- https://github.com/rust-lang/cargo/labels/Needs%20RFC
- https://github.com/rust-lang/cargo/labels/S-blocked sometimes implies some design works needed.
In terms of consensus, we have:
- https://github.com/rust-lang/cargo/labels/Feature%20accepted
- https://github.com/rust-lang/cargo/labels/I-nominated implies that we don't really have a consensus at this moment.
-
disposition-merge
FCP with intent to merge on an issue means it has reached a consensus to accept a proposal among tagged people. - disposition-close on an issue means it has reached a consensus to reject a proposal among tagged people.
Apart from I-nominated. All other labels seems more like “there is a consensus” instead of “what is the level of consensus here?” or “how risky is my work to get rejected?”.
Factors affecting contributor experience
Jon's original text summarizes these aspects down to three factors: complexity, ambiguity, and consensus. I copied the original words from him verbatim. Click to expand for details.
Complexity: how hard is this issue to fix/feature to implement in terms of just code adjustments.
This focuses solely on the “mechanistic” aspects of the change: how many places need to be changed, how involved are those changes, how many knock-on changes do we expect, etc. A complexity-low issue would only require changing one or two files (ideally just one or two functions) in fairly straightforward ways, or if there are more widespread knock-on changes they are entirely deterministic and could (ideally) be performed by search-replace. A complexity-high issue on the other hand is likely to require involved changes across multiple points in the code-base, and those changes are in turn likely to require non-trivial changes to other parts of the code that interface with the first set.
Ambiguity: how much design work is necessary to land the necessary change.
This is all about the degree to which the bug/feature is un-/under-specified and requires active decision-making — to what extent will a prospective contributor have to sit down and think/consider how to best fix this issue/implement this feature. An ambiguity-low issue should require very few decisions, if any, to be made. A contributor should be able to go make “the changes”, and it should be relatively well-understood (or obvious) what “the changes” are. An ambiguity-high issue on the other hand will require a signifiant exploration of the problem domain and several decisions will have to be made as part of the implementation. Those decisions need to be documented, and a rationale given, in case a subset of them need to be discussed or changed. A a contributor to such an issue is also going to have to think about implications beyond just the change they're making, such as how it interacts with other project features, backwards-compatibility, etc. Very high ambiguity issues are likely to require an RFC.
Consensus: how comfortable are the core project maintainers with the proposed path forward.
This is essentially scoring “how likely is it that this will land without too much discussion after I've done the work”. Alternatively, it can be thought of as how controversial the change is. Unlike the other two, -high here implies “easier” and -low implies “harder”. It is also generally going to be inversely proportional to the ambiguity score, but not always. A consensus-high issue is one where little discussion is likely to have to happen once a PR is made. It'll still have to go through review and such, but it's unlikely that the fundamental approach is going to be questioned or endlessly debated. A consensus-low issue on the other hand is likely to require extensive debate before landing, and has a real danger of not landing even with a perfect implementation because of considerations beyond the quality of the feature or bugfix itself. PRs for consensus-low issues will almost certainly require an FCP.
Labels indicating these factors
The proposal above proposes 9 labels for these factors:
Complexity-highComplexity-mediumComplexity-lowAmbiguity-highAmbiguity-mediumAmbiguity-lowConsensus-highConsensus-mediumConsensus-low
It is not clear how these labels fit in the existing label system in the wider Rust community. We could probably make E-easy means Complexity-easy. We could also make Ambiguity-* become E-ambiguity-*, so that contributors can figure out “Oh! I expected to experience more discussions if working on that.” Consensus-* may indicate that maintainers would have to get a conclusion, so they may fit in I-consensus-* category.
But perhaps they should all fit in E-* so people know they are all about contributor experience.
When will it start
I do think this kind of granular labels are invaluable. It is also a two-way door decision — it can always easily be reverted. The only stuff I am not comfortable is naming and how to fit in the current system (and also choosing label colors). So, if any of the team member give me a green light. I may just try one approach and see how far it goes.
Additional information
We could also add a E-needs-driver to indicate that we need a mid- to long-term driver for a feature/issue, though this is slightly tangent to the issue.