-
Notifications
You must be signed in to change notification settings - Fork 65
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #29 from jupyter/safia/feb/meta-jep
Add meta-JEP for new JEP process
- Loading branch information
Showing
1 changed file
with
150 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,150 @@ | ||
| | Item | Value | | ||
| | ---------- | ------------------------------------------------------------ | | ||
| | JEP number | 29 | | ||
| | Title | Jupyter Enhancement Proposal | | ||
| | Authors | Jason Grout ([jason@jasongrout.org](mailto:jason@jasongrout.org)), Safia Abdalla ([safia@safia.rocks](mailto:safia@safia.rocks)), John Lam ([jflam@microsoft.com](mailto:jflam@microsoft.com)), Kevin M. McCormick ([mckev@amazon.com](mailto:mckev@amazon.com)), Pierre Brunelle ([brunep@amazon.com](mailto:brunep@amazon.com)), Paul Ivanov ([pi@berkeley.edu](mailto:pi@berkeley.edu)) | | ||
| | Status | Submitted | | ||
| | Type | P - Process | | ||
| | Created | 23-Feb-2019 | | ||
| | History | 04-Mar-2019 | | ||
|
|
||
| ### Background | ||
|
|
||
| Project Jupyter has an existing repository and process for “Jupyter Enhancement Proposals” or JEP for short. The repository is here: | ||
|
|
||
| <https://github.com/jupyter/enhancement-proposals> | ||
|
|
||
| This format was originally used by IPython (IPEP) and continues to be used by the Python Core language (PEP). As currently envisioned, JEPs are written documents, in the form of a Pull Request to this repo, which describes significant proposed units of work on the projects. The README of the repo summarizes this as: | ||
|
|
||
| “Jupyter Enhancement Proposals will be used when presenting changes or additions that affect multiple components of the Jupyter ecosystem OR changes to a single key component.” | ||
|
|
||
| Since the JEP process was created, Jupyter has grown in the number of users, diversity of usage cases, number of contributors, and number of repos/org. For a variety of reasons, the JEP process has not scaled with this growth. The current situation is characterized by: | ||
|
|
||
| 1. Given the scope of development, it is difficult to follow all of the activity; | ||
| 2. As a result, most decisions are made in a distributed manner by local teams of contributors working on particular repos and orgs (hub, lab, ipython, jupyter-widgets, etc.); as a consequence: | ||
| 3. Distributed decisions that affect the project as a whole are made, without all relevant parties being able to participate and contribute. | ||
| 4. This is leading to fragmentation and slow progress in project wide concerns, such as the core protocols and standards of Jupyter. | ||
|
|
||
| As a result, there is a need to revitalize and reorganize an efficient and robust centralized proposal and decision making process for major work units across the project. | ||
|
|
||
| ## Goals and Tenants | ||
|
|
||
| Project Jupyter has adopted the Jupyter Enhancement Proposal process (JEP) to address distributed collaboration and experimentation as the project scales in the dimensions of contributors, components, and lines of code. At a high level, the primary guiding principle of the JEP process is to encourage collaboration and discussion as early as possible in the lifecycle of a major proposed change in Jupyter, with the goal of preventing costly rework, competing ideas, and unnecessary forking or fragmentation of ideas. | ||
|
|
||
| Several sub-goals exist for this process: | ||
|
|
||
| - **Maximize success of large proposals** that get stalled in the wrong venue (e.g. a single PR comment thread) | ||
| - Provide a **better alternative to “piecemeal” development** where multiple PRs to build an end-to-end set of functionality are split across multiple GitHub project without broad consensus, context, or guidance. | ||
| - Provide a **clear, time-limited, and authoritative process for work proposals**, to facilitate funding conversations. (e.g. provide a concrete artifact to reference in a grant proposal, roadmap item, etc.) | ||
| - Provide a **consolidated “stream” of all proposals across the entire Jupyter community** that contributors of all levels can monitor and selectively engage in. | ||
|
|
||
| With that in mind, the JEP process operates under the following tenets: | ||
|
|
||
| - **The JEP process is intended for proposed changes of non-trivial scope. “**Non-trivial” is addressed below in the “JEP / Not-a-JEP Rubric” of this document. If proposals that go through the JEP process do not receive the benefits listed above, the JEP process should be amended to better scope what applied. | ||
|
|
||
| - **The JEP process naturally complements the PR process, but does not replace it.** A thoroughly-reviewed and approved JEP is a valuable reference during a PR to reduce friction, reduce time-consuming context sharing, and encapsulate decisions and other discussions. Moving a premature PR into a JEP should be a lightweight process that doesn’t cause friction for the contributor. | ||
|
|
||
| - - GitHub issue and PR templates, for example, across the entire Jupyter project, should have references to the JEP process as a possible outcome of a given PR. | ||
|
|
||
| - **There is one JEP repository, all Jupyter-governed projects must use it.** To faciliate the easiest possible adoption and high visibility of ideas, a single JEP repository will be used. Even if a JEP only applies to a single organization. | ||
|
|
||
| - The JEP process **has multiple valid use cases**. Each use case might have a slightly different expected workflow or base JEP template. Some expected use cases include: | ||
|
|
||
| - - Non-trivial feature proposals within a single component that would benefit from process. (e.g., a non-trivial change to JupyterLab that would benefit from formal process within the JupyterLab project) | ||
| - Non-trivial features or improvements that span multiple projects. | ||
| - Any proposed changes to published APIs or core specifications (e.g., nbformat) | ||
| - Changes to the JEP process itself. | ||
| - Creating a new GitHub repo in one of the official Jupyter orgs | ||
|
|
||
|
|
||
| ## JEP Submission Workflow | ||
|
|
||
| ### Phase 1: Pre-proposal | ||
|
|
||
| This is the least formal stage of any jupyter enhancement proposals. During this phase, discussions on the mailing list, in-person, on github issues are all fine to gauge community interest, feasibility, consequences, and to scope out technical impact and implementation details. | ||
|
|
||
| In order to transition out of the pre-proposal stage, the following checklist must be complete: | ||
|
|
||
| 1. A github issue on the Jupyter Enhancement Proposals repo is created that | ||
| 2. 1. Briefly outlines the proposal | ||
|
|
||
| 2. Suggested review team (optional) | ||
|
|
||
| 3. Why it should be a JEP | ||
|
|
||
| 4. 1. See the “JEP / Not-a-JEP Rubric” below. | ||
| 3. A *Shepherd* is identified to see the process through. (Shepherds are assigned on a round-robin basis from a set of interested engaged volunteers). | ||
| 4. A number is assigned to the JEP to track it through the rest of the process. | ||
|
|
||
| Outcome: | ||
|
|
||
| The Shepard decides if the JEP criteria have been met. | ||
|
|
||
| - *It's a JEP! Please create a new PR with the JEP contents using template X. On that basis, you can resolve this issue unless you have further questions.* | ||
| - It’s not a JEP. (Provide reasons and close the issue.) | ||
|
|
||
| ### Phase 2: RFC for the JEP | ||
|
|
||
| Submission: The author submits an initial draft of the JEP as a PR to the JEP repository starting from the relevant template decided in the pre-proposal stage. The Shepherd assigns a number (the GitHub PR number? A sequential number? Perhaps the number isn't assigned until the JEP is merged?). The Shepherd assigns a Review Team. | ||
|
|
||
| Request For Comment (RFC) phase: The proposal is iterated on with feedback from the Review Team and the community at large. The Shepherd helps engage the Review Team. After the Review Team members have signed off on the JEP, with the criteria that there are no major objections, and at least some of the Review Team are in favor, the Shepherd initiates the Final Comment Period. | ||
|
|
||
| Final Comment Period (FCP): The community at large has 10 calendar days in which to provide any final comments on the JEP. A JEP may revert back to RFC if objections are supported by the Review Team. If not reverted to the RFC phase, the JEP is approved at the end of the FCP. | ||
|
|
||
| ### Phase 3: Work Commences | ||
|
|
||
| Once a JEP has been merged into the jupyter/enhancement-proposal repository, development work can commence on the JEP. As the implementer(s) is submitting a pull request or pull requests in relation to the JEP, they should provide a reference to the JEP so that reviewer has background context on the JEP. | ||
|
|
||
| As the JEP is being implemented, the implementer(s) are submitting pull requests for the JEP, they should update the JEP with addendums to denote the progress of the implementation using the following stages. | ||
|
|
||
| 1. In progress implementation via (list of PRs). | ||
| 2. Fully implemented via (list of PRs). | ||
|
|
||
| If in the course of implementation, it is discovered that the implementation needs to be radically different from what was defined in the original JEP, then a pull request needs to be submitted to modify the original JEP with the new necessary implementation and a note sighting the need for a modification to the JEP. This pull request should be re-approved by the original review team. | ||
|
|
||
| If in the course of the implementation, the implementer(s) can choose to withdraw from the original JEP if they are no longer interested in implementing the JEP or see infeasibilities in the JEP. | ||
|
|
||
| ### Paths of the status of JEPs | ||
|
|
||
|  | ||
|
|
||
| ## What qualifies as a JEP? | ||
|
|
||
| This section contains a list of principles and scenarios that can be used to help define a set of principles for determining when something is a JEP. The principles will be used both to determine when something becomes a PR during the JEP pre-proposal stage, as well as to determine when a PR becomes a JEP at an individual repo level. | ||
|
|
||
| **Principles (If yes, Requires a JEP)** | ||
|
|
||
| - Does the proposal/implementation require PRs across multiple orgs? | ||
| - - Defining a unique cell identifier | ||
| - Does the proposal/implementation PR impact multiple orgs, or have widespread community impact? | ||
| - - Updating nbformat | ||
| - Does the proposal/implementation change an invariant in one or more orgs? | ||
| - - Defining a unique cell identifier | ||
| - Deferred kernel startup | ||
| - Does the proposal/implementation create a new concept that will impact multiple repositories? | ||
| - - Sandboxed cell outputs | ||
| - The proposal involve creating a new repository or subproject? | ||
|
|
||
| ### Glossary | ||
|
|
||
| - Jupyter Enhancement Proposal (JEP): A written document that describes a proposed unit of significant work on Jupyter. | ||
|
|
||
| - Contributor: The person who is submitting the JEP, and might but not necessarily also have intention on organizing the implementation if so approved. | ||
|
|
||
| - Shepherd: A senior Jupyter contributor that manages the pre-proposal, submission, review, approval process of a JEP. | ||
|
|
||
| - Review Team (RT): A group of Jupyter contributors, with expertise in a particular area of the project that reviews and approves JEPs related to that area. | ||
|
|
||
| - Final Comment Period (FCP): A finite-time, post-approval period for the community to make final comments. | ||
|
|
||
| - JEP Status: | ||
|
|
||
| - - Inactive | ||
| - Submitted | ||
| - Assigned | ||
| - Rejected | ||
| - Postponed | ||
| - Withdrawn | ||
| - Approved | ||
| - In progress | ||
| - Completed |