This directory contains information about the process of making Delta protocol changes via RFCs and all the RFCs that have been proposed since this process was adopted.
Here is the history of all the RFCs propose/accepted/rejected since Feb 6, 2024, when this process was introduced.
| Date proposed | RFC file | Github issue | RFC title |
|---|---|---|---|
| 2023-02-02 | in-commit-timestamps.md | delta-io#2532 | In-Commit Timestamps |
| 2023-02-09 | type-widening.md | delta-io#2623 | Type Widening |
| 2023-02-14 | managed-commits.md | delta-io#2598 | Managed Commits |
| 2023-02-26 | column-mapping-usage.tracking.md | delta-io#2682 | Column Mapping Usage Tracking |
| 2023-04-24 | variant-type.md | delta-io#2864 | Variant Data Type |
| 2024-04-30 | collated-string-type.md | delta-io#2894 | Collated String Type |
| Date proposed | Date accepted | RFC file | Github issue | RFC title |
|---|---|---|---|---|
| 2023-02-28 | 2023-03-26 | vacuum-protocol-check.md | delta-io#2630 | Enforce Vacuum Protocol Check |
| Date proposed | Date rejected | RFC file | Github issue | RFC title |
|---|---|---|---|---|
| ... |
Create a Github issue of type [Protocol Change Request].
- The description of the issue may have links to design docs, etc.
- This issue will serve as the central location for all discussions related to the protocol change.
- If the proposal comes with a prototype or other pathfinding, the changes should be in an open PR.
After creating the issue and discussing with the community, if a basic consensus is reached that this feature should be implemented, then create a PR to add the protocol RFC before merging code in master.
- Clone the RFC template
template.mdand create a new RFC markdown doc. - Cross-link with the issue with "see #xxx". DONT USE "closes #xxx" or "fixes #xxx" or "resolves #xxx" because we don't want the issue to be closed when this RFC PR is merged.
Note:
- For table features, it is strongly recommended that any experimental support for the feature uses a temporary feature name with a suffix like
-dev. This will communicate to the users that are about to use experimental feature with no future compatibility guarantee. - Code related to a proposed feature should not be merged into the main branch until the RFC attains "proposed" status (that is, the RFC PR has been through public review and merged). Until the RFC has been accepted (that is, the proposed changes have been merged into the Delta specification), any code changes should be isolated from production code behind feature flags, etc. so that existing users are not affected in any way.
For a RFC to be accepted, it must satisfy the following criteria:
- There is a production implementation (for example, in delta-spark) of the feature that has been thoroughly well tested.
- There is at least some discussion and/or prototype (preferred) that ensure the feasibility of the feature in Delta Kernel.
When the success criteria are met, then the protocol can be finalized by making a PR to make the following changes:
- Closely validate that the protocol spec changes are actually consistent with the production implementation.
- Cross-link the PR with the original issue with "closes #xxx" as now we are ready to close the issue. In addition, update the title of the issue to say
[ACCEPTED]to make it obvious how the proposal was resolved. - Update
protocol.md. - Move the RFC doc to the
acceptedsubdirectory, and update the state in index.md. - Remove the temporary/preview suffix like
-devin the table feature name from all the code.
However, if the RFC is to be rejected, then make a PR to do the following changes:
- Cross-link the PR with the original issue with "closes #xxx" as now we are ready to close the issue. In addition, update the title of the issue to say
[REJECTED]to make it obvious how the proposal was resolved. - Move the RFC doc to the
rejectedsubdirectory. - Update the state in
index.md. - Remove any experimental/preview code related to the feature.