Skip to content

Add design process section to the docs #16397

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 16, 2025
Merged

Add design process section to the docs #16397

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0ef3777
Add design process section to the docs
alamb Jun 13, 2025
aebdf1b
Update docs/source/contributor-guide/index.md
alamb Jun 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions docs/source/contributor-guide/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,26 @@ Features above) prior to acceptance include:
[extensions list]: ../library-user-guide/extensions.md
[design goal]: https://docs.rs/datafusion/latest/datafusion/index.html#design-goals

### Design Build vs. Big Up Front Design

Typically, the DataFusion community attacks large problems by solving them bit
by bit and refining a solution iteratively on the `main` branch as a series of
Pull Requests. This is different from projects which front-load the effort
with a more comprehensive design process.

By "advancing the front" the community always makes tangible progress, and the strategy is
especially effective in a project that relies on individual contributors who may
not have the time or resources to invest in a large upfront design effort.
However, this "bit by bit approach" doesn't always succeed, and sometimes we get
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
However, this "bit by bit approach" doesn't always succeed, and sometimes we get
However, this "bit by bit approach" doesn't always succeed, and sometimes we get

wondering if this is needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea behind this sentence is to acknowledge the tradeoffs inherent in "design / build" vs "big design all upfront" (it is this tension that actually sparked the original comment in the first place))

stuck or go down the wrong path and then change directions.

Our process necessarily results in imperfect solutions being the "state of the
code" in some cases, and larger visions are not yet fully realized. However, the
community is good at driving things to completion in the long run. If you see
something that needs improvement or an area that is not yet fully realized,
please consider submitting an issue or PR to improve it. We are always looking
for more contributions.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Of course we always have to be 🎣


# Developer's guide

## Pull Request Overview
Expand Down