docs: Update documentation on Epics and Supervising Maintainers #17505

alamb · 2025-09-10T17:10:50Z

Which issue does this PR close?

Part of Discussion: DataFusion Improvement Proposal (DIPs) Process? #16886

Rationale for this change

It can be unclear

What projects are currently active and under discussion
How larger projects are done in the context of DataFusion

What changes are included in this PR?

Add links to EPIC and PLANNED epic (as described here)
Add a description of "Sponsoring Maintainer" as proposed by @2010YOUY01 here -- rather than a "requirement" as a helpful description of how things work

Here is a rendered preview:

(updated)

Are these changes tested?

by ci

Are there any user-facing changes?

yes, new docs

comphead · 2025-09-10T20:05:09Z

docs/source/contributor-guide/roadmap.md

+maintainers are more likely to sponsor features they are particularly interested
+in or align with their use of DataFusion.
+
+If you are willing to be a sponsoring maintainer for a feature, please say so


Thanks @alamb please help me to understand Sponsoring Maintainters. Are you suggesting to create a pool of active maintainers for specific feature, like XYZ, ABC is supporting Join development, QWE for planning, ZXC for optimizer work?
So the PR author can tag all of them and depending on availability the maintainers can help review.

Is my understanding correct?

I was trying to make it clear that any project that doesn't have an active maintainer that is willing to help move it along to completion is unlikely to succeed. Maybe this is obvious but I think it might help to make it explicit.

I don't think we have a formal process (yet) to find such a maintainer other than the normal course of comments, etc. Clearly I play this role for many of the projects, but I am hoping to move to a place where other committers can also drive move projects too.

Hopefully that makes sense

I'll try and reword this a bit for clarity too

Omega359 · 2025-09-10T22:37:45Z

LGTM

comphead

Thanks @alamb it is a great start

2010YOUY01

Thank you for the great explanations 👍🏼

2010YOUY01 · 2025-09-11T04:26:24Z

docs/source/contributor-guide/roadmap.md

+
+We have found that most successful epics have one or more "sponsoring
+maintainers", a committer ([see here for current list]) who take the lead on reviewing and committing PRs, help with design,
+and coordinate and communicate with the community. Without a sponsoring


Maybe we can explicitly say: If you want to ship a large feature, we recommend finding a sponsoring maintainer upfront; otherwise, the PRs might remain unreviewed for a very long time.

This is a great idea -- I added this in 075ebde

Jefffrey

Is it worth changing the term "sponsoring" to something like "driving" or "championing", to avoid any confusion with the monetary meaning of sponsoring?

alamb · 2025-09-12T10:23:36Z

Is it worth changing the term "sponsoring" to something like "driving" or "championing", to avoid any confusion with the monetary meaning of sponsoring?

That is a really good point. Here are some brainstorm options

"Driving Committer"
"Championing Committer"
"Shepherding Committer" (my personal favorite)
"Supervising Committer"
"Herding Committer"
"Stewarding Committer"
"Managing Committer"

🤔

comphead · 2025-09-12T14:45:13Z

Is it worth changing the term "sponsoring" to something like "driving" or "championing", to avoid any confusion with the monetary meaning of sponsoring?

That is a really good point. Here are some brainstorm options

"Driving Committer"

"Championing Committer"

"Shepherding Committer" (my personal favorite)

"Supervising Committer"

"Herding Committer"

"Stewarding Committer"

"Managing Committer"

🤔

supervising sounds great IMO.

BlakeOrth · 2025-09-12T21:11:18Z

That is a really good point. Here are some brainstorm options

"Driving Committer"
"Championing Committer"
"Shepherding Committer" (my personal favorite)
"Supervising Committer"
"Herding Committer"
"Stewarding Committer"
"Managing Committer"

Of the options given here I feel that "driving" and "championing" might lean a bit too colloquial and might be more difficult for non-native English speakers in the community to understand (caveat: I am a native English speaker, so I may be entirely off base here as I have absolutely no personal reference). I believe that "supervising" or "managing" is likely the most clear. I think other options in this vein would be "lead committer" or "principal committer".

"Shepherding Committer" (my personal favorite)

I think this is perhaps a bit less clear than some other options, but like how it implies a more collaborative relationship between the community member and the committer. Another potential option for a more collaborative word might be "mentoring committer".

BlakeOrth · 2025-09-12T21:22:29Z

docs/source/contributor-guide/roadmap.md

+the role, as it can be hard to tell sometimes whether a committer is simply
+participating and giving general feedback.
+
+[see here for current list]: governance.md


Would it be reasonable to directly link to the "phone book" here (https://projects.apache.org/committee.html?datafusion) instead of the governance doc? I'm personally not very familiar with these docs and skipped right over the "phone book" link on the governance doc when I first looked at it because I was looking for a list of usernames of committers.

Yes, and this is a good point about the phone book in general -- I'll try and make that easier to understand

I made a separate PR to do this

Add explicit PMC/committers list to governance docs page #17574

Omega359 · 2025-09-12T22:02:11Z

I'll add my vote for "Supervising Committer"

2010YOUY01 · 2025-09-13T03:14:45Z

That is a really good point. Here are some brainstorm options
"Driving Committer"
"Championing Committer"
"Shepherding Committer" (my personal favorite)
"Supervising Committer"
"Herding Committer"
"Stewarding Committer"
"Managing Committer"

Of the options given here I feel that "driving" and "championing" might lean a bit too colloquial and might be more difficult for non-native English speakers in the community to understand (caveat: I am a native English speaker, so I may be entirely off base here as I have absolutely no personal reference). I believe that "supervising" or "managing" is likely the most clear. I think other options in this vein would be "lead committer" or "principal committer".

"Shepherding Committer" (my personal favorite)

I think this is perhaps a bit less clear than some other options, but like how it implies a more collaborative relationship between the community member and the committer. Another potential option for a more collaborative word might be "mentoring committer".

+1 for supervising. I agree shepherding is great, but it might confuse non-native english users.

2010YOUY01 · 2025-09-15T10:48:07Z

Perhaps we can add it to the issue template:

# For non-trivial features, is there any supervising maintainer?
<!--
(link to explanation)
-->

This issue is looking for a supervising maintainer.

alamb · 2025-09-16T12:14:32Z

Perhaps we can add it to the issue template:

I think this is likely a good idea.

However, I suggest we wait to see how much this idea of supervising maintainers works out and then add it to the template if it seems it is gaining traction

alamb · 2025-09-16T12:16:55Z

I'll plan to merge this tomorrow unless anyone has additional comments

alamb · 2025-09-17T16:17:06Z

All right -- I put it in the merge queue and hopefully it will merge shortly

alamb · 2025-09-17T20:05:34Z

Whoo hoo the merge bot is working !

docs: Update documentation on Epics and Sponsoring Maintainers

2f4f73e

github-actions bot added the documentation Improvements or additions to documentation label Sep 10, 2025

alamb mentioned this pull request Sep 10, 2025

Discussion: DataFusion Improvement Proposal (DIPs) Process? #16886

Open

alamb self-assigned this Sep 10, 2025

comphead reviewed Sep 10, 2025

View reviewed changes

comphead approved these changes Sep 10, 2025

View reviewed changes

2010YOUY01 approved these changes Sep 11, 2025

View reviewed changes

alamb added 2 commits September 11, 2025 06:07

Merge remote-tracking branch 'apache/main' into alamb/document_epics

1b4bcb0

Improve wording about finding sponsoring maintainers

075ebde

alamb mentioned this pull request Sep 11, 2025

Support SQL pipe operator #14660

Open

18 tasks

Jefffrey reviewed Sep 12, 2025

View reviewed changes

BlakeOrth reviewed Sep 12, 2025

View reviewed changes

BlakeOrth mentioned this pull request Sep 12, 2025

Enable the ListFilesCache to be available for partitioned tables #17211

Closed

alamb changed the title ~~docs: Update documentation on Epics and Sponsoring Maintainers~~ docs: Update documentation on Epics and Supervising Maintainers Sep 15, 2025

rename to supervising maintainer

9eda70b

alamb mentioned this pull request Sep 15, 2025

Add explicit PMC/committers list to governance docs page #17574

Merged

alamb enabled auto-merge September 17, 2025 16:16

Merge branch 'main' into alamb/document_epics

ae3bec8

alamb added this pull request to the merge queue Sep 17, 2025

Merged via the queue into apache:main with commit 9cc8c8d Sep 17, 2025
5 checks passed

alamb deleted the alamb/document_epics branch September 17, 2025 20:05

docs: Update documentation on Epics and Supervising Maintainers #17505

docs: Update documentation on Epics and Supervising Maintainers #17505

Uh oh!

Conversation

alamb commented Sep 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

comphead Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Sep 10, 2025

Choose a reason for hiding this comment

Uh oh!

Omega359 commented Sep 10, 2025

Uh oh!

comphead left a comment

Choose a reason for hiding this comment

Uh oh!

2010YOUY01 left a comment

Choose a reason for hiding this comment

Uh oh!

2010YOUY01 Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Sep 11, 2025

Choose a reason for hiding this comment

Uh oh!

Jefffrey left a comment

Choose a reason for hiding this comment

Uh oh!

alamb commented Sep 12, 2025

Uh oh!

comphead commented Sep 12, 2025

Uh oh!

BlakeOrth commented Sep 12, 2025

Uh oh!

BlakeOrth Sep 12, 2025

Choose a reason for hiding this comment

Uh oh!

alamb Sep 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alamb Sep 15, 2025

Choose a reason for hiding this comment

Uh oh!

Omega359 commented Sep 12, 2025

Uh oh!

2010YOUY01 commented Sep 13, 2025

Uh oh!

2010YOUY01 commented Sep 15, 2025

Uh oh!

alamb commented Sep 16, 2025

Uh oh!

alamb commented Sep 16, 2025

Uh oh!

alamb commented Sep 17, 2025

Uh oh!

Uh oh!

alamb commented Sep 17, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

alamb commented Sep 10, 2025 •

edited

Loading

alamb Sep 13, 2025 •

edited

Loading