Community contributions - expectations on changes being integrated

[Danielku15]

Title: How do community contributions get triaged?

Hey folks,

First off, respect for the pace here. It's impressive how much is moving to make Vector better, faster, stronger 💪

Some context on where we're coming from: we're planning to build a key data exchange flow of our upcoming service platform on Vector, something we'd otherwise have written ourselves. We found a couple of pieces missing for our use case, and rather than maintaining our own thing on the side, we figured we'd contribute them upstream so everyone benefits. Felt like a win-win.

Concretely it's two closely related features: #22316 and #19397. The second builds on the first, so they need to land in order. I opened the first PR on June 1 (#25545) hoping we'd see it reviewed reasonably soon and could then follow up with the second.

Two months on, it hasn't been picked up yet. Development elsewhere in the project is clearly very active, so I don't read this as the team being idle — I just can't tell where our PR sits, or what would help move it along. On our side, what started as an opportunity to build on Vector has slowly turned into something we have to plan risk around. We're getting close to deciding whether to (a) maintain our own build with the missing features, or (b) look at alternatives.

I'm not trying to nag anyone. What I'm really after is enough insight to judge how contributions are handled here, so we can plan properly — both for this PR and for the ones we'd like to send in future. So:

• Is there a triage or prioritization process for community PRs, and what makes something get picked up?
• How does the roadmap get balanced between what the team has planned and what comes in from users?
• Now that Vector is part of Datadog: is there a commercial route — enterprise support or similar — for companies that need a specific feature or fix prioritized?
• Is a richer plugin system on the table at all? Being able to add our own sinks, transforms and sources on top of the core would solve much of this for us. It comes at a cost, sure, but it would also let the ecosystem grow without every change having to live in core.

Thanks a lot,
Daniel

[pront]

Hi @Danielku15, thanks for starting this discussion, and for laying it out as clearly as you did.

Currently we have ~200 PRs that are blocked awaiting our review.

The PR backlog grew significantly this year because we had team members on multi-month leave, we had more PRs than usual, and we also had some other urgent work that could not wait. The perfect storm. You can see some trends here:

• PRs trend
• Contributors monthly

The team is still small and we are working to grow it in the second half of this year. Over the next few months we will also publish a short policy to document our backlog trimming strategy.

We do plan to prioritize community PRs more. In the meantime, the short version of what gets picked up faster: small, single-purpose PRs with no unresolved threads. That is a heuristic rather than a rule, so we may still prioritize a large PR when it matters, but it holds most of the time.

#25545 is now approved. Once it lands, please open the append blob follow-up. We would much rather merge these contributions than have you carry a fork. It's easier for you, and the whole community gets the features. That said, forks are genuinely useful: they accelerate development, they let you prove out a concept, and what comes back upstream afterwards is more battle-tested.

Now that Vector is part of Datadog: is there a commercial route — enterprise support or similar — for companies that need a specific feature or fix prioritized?

Vector will remain open source software without paid support for the foreseeable future.

Is a richer plugin system on the table at all?

It is a genuinely cool idea. The catch is that there are a lot of open questions we need to think through before implementing it. That being said, it is high on the list of "ambitious but time-consuming project ideas".

[Danielku15]

Thanks a lot for this valuable insight. The charts give a nice insight on how Vector is evolving in both people and features. I'm also maintaining some OSS projects and I know how hard it can be to balance the capacities and priorities right but I'm happy to see Vector thriving and as a technology it feels to be the right choice for our needs.

Thanks for putting #25545 on the fast lane and I'll do my best to also get #25627 ready for integration ASAP. If there is anything I can do to make your work easier I'm happy to adapt.

On the plugin system I asked Claude to do some research on how a plugin system could look like by looking at the current code architecture and the options adopted in the market. Balancing ABI, performance and the right level of extensibility is not easy. I saw the experimental WASM based system and also dug lately into how other products handle their extensibility (like HAProxy and Envoy). I'm exploring options of externally developed plugins (C-ABI / Rust specific) but also how VRL could be extended for that purpose.

I hope to have some documents available soon to share.

[Danielku15]

PR #25627 is now also ready.