Proposing the Incubator pattern.

[gyehuda]

Based on the description of the pattern https://www.linkedin.com/pulse/culture-behaviors-innersource-three-part-blog-series-3-gil-yehuda/ we are proposing the document and propose a new pattern. This is for teams that manage a centrally used software resource that other engineers at the company also have to use (e.g. UI components, build pipeline code, coding pattern that underlies a set of products, APIs, microservices, etc.)

The central "owning" team needs a way to allow internal "user/consumer" teams to participate in the development and enhancement of the central team's code assets. Yet, they may be reluctant to open it up to all engineers if they lack an acceptance process. Not opening up will often result in other teams using their own code instead of the approved code, or being forced to use the inadequate central code.

This pattern suggests a formal process that operates as an incubator pipeline. There is a low bar to entry into the incubator. and a higher bar to exit into the approved top-level project. These two bars are managed by test cases. This pattern should allow teams to get on board into a process that allows them to use incubee code as well as top-level code. Comments are welcome (kindly read the blog post for more context).

[spier]

I really like this proposal. Thank you for sharing it.

You mention in your post that the suggested Incubator pattern may have some overlap with other patterns, or that they may be complementary. That is completely fine, as patterns are meant to be used in combination with one another anyways. They are really a menu that people can choose from and adapt to their specific organization context. So let's not worry about any potential overlap for now.

Your story with the UIC team in the post would also make for a really nice way to explain the concept, as many companies will have components and teams similar to this.

One question:

Is this pattern specifically trying to help InnerSource to thrive in bureaucratic cultures? You mention this challenge in your blog post too.

It is also possible that you are observing and discovering multiple patterns here.

Once you start writing the pattern you would pretty quickly discover if you are actually after multiple patterns here. So we can still split things into multiple patterns then.

To start creating the pattern I would suggest these next steps:

1. take the pattern template and copy the content into a new file e.g. "patterns/1-initial/incubator-pipeline.md"
2. start filling in the different sections based on the documentation in the template
3. submit a PR with your first draft, to get feedback from the InnerSource Patterns working group (and anybody else reading your PR really)

My recommendation it to spend max 1-2 hours on this first draft. That way we can quickly confirm if the different sections of the pattern template make sense to you, and if you have any further questions. We can then improve the pattern in further iterations until we have a version that we are happy to share more broadly.

[gyehuda]

Thanks, please see #338 and we can discuss what we can do as a next step. I really appreciate the help and guidance.

[fioddor]

The suggested Incubator Pipeline pattern resembles the process ISC patterns go through, escalating through maturity levels.

If we can assimilate the ISC process for patterns as a particular application of this pattern, we could and ought to include InnerSource Commons to the list of known instances.

[spier]

That's a great observation @fioddor!
One could indeed argue that the level 1 (Initial) of our maturity levels is a bit like an incubator. Once a pattern advances to the next level (2/structured) that pattern is considered mature enough to be published in our online book, while initial patterns are only published in this repo.

We would likely not list the ISC Patterns maturity level approach as a Known Instance though because our patterns project is open source, rather than InnerSource. So far we have tried to only use Known Instances of companies that use a given pattern in a company-internal InnerSource scenario.

[spier]

I thought about this more and had a question for @gyehuda:
The UIC team in the story of your blog post is building a UI component library.

Are you aware of any open source project, maybe even something related to such UI component libraries, that is using a similar an incubation process for community contributions?

[gyehuda]

When describing the pattern, I was inspired by things like the Apache Software Foundation's incubator, as well as Linux Foundation's sandbox/incubator processes used in CNCF, CDF, and other open source foundations. Those help entire projects get into the foundation. In my open source work, I've found those to be very successful. They helped me raise the bar on projects that wanted to get more visibility by getting them into an incubator. Many of those projects graduated and are very successful today, in open source. This pattern here is like a smaller fractal of that pattern -- to help components get into an open source library.

Note: the example used in my blog was about a UI components team. As that is one of the teams I'm working with now, and I think they are the furthest along in my workplace (and have more to go too). I've seen this operate with build-pipelines too. My previous open source experience modeled this pattern with various degrees of success, but not as much as I've wanted.

• Screwdriver.cd (both InnerSource within Yahoo and Yahoo Japan as well as Open Source in the CD Foundation incubator) allows for internal and external contributors to contribute build scripts. These did not have a formal "incubator pipeline" process, but somewhat of an informal one.
• Denali.design is another project I helped publish, It is UI design, and it too invited external participants, but I don't think it got formalized to work a pipeline. In part, it's hard to contribute to a project when the project has very strong opinions about how components should work.

This is what makes this pattern so interesting. Without it, it's hard to get an out-group contribution into a project when the in-group maintainers have a deep responsibility for achieving certain outcomes. The in-group has many reasons to resist. Yet, with better clarity on the pipeline, I'm seeing an in-group welcome contributions. That's happening internally at my company and I'm hoping with more formality in this pattern (and suggestions from the community for improvements) we can grow this pattern to address those cases where the in-group is hesitant to accept contributions, but realizes they are benefited if they do.