proposal: dep: move out of github.com/golang

[theckman]

Hi there,

Over the past few months the signals we've gotten is that The Core Go team has no interest in continuing the work of dep, or to even contribute code to it to help make the modules transition easier. After the tweet from @rsc apologizing for how the dependency management story wasn't handled well[1], I'm not aware of active work from the core team to help improve the current situation which isn't a confidence booster.

As a result, I'd like to ask that the dep source and licenses be transferred out from Google/Golang to the Gofrs GitHub organization so that, at a minimum, we can try to make dep work with projects that have modules enabled / use the semantic import versioning. We'd like to retain the current issue and PR history, while having full administrative control over the project and repository.

We may also want to pick up additional work to continuing to support dep long-term, as there is a deep philosophical divide between those who want a dep-like experience and those who are happy with modules. I don't believe these disagreements can be resolved in a single solution or implementation, and we should instead work together to ensure we can interoperate.

While I'd love to be able to have modules work for me the way I want, I don't think that's a reasonable goal. I now think it's more realistic to support dep in the interest in strengthening the Go community and the tools that are used to support it. I also think competition in dependency management tooling is as important as having competing businesses to help drive progress, and so I believe adopting dep is one step we can take towards that.

Who would be the best person to talk to get the ball rolling?

Cheers!
-Tim

[1] https://twitter.com/_rsc/status/1022588240501661696

Edit: I had someone reach out to me privately to clarify what I meant by the following line:

...I'm not aware of active work from the core team to help improve the current situation which isn't a confidence booster.

The current situation I was citing here was the gap in interoperability between modules and dep, which can result in people being in a less-than-ideal situation if their dependencies have adopted modules before they are comfortable moving.

[theckman]

For some additional context, I tweeted about this today and Sam Boyer replied:

it would certainly make sense to move dep out, given that we wrote the core of it elsewhere and then relicensed it to the Go team in good faith

• https://twitter.com/sdboyer/status/1081887613341130753

[theckman]

@adg @bradfitz @rsc I see that you were all tagged in earlier efforts to move code in here; so being the opposite I figure you may have some insight on this process.

Who would be the right people to talk to about having this project donated to the Go Community?

[ianlancetaylor]

CC @spf13

[bradfitz]

Assuming @sdboyer et al want this (please confirm), we can discuss at our next proposal review meeting. We're a bit behind due to the holiday but we'll catch up in the next couple weeks.

[carolynvs]

[rsc]

As a result, I'd like to ask that the dep source and licenses be transferred out from Google/Golang to the Gofrs GitHub organization

I don't understand what you mean by this.

The source code copyright is held by "The Go Authors", which is the group of relevant contributors. Google does not hold the copyright and therefore cannot transfer it to anyone else.

The license is BSD. It already grants permission for anyone to make a copy, make changes, and distribute those changes, with the usual notice requirements. Google is not going to change the license on this code any more than we are going to change the license on the Go project itself.

If you want to start a fork of dep, the "source and licenses" are not stopping you.

We'd like to retain the current issue and PR history, while having full administrative control over the project and repository.

This is different. Here you are asking that the GitHub project itself be transferred. That's something we could do (while leaving the copyright and licenses alone), but I am not even sure that this is appropriate. Dep both helped us learn what package management in the go command might look like and also still serves as a useful tool until modules are completely ready. I don't see why that wouldn't stay in github.com/golang. And once modules are ready, dep should go away. If others want to fork it and maintain a fork then that's fine. But the official dep that the Go project recommends should probably be the one in our GitHub org. If more people need commit access to github.com/golang/dep we can clearly do that.

I hadn't seen the gofrs org until today. I am skeptical of that org because it purports to be (according to the headline at github.com/gofrs) "The Go Community". The only crisp definition of "The Go Community" that I've ever seen is "all Go programmers". Clearly the gofrs github org is a small, self-selected set of developers, not the entire Go community. A small group of 17 people literally claiming to be (or even to represent) the entire Go community is for me a red flag. Why should we trust gofrs to take over dep?

[rsc]

Sorry, I clicked the wrong "send" button; meant to leave this open for further discussion.

[theckman]

@ianlancetaylor @carolynvs @bradfitz Thank you so much for your 👀 on this, and I appreciate you being willing to discuss it when getting back to things after the holidays.

[theckman]

@rsc Thank you for taking time to look in to this and reply. There are quite a few points you've raised, so hopefully I can address them individually. I'm sorry that this ended up becoming a bit more long-winded than I had originally intended, but I think sharing context transparently is the best I can do to help alleviate any concerns that you or others have.

I hadn't seen the gofrs org until today. I am skeptical of that org because it purports to be (according to the headline at github.com/gofrs) "The Go Community". The only crisp definition of "The Go Community" that I've ever seen is "all Go programmers".

It's probably worth starting with some context sharing on the Gofrs, since I think that might be the more important piece here. It's unfortunate you weren't able to make it to GopherCon this year; I'd have tried to get you to come to my lightning talk[1] where I broke ground on the idea. That said, not knowing me from Adam it might have been a hard sell to get you to attend. 😄

The Gofrs initiative originally came out of the general channel on the Slack Workspace (where we now have over 35.3k registered users). While it started there, it's an effort we're hoping to keep open to the larger Go Community no matter how you interact with it. I do volunteer on Slack as an admin, so I'm generally keeping an eye on things throughout the day for things to improve. Whether in how the community is running, or for feature requests to Go itself. In doing so I noticed two patterns emerging that made me feel like there was a gap to be filled:

• Because our ecosystem is a bit newer than other languages out there, there are often a few options for solving the same problem without a clear choice of which to use. This was especially challenging to those newer to Go or software development who don't have as much experience measuring the quality of a project
• Individuals who invested their time in building libraries that had become widely used in the Go ecosystem had no explicit support from the community, and there were a few cases where those authors vanished with outstanding critical bugs.

One of the biggest things that continually comes up and sticks with me is the number of people who comment that our community is one of the most supportive and helpful they've ever interacted with, especially when coming in with a Go problem and looking for some insight. It brings me lots of pride and joy to know that we were able to build that community together, and that everyone (even non-admins) is biased towards making sure it remains welcoming.

Knowing that made me wonder if there was more we could do as a community to support those who are investing their time in making our ecosystem attractive to so many developers, and could we also build a more clear set of community-maintained "trusted" packages that we can suggest to newcomers? That is basically the foundational origin of The Gofrs. If there's more I can share I'd be happy to do it, but I'm not sure this issue would be the best medium for that.

Clearly the gofrs github org is a small, self-selected set of developers, not the entire Go community. Why should we trust gofrs to take over dep?

You're totally right that we're self-selected, and we see it as being similar to how these community spaces (Slack, the forum, etc.) and GopherCon itself were started. We feel they've ended up being effective because of the passion of those who are involved, and we hope to use that same kind of passion in The Gofrs.

A small group of 17 people literally claiming to be (or even to represent) the entire Go community is for me a red flag.

Specifically around the number of users in the Gofrs organization, most have chosen not to publicize their membership so we're actually above 40 members. I see this only growing as we put in more time in supporting the community, and our needs become more diverse. That said I don't think our size should be the judge of how effective we can be, but rather the passion we have for the Go programming language and the community that's formed around it. We really care about making Go, and those who use it and build libraries in it, successful.

Why should we trust gofrs to take over dep?

This is a great question.

Trust is something that needs to be built, and we feel that our work on the gofrs/uuid package has so far started to build trust in the Go community. We can only continue to build trust by delivering on our goals to support the community, and we feel alleviating some of the pains today around dep is important. We also value how important it is not to destroy trust, because it's harder to bounce back from that than it was to build the trust in the first place.

Not only that, longer-term we're hoping to use the projects we manage as an opportunity for more junior developers to contribute and pair with more senior Gophers and to learn from that experience. While we're not set up for this yet, it's honestly something I wish I could spend more of my community time doing (pair programming).

This is different. Here you are asking that the GitHub project itself be transferred. That's something we could do (while leaving the copyright and licenses alone), but I am not even sure that this is appropriate.

I think it could be an agreeable solution to simply have the project moved, the license and copyrights for current code retained, and future contributions kept under the same licenses albeit copyrighted to a different organization. Retaining the history is absolutely the most important part for us.

I believe I crossed the streams in my head with the Google CLA and another, which is why I mentioned the copyright change. Apologies on that one.

Dep both helped us learn what package management in the go command might look like and also still serves as a useful tool until modules are completely ready.

In its current form the latter is becoming less true, and is what prompted this issue being raised. If you are a dep user, and one of your dependencies moves to modules and changes their import statements to use SIV, dep is not going to appreciate that since it cannot find the /vN/ folder in the repo.

Things like this are starting to become more common as adoption grows, and so we're hoping to address these sharp edges sooner rather than later.

I don't see why that wouldn't stay in github.com/golang. And once modules are ready, dep should go away. If others want to fork it and maintain a fork then that's fine. But the official dep that the Go project recommends should probably be the one in our GitHub org. If more people need commit access to github.com/golang/dep we can clearly do that.

Our goal here is to provide the best support/experience to the Go community and we're worried that having it alongside the official Go repositories could cause unnecessary confusion, especially if we intend to continue development of dep after modules are deemed "ready". When we hear things like "dep should go away", it makes it unclear what would happen to our work at that moment of "ready". It's the uncertainty that makes us uncomfortable, and we feel being in control of the project would help alleviate those concerns.

This isn't an ask to take our cookies and run away, but rather giving us the freedom and responsibility to support the community and to provide a solution to those who have needs outside of the scope of modules. Maybe by the time modules are ready we won't see a need, but it feels like a disservice to let it languish until then.

Sorry, I clicked the wrong "send" button; meant to leave this open for further discussion.

Heh. I do this all the time, I really wish they'd make it harder to do.

[1] https://www.youtube.com/watch?v=BB09FyaQN5I

Edit: I am traveling in Japan to see family; so with timezones and what-not I may be a bit delayed on responding. I'll definitely do my best to get back to any questions/concerns as soon as I can.

[therealplato]

I have no current interest in modules, they do not solve any problem that I currently encounter I have not encountered a problem that requires them. Dep works great and I have no reason to believe it will stop working. I would appreciate dep being placed in custodianship of those who care about continuing to use it.