go_modules doesn't update vendor folder when vendoring is enabled

[jerbob92]

Package manager/ecosystem
go_modules
Manifest contents prior to update
github.com/buger/jsonparser v1.0.0
Updated dependency
github.com/buger/jsonparser v1.1.1
What you expected to see, versus what you actually saw
The go.mod and go.sum get updated in the MR, but the vendor folder isn't, causing the MR to fail because the modules.txt in the vendor folder doesn't match the go.mod/go.sum files.

[jurre]

Are you using the GH native version of dependabot or dependabot-preview? I don't believe vendoring currently works in preview, and I would suggest updating to the GH native version. If this happens in the native version, could you link to the pull request in question (or contact support if this is happening in a private repo)

[jerbob92]

@jurre I'm using https://gitlab.com/dependabot-gitlab/dependabot
When looking at the docs and at the source code of dependabot-core it does have a vendor option

[jurre]

@jurre I'm using https://gitlab.com/dependabot-gitlab/dependabot
When looking at the docs and at the source code of dependabot-core it does have a vendor option

Ah, that is not a project that we maintain, so I'm afraid I can't help here. You should contact the author, if they run into things I'm happy to provide some guidance on how to deal with vendoring from a dependabot-core point of view.

[jerbob92]

@jurre That project includes dependabot-core directly, the only thing it does is making it interact with Gitlab.
It doesn't do anything to the working of the implementations of the package managers/ecosystems.

[jerbob92]

@jurre Sorry, my bad, it does look like it has some own config parsing logic... I'll open an issue over there.

Also, hello fellow Stadjer :)

[jurre]

@jurre Sorry, my bad, it does look like it has some own config parsing logic... I'll open an issue over there.

Also, hello fellow Stadjer :)

👋!

There needs to be some level of interop, dependabot-core by itself cannot open any pull requests, it needs to be orchestrated somehow, and that code will need to figure out when/how to deal with vendored code. I don't know the dependabot-gitlab codebase, so I can't point to exactly where the changes will need to be made. For the GitHub service, we check this, which returns true for Go modules, we can then figure out if vendoring is configured on the repo, and configure the dependabot-core classes to deal with it

[jerbob92]

Thanks for the extra information, I'll try to get this implemented correctly in dependabot-gitlab!
I think the main problem for now is not passing the vendor/tidy option.

[jerbob92]

Actually, while checking the code it looks like the current version of dependabot-core doesn't even use those options:
https://github.com/dependabot/dependabot-core/blob/main/go_modules/lib/dependabot/go_modules/file_updater.rb#L125

It checks the content of the repo to decide whether it should vendor or not, so I don't think it's a config option 🤔
Well, I'll try to figure it out in the other project first and will report back.

[jurre]

It checks the content of the repo to decide whether it should vendor or not, so I don't think it's a config option

Yep that's what I was trying to get at in my previous version. For go modules we use the always_clone_for_package_manager check to configure the FileFetcher to clone the repo content.

The code in dependabot-gitlab will have to set up a place for the repo to be cloned into, and pass that location on to other parts of the update process.

[jerbob92]

Yep, think I have found the problem:
https://gitlab.com/dependabot-gitlab/dependabot/-/blob/master/app/services/dependabot/file_fetcher.rb

Looks like it currently does not have the ability to clone the repo. I hoped this would have been an easy fix 😥

[jerbob92]

@jurre it looks like cloning is also required for bundler, but bundler isn't registered through register_always_clone, can you explain why that is?

[jurre]

@jurre it looks like cloning is also required for bundler, but bundler isn't registered through register_always_clone, can you explain why that is?

Bundler currently doesn't always clone, it only clones when vendoring is configured

[jerbob92]

Thanks :) I think we can replicate that behavior then.