Azure DevOps Client

[chris5287]

Fixes #729
Fixes dependabot/feedback#128

• FileFetcher
• PullRequestCreator
  • Remove <details> as not supported (https://developercommunity.visualstudio.com/idea/609415/add-support-for-the-html-tag-in-markdown.html)
  • Investigate better method for truncating PR description to 4000 characters (https://developercommunity.visualstudio.com/content/idea/365708/raise-the-character-limit-for-pull-request-descrip.html)
• PullRequestUpdater
  • Not required
• Tests

[chris5287]

@greysteil @feelepxyz @hmarr is there an example script of how the PullRequestUpdater is used, so I know how to implement for Azure DevOps?

[colbywhite]

this pr is awesome! (minus the lack of specs 😄 )

i think the concept of updating a PR is github-specific, right? i don't think gitlab has that. thus i dont think azure needs the updater. and thus i think this is good to merge right now if this works.

Edit: FYI: i got this branch to create a pr for an ADO repo i have. great work!

[chris5287]

I'll attempt to write some tests later, but would appreciate feedback as this now works as a MVP

[greysteil]

i think the concept of updating a PR is github-specific, right? i don't think gitlab has that. thus I don't think azure needs the updater. and thus i think this is good to merge right now if this works.

Yes - updating a PR only happens from the hosted Dependabot app at the moment (I don't think anyone has written a script that uses the PullRequestUpdater). At the time when the classes were written it wasn't possible for GitLab because it wasn't possible to force push a commit through the GitLab API. I'd consider it non-essential to getting this added.

[greysteil]

And also, high level, this looks really nice!

[chris5287]

@greysteil @feelepxyz added tests for file fetching/pull request creating, hopefully this good to review/merge now 🙏

[chris5287]

@greysteil in case the lawyers need it:

I'm gifting the IP of the contents of this PR and the other contributions I authored to Dependabot for it to be used however Dependabot wishes.

[greysteil]

The lawyers need a full PDF signed these days! I'll get one over to you, and will review this today. (Was travelling on Monday and in meetings all yesterday).

[greysteil]

This is awesome. One miniscule comment and I'm 👍. I'll upload the PDF we need for the lawyers now.

[greysteil]

@chris5287 @colbywhite final step to get this merged is for you to sign these PDFs. I'll make it better soon... Sorry!

Dependabot IP Letter - Colby M. White.pdf
Dependabot IP Letter - Chris Stylianou.pdf

[chris5287]

Sent signed copy to support@dependabot.com. Might be worth looking into https://github.com/apps/cla-bot to make this painless for future contributors

[greysteil]

Received! @colbywhite just waiting on yours and I can get this merged.

Might be worth looking into https://github.com/apps/cla-bot to make this painless for future contributors

I could not agree more, but need to clear legal first 🙂

[chris5287]

@colbywhite are you able to sign as it’s blocking this from being merged :(

@greysteil if we get no response, are we able to merge anyway as @colbywhite contribution was the test class (see https://github.com/chris5287/dependabot-core/pull/1/files) or I could squash the commits and remove this file as an alternative?

[greysteil]

Yeah - if we refactor his contribution then we're able to merge. Let's get this in today either way!

[chris5287]

Ok I’ll refactor later this evening if @colbywhite doesn’t respond

[chris5287]

@greysteil I've squashed the commits and removed common/spec/dependabot/clients/azure_spec.rb. I've also fixed the merge conflict so should be good to merge

[Tosters]

Hi!
Great product you have created here! Exactly what we have been searching for in our organisation to help with dependency versioning. We are heavily using Azure DevOps (VSTS) with .NET and really excited for this merge. Unfortunately I can't seem to find any documentation on how to implement and start to use Dependabot with Azure DevOps.

[greysteil]

Thanks so much for this @chris5287! It's a really great addition to Dependabot Core.

If you're reading this wondering what it means for Azure DevOps, here's the latest:

• Dependabot Core now fully supports Azure DevOps. You can use this library to build your own dependency-updating robot (as long as you follow the license)
• Dependabot Script now fully supports Azure DevOps (since it uses this library)
• dependabot.com does not support Azure DevOps yet. It may do in future, but the work required is entirely on the (private) Dependabot Backend codebase and we have a lot of competing priorities there (e.g., completing the integration with GitHub)

Thanks again @chris5287!

[colbywhite]

@greysteil @chris5287, sorry for the delay on the CLA. i'll get one into you later today before i make a pr to add more tests to the client.

but great work getting this done.

[greysteil]

@colbywhite thanks for the signed PDF! I've cherry-picked your commit off of your fork and pulled it into master so you're officially a contributor (and the Azure client has specs).

[Tosters]

Hi,
Could anyone explain how to configure and run dependabot-script with Azure client?
I already spent almost a day setting up ubuntu, ruby etc. (Full .Net dev..) and can't get so that script updates Azure DevOps repo. I managed to run the script to update forked github nuget repository tho.

I added comments next to every property.
This could be useful to anyone who is trying out this too.

credentials << {
    "type" => "git_source", Leave as is?
    "host" => azure_hostname, **dev.azure.com or organization and project should be added also?**
    "username" => "x-access-token", Leave as is?
    "password" => ENV["AZURE_ACCESS_TOKEN"] **Created Key with full access in Azure DevOps in user security settings**
  }

  source = Dependabot::Source.new(
    provider: "azure", Leave as is?
    hostname: azure_hostname, dev.azure.com or organization and project should be added also?
    api_endpoint: "https://#{azure_hostname}/api/v4", I could not find any Azure DevOps API documentation with /api/v4
    repo: repo_name, Should this be only repo name or should contain organization/project names? Do I also add here the path to *.cproj?
    directory: directory, not sure what to put here
    branch: nil, also not sure what to put here
  )

[chris5287]

@Tosters heres the settings we use:

credentials << {
    "type" => "git_source",
    "host" => "dev.azure.com",
    "username" => "x-access-token",
    "password" => ENV["AZURE_ACCESS_TOKEN"]
  }

Dependabot::Source.new(
        provider: 'azure',
        repo: "#{@organisation}/#{project}/_git/#{repo}"
      )

directory will default to '/', its where dependabot should start looking for dependency files
branch will default to your default branch (ie: usually master)

[Tosters]

@chris5287 Thank you very much. It seems that it is working now. At least almost..
Now I am getting an error with HTML mentioning "Microsoft Internet Explorer's Enhanced Security Configuration". It seems that there is a problem with authorization..
Is personal access token with full access from Azure DevOps enough to authorize for requests? Or do I need to add symbols like ":" in front and encode to Base64?
Also I am running the script on Ubuntu that is hosted on windows 10 Hyper-V VM. (with github everything worked fine)
Thanks again for your help :)

[chris5287]

I just used the generated PAT from Azure DevOps with no problems (I limited mine to just code access but full access should work obviously). Confused why you are seeing IE ESC error, as I assume you are running a ruby script from the command line.

[acraven]

@chris5287 I'm seeing the same IE ESC error as @Tosters. I am using a full access token and running within a Docker container. I get the same error whether I use the token or some garbage.

[chris5287]

Any screenshots/logs you can share?

[acraven]

@chris5287 My mistake, I'm no Ruby expert, clearly. I had taken your snippet and mistakenly replaced << with = thinking it to be assign to existing variable rather than add to array.