Add AllowWorkDirectoryRepositories knob

[aleksandrlevochkin]

Description

Added AllowWorkDirectoryRepositories knob that allows to set a relative path to the _work directory during checkout.

Related issues: #1506, #4192

Based on this PR by @echalone #3475

Testing

• Added unit tests
• Tested locally with different values for AZP_AGENT_ALLOW_WORK_DIRECTORY_REPOSITORIES variable

[PaulVrugt]

It is awesome that this is being added.

• Will scaleset agent pools be able to use this?
• it would be even more awesome of we can change the default base path for checkouts. Otherwise will we have to update all the checkout steps of all our pipelines in our repo (which we have a lot), and make sure that every time we add a new pipeline we don't forget to set the checkout path

[aleksandrlevochkin]

@PaulVrugt

1. Yes, I think it should be working with scaleset agent pools
2. Thanks for your suggestion. We might consider this feature for the future, but not in the scope of this PR

[echalone]

[NiuBClass666]

I see this feature got merged into master. May I ask if it meant this will be released soon?

[echalone]

I see this feature got merged into master. May I ask if it meant this will be released soon?

I don't know for sure of course, but since it got merged into master right after they created the 3.229 release branch I guess it will come with release 3.231 around end of November or in December.

[echalone]

By the way, I've already compiled the Windows x64 agent with this feature (commit a3067b7 from master merger) myself and we're using it for our tests with the newly structure pipelines that use multiple centralized repositories. Everything works as expected. There's the bug with the wrong detection of the primary repository if the self repository isn't checked out first of course (see #4217), but that's not urgent since all you need to do to fix it is check out the self repository first. There's also the fun behaviour of Build.SourcesDirectory which will always point to \s in a multi-checkout scenario and System.DefaultWorkingDirectory which will always point to the same directory as Build.SourcesDirectory. So people have to keep that in mind, but we've already adapted the documentation accordingly a few years ago, so the behaviour is clearly documented. There's of course my feature request and pull request for the adaptable working directory by just choosing one of the repositories as the working directory (#3479), but that would need to be added to the yaml definition of the Azure DevOps Server as well. However, as long as you know where you've checked out to and just use that exact path when calling scripts or similar that shouldn't be a problem either. The only problem I encountered is with the SonarQube extension which just assumes the System.DefaultWorkingDirectory as the projectBaseDir when the MSBuild scanner is used, but that's their problem really, since they should have made that adaptable.

tl;dr: everything works as expected and as documented

[mjthurlkill]

Glad to see there has been some discussion around this issue and some movement.
I just ran into this issue a couple days ago with a new little pipeline, so I'll go through the comments and see if there is a resolution for my issue.

In the latest case, it is very simple 3 step pipeline using a single repo. The .yml is currently just in my private branch.
I push the initial changes to the yml, run the pipeline, ADO sends it to an on-prem agent, it clones the full 30gb git repo.
I push some other small changes to the yml, run the pipeline, ADO sends it to the same agent, it again clones the full 30gb (instead of just pulling the recent changes).

Of course, pulling the changes in git in another copy of the repo on one of my machines pulls down just the change to the yml very quickly. So appears to be an issue with ADO keeping track of which repo has been used in which workdir rather than a git issue. Though it does seem to keep track to a certain degree. When running the same job multiple times, ADO does select the same agent for the job each time unless there is alot of activity and the agent gets assigned a different job before this job is run again.

Anyway, I'll check the comments and see if there are new settings or clues for how to address this.

[mjthurlkill]

Part of the problem I had above was a result of a recent change to ADO where, for new pipelines, it uses shallow by default.
Shallow seems like more trouble than its worth. I would start transitioning to partial clones.