Merge error (Bitbucket) when using Git 2.27 - works with Git 2.24

[patrick-compass]

• I was not able to find an open or closed issue matching what I'm seeing

Setup

• Which version of Git for Windows are you using? Is it 32-bit or 64-bit?

$ git --version --build-options

**Working version**
git version 2.24.0.windows.2
cpu: x86_64
built from commit: 02af2cc5bfc37d4500e8a09333269e57f442198a
sizeof-long: 4
sizeof-size_t: 8

**Failing version**
version 2.27 64-bit
can't install due to live server

• Which version of Windows are you running? Vista, 7, 8, 10? Is it 32-bit or 64-bit?

$ cmd.exe /c ver

Microsoft Windows [Version 10.0.14393]

• What options did you set as part of the installation? Or did you choose the
  defaults?

# One of the following:
> type "C:\Program Files\Git\etc\install-options.txt"
> type "C:\Program Files (x86)\Git\etc\install-options.txt"
> type "%USERPROFILE%\AppData\Local\Programs\Git\etc\install-options.txt"
$ cat /etc/install-options.txt

Editor Option: VIM
Custom Editor Path: 
Path Option: Cmd
SSH Option: OpenSSH
Tortoise Option: false
CURL Option: OpenSSL
CRLF Option: CRLFAlways
Bash Terminal Option: MinTTY
Performance Tweaks FSCache: Enabled
Use Credential Manager: Enabled
Enable Symlinks: Disabled
Enable Builtin Interactive Add: Disabled

• Any other interesting things about your environment that might be related
  to the issue you're seeing?

Manifested in Bitbucket pull requests. Error trying to display the merge diff. Attempt to merge produced this error.

Note: The error is definitely concerned with the repo containing a file named AUX which is invalid in Windows

Details

• Which terminal/shell are you running Git from? e.g Bash/CMD/PowerShell/other

Testing with Bash, errors occuring within Bitbucket

• What commands did you run to trigger this issue? If you can provide a
  Minimal, Complete, and Verifiable example
  this will help us understand the issue.

I'm not sure what commands Bitbucket runs trying to render a pull request.
Error above shows failing command when attempting to merge anyway.

• What did you expect to occur after running these commands?

When using 2.24 the pull request renders correctly and we're able to merge.

• What actually happened instead?

When using 2.27 the pull request shows an error and trying to merge produces the error inserted above related to the invalid Windows filename "AUX"

• If the problem was occurring with a specific repository, can you provide the
  URL to that repository to help us with testing?

working on private on-premise copy of linux kernel
specifically files like https://github.com/torvalds/linux/blob/master/drivers/gpu/drm/nouveau/nvkm/subdev/i2c/aux.c

[derrickstolee]

Unfortunately, the Linux kernel repo uses paths that are not available on Windows. ("aux.*" is forbidden due to DOS compatibility. Let's...not get into that too much.)

The issue you are having is related to the security work introduced in v2.24.1, I believe. In order to work with such paths you will need to disable the core.protectNTFS setting. BE ADVISED THAT THERE ARE SECURITY IMPLICATIONS TO DISABLING THIS SETTING. Only apply this setting to repositories where it is required, never globally. Further, only fetch from trusted remotes.

Even worse, you might have problems when these paths are trying to be written. When I use the Linux kernel repo on Windows, I write the following into .git/info/sparse-checkout:

!*/aux\.*

then run git sparse-checkout init followed by git reset --hard to make sure everything is in a good state.

[patrick-compass]

@derrickstolee Agreed, let's not talk about super-legacy device files...

I understand how to apply those settings on my workstation, but any idea how to force Bitbucket to do the same?

[derrickstolee]

I understand how to apply those settings on my workstation, but any idea how to force Bitbucket to do the same?

If this is an on-prem server, then maybe you can do the same -- assuming you can find the repo on-disk somewhere. You'll need to reach out to BitBucket support to be sure you don't break anything.

The only person at Atlassian I know is Bryan Turner, who is probably @bturner here on GitHub. Maybe they can help?

[patrick-compass]

We're also going to be doing a Bitbucket upgrade soon. Hopefully this was addressed on Atlassian's side.

[bturner]

Hello all. Thanks for the mention, @derrickstolee.

"Addressed" is a difficult thing, when it comes to this change. We're aware of it, yes, and I invested a significant amount of time into experimenting with different options. But we're pretty much stuck between a rock and a hard place on it:

• If we "just" disable core.protectNTFS everywhere when the system is running on Windows, then we're actively defeating security enhancements that have been made in response to vulnerabilities
• If we "just" leave it enabled... Well, you know what happens because you're living it
• There are other possibilities, of course, and they come with their own pitfalls

Prior to the way the restriction was enforced changing in Git for Windows 2.25 (pull request here), we were investigating approaches for disabling core.protectNTFS because the 2.24.1 restriction was too draconian; being unable to push a repository to a server hosted on Windows simply because it had ever contained a restricted path was too restrictive. With that change applied, however, we decided to stick with the defaults.

If you're upgrading to Bitbucket Server 7.x (any version), then you'll also pick up the change that replaces 3-way diffs (merge-based diffs) for pull requests with 2-way diffs (common ancestor-based diffs). At that point the series of operations that are resulting in the failure you're seeing will no longer run--except when actually merging a pull request. If the merge for the diff was failing, chances are the "real" merge will fail too. Based on that, I suspect no upgrade will (presently) help with this.

Can you confirm, does the pull request in question actually modify a restricted path? Probably the best way to proceed with this issue is to create a support request so our supporters (and likely myself) can work with you directly to gather details about your situation and help you resolve it.

(Lastly, I'll note that if you're still running a version of Bitbucket Server that uses 3-way diffs for pull requests, it means you must be on 6.10 or older. There are no 6.x releases which support running Git 2.27 on the server; 2.24 is the latest supported version. In general you likely won't have problems, since Git is generally highly backward-compatible, but you may. We're constantly making small (and sometimes large or very large) tweaks to Bitbucket Server to address changes in newer Git versions, so upgrading the server with a version it doesn't support can cause problems.)

Best regards,
Bryan Turner
Atlassian Bitbucket

[patrick-compass]

I'll report back in after our upgrade.
Thanks Bryan

[patrick-compass]

We updated to Bitbucket 7.3.1 which was supposed to support git 2.27, however we've run into the same issue as before.

Is there something I can configure on my server to allow this to work?

[dscho]

Unfortunately, the Linux kernel repo uses paths that are not available on Windows. ("aux.*" is forbidden due to DOS compatibility. Let's...not get into that too much.)

The issue you are having is related to the security work introduced in v2.24.1, I believe. In order to work with such paths you will need to disable the core.protectNTFS setting. BE ADVISED THAT THERE ARE SECURITY IMPLICATIONS TO DISABLING THIS SETTING. Only apply this setting to repositories where it is required, never globally. Further, only fetch from trusted remotes.

Even worse, you might have problems when these paths are trying to be written. When I use the Linux kernel repo on Windows, I write the following into .git/info/sparse-checkout:

!*/aux\.*

then run git sparse-checkout init followed by git reset --hard to make sure everything is in a good state.

I think that this advice still holds.

[patrick-compass]

Unfortunately, the Linux kernel repo uses paths that are not available on Windows. ("aux.*" is forbidden due to DOS compatibility. Let's...not get into that too much.)
The issue you are having is related to the security work introduced in v2.24.1, I believe. In order to work with such paths you will need to disable the core.protectNTFS setting. BE ADVISED THAT THERE ARE SECURITY IMPLICATIONS TO DISABLING THIS SETTING. Only apply this setting to repositories where it is required, never globally. Further, only fetch from trusted remotes.
Even worse, you might have problems when these paths are trying to be written. When I use the Linux kernel repo on Windows, I write the following into .git/info/sparse-checkout:

!*/aux\.*

then run git sparse-checkout init followed by git reset --hard to make sure everything is in a good state.

I think that this advice still holds.

If it still applies, how do I configure this on the server? As far as I know, .gitconfig files are for the client side of the equation.

[patrick-compass]

Can you confirm, does the pull request in question actually modify a restricted path? Probably the best way to proceed with this issue is to create a support request so our supporters (and likely myself) can work with you directly to gather details about your situation and help you resolve it.

@bturner No, the PR in question doesn't even try to modify this file.

[mfriedrich74]

Hi, As far as I understand it. Bitbucket runs here on windows. Whether a PR changes the file or not does not matter. The server will check out the repo to perform a merge even before hitting the merge button. This happens to support automatic build systems that have to run on the merge result and probably also for the PR web page and code review functionality. Best regards, Mike

…

On Tue, Jun 30, 2020, 1:04 PM Patrick Little ***@***.***> wrote: Can you confirm, does the pull request in question actually modify a restricted path? Probably the best way to proceed with this issue is to create a support request <https://support.atlassian.com/contact> so our supporters (and likely myself) can work with you directly to gather details about your situation and help you resolve it. No, the PR in question doesn't even try to modify this file. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2682 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZH5SFEEEUWNRBFPB2G2QDRZILLVANCNFSM4N2JJHHA> .

[mfriedrich74]

Please also note that you will not be able to inject custom commands like git sparse-checkout init in the server. As far as I know you cannot enable sparse checkouts anywhere on the server. I recommend you contact Atlassian support and also seriously consider switching the server to Linux. This will not affect your Windows users apart from enjoying a more stable server. But let Atlassian tell you this. On Tue, Jun 30, 2020, 1:12 PM Mike Friedrich <mike.friedrich@gmail.com> wrote:

…

Hi, As far as I understand it. Bitbucket runs here on windows. Whether a PR changes the file or not does not matter. The server will check out the repo to perform a merge even before hitting the merge button. This happens to support automatic build systems that have to run on the merge result and probably also for the PR web page and code review functionality. Best regards, Mike On Tue, Jun 30, 2020, 1:04 PM Patrick Little ***@***.***> wrote: > Can you confirm, does the pull request in question actually modify a > restricted path? Probably the best way to proceed with this issue is to create > a support request <https://support.atlassian.com/contact> so our > supporters (and likely myself) can work with you directly to gather details > about your situation and help you resolve it. > > No, the PR in question doesn't even try to modify this file. > > — > You are receiving this because you are subscribed to this thread. > Reply to this email directly, view it on GitHub > <#2682 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABZH5SFEEEUWNRBFPB2G2QDRZILLVANCNFSM4N2JJHHA> > . >

[bturner]

@lpd-patrick,

I'm not too surprised a Bitbucket Server upgrade didn't help. I tried to note up front that it wouldn't: "If the merge for the diff was failing, chances are the "real" merge will fail too. Based on that, I suspect no upgrade will (presently) help with this." 7.3.1 will support Git 2.27, but this issue isn't really about supporting Git 2.27; it's about a restriction that was added in Git 2.24.1 that, as I tried to explain above, we're basically stuck between a rock and a hard place on how to handle. Given we don't have any "perfect" options we've elected to go with what the Git maintainers have chosen and leave the protections enabled.

At this point, my suggestion would be, as @mfriedrich74 suggested, that you open a support case with Atlassian. There are a few different options we can investigate. The "easiest" would be to simply downgrade Git on your server. Of course, you lose all the bugfixes and other improvements that come with newer versions, but it also "disables" the core.protectNTFS checks. Of course, there's functionally no difference between that approach and simply adding core.protectNTFS=false to your global .gitconfig for the user your Bitbucket Server install runs as. As @derrickstolee and @dscho indicated, though, that has some security implications; whether those are acceptable to you and your team is for you to decide.

There are other options we can investigate that would make the workaround a bit more targeted, but a support case with Atlassian seems like the correct place to do that, rather than here on the Git for Windows issue board. I don't think there's a lot the Git for Windows developers can do to "fix" this; I'd imagine they're in a similar bind to us, in that having the safeguards on by default protects more users at the cost of inconveniencing some and having them off by default is more convenient but protects no one.

Best regards,
Bryan Turner
Atlassian Bitbucket

[patrick-compass]

@bturner Thanks for checking back in. I'm not surprised either and I did open a support ticket this morning.

[bturner]

Thanks @lpd-patrick. If you mentioned this issue on the support case, I'm sure I'll get looped in on the Atlassian side to help with things.