backport: Find build-tool installed programs before programs in path

[alt-romes]

A backport of 443c890 (#9762)

[alt-romes]

@gbaz here it is

[geekosaur]

https://github.com/haskell/cabal/actions/runs/8116319454/job/22186060894?pr=9767#step:5:542

[alt-romes]

@geekosaur any chance you could drive this to completion? I don’t know when I will be able to get back to this.

[geekosaur]

I can try. My availability can be spotty due to sensory overload issues. (This is also why I usually have the camera off during calls; I can't handle the light for long.)

[geekosaur]

So somehow this breaks cabal-testsuite/PackageTests/PkgConfigParse, which can't find its replacement pkg-config any more. I'm lost as to why; the code seems correct to me.

[geekosaur]

Also, testing with a newer Cabal causes other test failures, which I presume are related to #9671 not being backported.

[Mikolaj]

@Kleidukos, @gbaz, IIRC you advocated for squeezing this PR into 3.10.3. Given that the fix is more troublesome than we expected, shall we pass in the interest of shipping 3.10.3 ASAP?

[gbaz]

After some investigation, I think we should close. if @andreasabel feels otherwise, please speak up. But there's something subtle going on that involves other changes as well, and I we could get more regressions if we port this back without figuring out exactly how everything interacts.

[andreasabel]

It is not a new regression, it is already there in 3.10.1, so it is not a blocker for 3.10.3. However, it would be good to fix it in 3.10 eventually. Maybe 3.10.4 then?

[gbaz]

alright, let me reopen. i'm just.. not sure exactly what's happening is all. and not sure how much we can dig in.

What's supposed to happen is we get [Default-Path, extra-lib-path], and that is indeed what's happening here. So in a sense the mystery is why it works on HEAD...

[gbaz]

Ah, so on HEAD the default path itself is also extended by extra-lib-dirs, which means that the extra-lib-dirs appear three times in the build -- twice from the inclusion of default-path before and after the explicit extra-lib-dirs and once in the middle. You might say "this is absurd" and it is, but this is sort of necessary because we need the Configure.hs file to work with v1-build as well, in which case we don't have any extensions to the default path.

So we need to backport whatever is necessary to fix this, as well...

[gbaz]

ok i think I figured it out. #8972 was never backported at all, and that contained the additional necessary fix I was looking for.

[Mikolaj]

@alt-romes, @geekosaur: could you have a look (ideally, review) now that Gershom cracked the nut?

[alt-romes]

Thanks @gbaz, this looks good. Glad you spotted it.

[alt-romes]

GH does not allow me to approve the pull request because I opened it

[geekosaur]

Strictly speaking, I probably shouldn't be reviewing because I worked on it.

[Kleidukos]

Looks like mergify didn't pick this one up. I am merging.