Create temp files in temp directory

[jasagredo]

TL;DR

This change ensures all temporal files are created in the system temp directory which usually is in a short path. This helps with Windows not being capable of creating temp files in long directories, like the ones that result from Backpack.

Description

To be precise, purely temporary files (ghc intermediate files, rsps, etc) were already created in this system temporary directory, because the code called withTempFile always with the result from getTemporaryDirectory. I just made it so that the user doesn't even have the option to provide a different directory now.

And the other case is the writeFileAtomic function which used to create temporary files already in the destination directory and then rename them. If the destination directory was a long path, this would fail on Windows. This function now creates the temp file in the temp directory too.

Note that the "system temporary directory" is what getTemporaryDirectory returns which can be controlled via environment variables as $TMPDIR. See getTemporaryDirectory haddocks.

Reference

See how GetTempFileNameW specifies:

The string cannot be longer than MAX_PATH–14 characters or GetTempFileName
will fail.

And actually there is a TODO in Win32Utils.c in GHC:

https://gitlab.haskell.org/ghc/ghc/-/blob/3939a8bf93e27d8151aa1d92bf3ce10bbbc96a72/libraries/ghc-internal/cbits/Win32Utils.c#L259

Therefore Windows cannot create temporary files in long paths. To be precise, it can do so if using --io-manager=native but this is not yet the default and it seems reasonable to not make people face other complications that might arise from using that always, even more if it is as easy as creating the temp files somewhere else.

---

Closes #10191.

Template Α: This PR modifies behaviour or interface

Include the following checklist in your PR:

• Patches conform to the coding conventions.
• Any changes that could be relevant to users have been recorded in the changelog.
• The documentation has been updated, if necessary.
• Manual QA notes have been included.
• Tests have been added. No new tests are needed as most of them already create temp files

[geekosaur]

@9999years, do the API changes here affect #10292?

[geekosaur]

(API changes: not backporting to 3.12.)

[Mikolaj]

Thanks a lot!

[9999years]

@geekosaur I'm not sure, honestly. I can check later. They might make it harder to test stuff like in #10292 (comment)?

[jasagredo]

I think the only change is that one would have to look in the global tmp directory instead of in the destination directory.

[mpickering]

I think the only change is that one would have to look in the global tmp directory instead of in the destination directory.

There might be anything in the global tmp directory, I think tests should run with a specific TMPDIR.

[jasagredo]

I find it reasonable that tests run with a dedicated TMPDIR, but is it ok that in general cabal uses the global temp dir instead of the destination path?

[geekosaur]

People expect to be able to control where cabal puts its files with $TMPDIR; you can search for TMPDIR in issues for examples.

[9999years]

People expect to be able to control where cabal puts its files with $TMPDIR; you can search for TMPDIR in issues for examples.

Oh yeah, I should've mentioned that we use this at Mercury earlier.

[jasagredo]

On Linux it will put temporary files in TEMPDIR. On Windows it will put temp files in TMP or TEMP. Linux users should be unaffected by temporary files (they were all created there already).

The only change is for writing files atomically via creating a temp file then renaming it to the proper filename. That used to happen in the destination directory, after this PR it will happen in TEMPDIR/TMP/TEMP.

[jasagredo]

I added some more docs to writeFileAtomic, also reworked the test, and extensively reworded the description of the PR. I hope it is more clear now. @9999years @mpickering @geekosaur

[geekosaur]

It occurs to me that you should be able to put them in the current directory, because all tests are copied to a temp directory before running (see #9717).

[9999years]

The only change is for writing files atomically via creating a temp file then renaming it to the proper filename. That used to happen in the destination directory, after this PR it will happen in TEMPDIR/TMP/TEMP.

This seems to not be true, and we can see some test failures following #10392 where the response files are now placed in a different directory:

Running: /usr/local/.ghcup/ghc/9.6.6/bin/haddock-ghc-9.6.6 '@/tmp/haddock-response328591-4.txt'
...
*** Exception: Expected glob "**/haddock-response*.txt"