EPERM when renaming files on Windows

[achingbrain]

• Version: 10.16.3
• Platform: Windows 10
• Subsystem: fs

The docs for fs.rename(oldPath, newPath, callback) say:

In the case that newPath already exists, it will be overwritten.

This happens on Linux/Mac OS X but doesn't appear to happen with Windows. If you run the following script:

const { tmpdir } = require('os')
const { join } = require('path')
const { writeFile, rename } = require('fs')

const dest = join(tmpdir(), 'dest.txt')
const file = Buffer.allocUnsafe(1024 * 1024) // 1MB
const count = 100

for (let i = 0; i < count; i++) {
  writeFile(`${dest}-${i}.tmp`, file, (err) => {
    if (err) {
      console.error('Could not write')
      throw err
    }

    rename(`${dest}-${i}.tmp`, dest, (err) => {
      if (err) {
        console.error('Could not rename')
        throw err
      }
    })
  })
}

You will see:

C:\Users\me> node test.js
Could not rename
C:\Users\me\test.js:21
        throw err
        ^

Error: EPERM: operation not permitted, rename 'C:\Users\me\AppData\Local\Temp\dest.txt-0.tmp' -> 'C:\Users\me\AppData\Local\Temp\dest.txt'

The same script runs to completion on Linux & Mac OS X without an error.

Setting count to a low value (e.g. 2) doesn't trigger the bug for me, so if it works for you on Windows, your computer is likely faster than mine - try setting it to something unreasonable like 1000.

This could be what was happening in #6335 and appears at least tangentially related to #21957

[bnoordhuis]

See discussion in libuv/libuv#1981 - which was subsequently reverted in response to libuv/libuv#2098.

Long story short: this is not something that's really under Node's or libuv's control and the best we could do is try to work around it by means of a grace period.

That's what modules like graceful-fs do, it arguably has no place in Node.js itself.

[vtjnash]

I noticed that Windows 10 recently added some features that perhaps might help here. In particular, FILE_RENAME_POSIX_SEMANTICS and FILE_RENAME_POSIX_SEMANTICS. Posting here (instead of libuv) to get more eyeballs on this and to ask whether these are flags that libuv should try to use.