Windows Container: fs.realpathSync() is broken on shared volumes

[StefanScherer]

• Version: 4.6.1, 6.9.1, 7.0.0
• Platform: Windows 10 1607 or Windows Server 2016 with Docker + Windows Containers
• Subsystem: fs

The function fs.realpathSync() and/or pathModule._makeLong() is broken using it on shared Docker volumes mapped from host into a Windows Container. It seems to interpret the junction point incorrectly which is an UNC path and interprets it as c:\ContainerMappedDirectories.

Steps to reproduce

You need Windows 10 Anniversary update with Containers feature and Docker installed. Or an Windows Server 2016 with Containers feature and Docker installed, eg. an Azure VM.

Create a test directory with a minimal Dockerfile to just put node.exe into a Windows container.

PS C:\> mkdir test
PS C:\> cd test
PS C:\test> notepad Dockerfile.

This is the content of the Dockerfile

FROM microsoft/nanoserver

ARG NODE_VERSION=4.6.1

RUN powershell -Command \
    Invoke-WebRequest https://nodejs.org/dist/v$env:NODE_VERSION/win-x64/node.exe \
      -OutFile node.exe -UseBasicParsing

CMD [ "node.exe" ]

Now build a Docker image eg. for Node.js 4.6.1 using a build argument:

PS C:\test> docker build -t nodeexe:4.6.1 --build-arg NODE_VERSION=4.6.1 .

We can check that the binary works inside the Windows container by showing the version

PS C:\test> docker run nodeexe:4.6.1 node --version
v4.6.1

Now check that fs.realpathSync() works with an existing directory inside the container.

PS C:\test> docker run nodeexe:4.6.1 node -p "const fs=require('fs'); fs.realpathSync('c:/windows')"
c:\windows

OK. Now we mount the current directory as shared volume into the container and have a look at it with the dir command:

PS C:\test> docker run -v "$(pwd):C:\test" nodeexe:4.6.1 cmd /c dir c:\
 Volume in drive C has no label.
 Volume Serial Number is B097-15C5

 Directory of c:\

10/09/2016  10:57 PM             1,894 License.txt
11/04/2016  12:50 PM        14,259,744 node.exe
07/16/2016  05:20 AM    <DIR>          Program Files
07/16/2016  05:09 AM    <DIR>          Program Files (x86)
11/04/2016  01:43 PM    <SYMLINKD>     test [\\?\ContainerMappedDirectories\6C4D3D41-7E32-4992-BC03-44E1FAC23C0B]
10/09/2016  10:58 PM    <DIR>          Users
11/04/2016  12:50 PM    <DIR>          Windows
               2 File(s)     14,261,638 bytes
               5 Dir(s)  21,210,304,512 bytes free

OK, the directory C:\test is there, but you can see that it is a symlink to an UNC path. Let's check that this directory is accessible.

PS C:\test> docker run -v "$(pwd):C:\test" nodeexe:4.6.1 cmd /c dir c:\test
 Volume in drive C has no label.
 Volume Serial Number is B097-15C5

 Directory of c:\test

11/04/2016  01:44 PM    <DIR>          .
11/04/2016  01:44 PM    <DIR>          ..
11/04/2016  01:46 PM               231 Dockerfile
               1 File(s)            231 bytes
               2 Dir(s)  21,837,713,408 bytes free

OK, we can see the Dockerfile from the host. So let's now check fs.realpathSync with that directory.

PS C:\test> docker run -v "$(pwd):C:\test" nodeexe:4.6.1 node -p "const fs=require('fs'); fs.realpathSync('c:/test')"
fs.js:839
  return binding.lstat(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, lstat 'c:\ContainerMappedDirectories'
    at Error (native)
    at Object.fs.lstatSync (fs.js:839:18)
    at Object.realpathSync (fs.js:1439:21)
    at [eval]:1:28
    at Object.exports.runInThisContext (vm.js:54:17)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:409:26)
    at node.js:579:27
    at nextTickCallbackWith0Args (node.js:420:9)
    at process._tickCallback (node.js:349:13)

Boom! As you can see the UNC path is misinterpreted as c:\ContainerMappedDirectories and then of course this directory does not exist inside the Windows container.

This error breaks eg. using npm install on such a shared volume or even running simple Node.js code from it.

Let's create a small hello.js, map the current folder into a container and try to run it.

PS C:\test> "console.log('hello');" | Out-File hello.js -Encoding Ascii
PS C:\test> docker run -v "$(pwd):C:\test" nodeexe:4.6.1 node c:\test\hello.js
fs.js:839
  return binding.lstat(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, lstat 'c:\ContainerMappedDirectories'
    at Error (native)
    at Object.fs.lstatSync (fs.js:839:18)
    at Object.realpathSync (fs.js:1439:21)
    at toRealPath (module.js:112:13)
    at Function.Module._findPath (module.js:151:20)
    at Function.Module._resolveFilename (module.js:323:25)
    at Function.Module._load (module.js:276:25)
    at Function.Module.runMain (module.js:441:10)
    at startup (node.js:139:18)
    at node.js:974:3

When we copy the file from the shared volume into a local directory of the Windows container Node.exe is able to run it:

PS C:\test> docker run -v "$(pwd):C:\test" nodeexe:4.6.1 cmd "/c copy c:\test\hello.js . & node hello.js"
        1 file(s) copied.
hello

Another problem of this bug is that running npm install also does not work in a shared volume.

[bnoordhuis]

Does node --preserve-symlinks c:\nodejs\node_modules\npm\bin\npm-cli.js install work?

[StefanScherer]

No, still the same. Can't see the --preserve-symlink in the argv output:

C:\code>node --preserve-symlinks c:\nodejs\node_modules\npm\bin\npm-cli.js install
npm info it worked if it ends with ok
npm info using npm@3.10.3
npm info using node@v6.7.0
npm ERR! Windows_NT 10.0.14393
npm ERR! argv "C:\\nodejs\\node.exe" "c:\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
npm ERR! node v6.7.0
npm ERR! npm  v3.10.3
npm ERR! path C:\ContainerMappedDirectories
npm ERR! code ENOENT
npm ERR! errno -4058
npm ERR! syscall lstat

npm ERR! enoent ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
npm ERR! enoent ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! Please include the following file with any support request:
npm ERR!     C:\code\npm-debug.log

C:\code>type npm-debug.log
0 info it worked if it ends with ok
1 verbose cli [ 'C:\\nodejs\\node.exe',
1 verbose cli   'c:\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
1 verbose cli   'install' ]
2 info using npm@3.10.3
3 info using node@v6.7.0
4 silly loadCurrentTree Starting
5 silly install loadCurrentTree
6 silly install readLocalPackageData
7 silly rollbackFailedOptional Starting
8 silly rollbackFailedOptional Finishing
9 silly runTopLevelLifecycles Starting
10 silly runTopLevelLifecycles Finishing
11 silly install printInstalled
12 verbose stack Error: ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
12 verbose stack     at Error (native)
13 verbose cwd C:\code
14 error Windows_NT 10.0.14393
15 error argv "C:\\nodejs\\node.exe" "c:\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install"
16 error node v6.7.0
17 error npm  v3.10.3
18 error path C:\ContainerMappedDirectories
19 error code ENOENT
20 error errno -4058
21 error syscall lstat
22 error enoent ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
23 error enoent ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
23 error enoent This is most likely not a problem with npm itself
23 error enoent and is related to npm not being able to find a file.
24 verbose exit [ -4058, true ]

[StefanScherer]

It seems that node can't find/load js files in the volume mount point.

C:\code>type test.js
console.log('hello');

C:\code>node
> process.cwd()
'C:\\code'
>
(To exit, press ^C again or type .exit)
>

C:\code>node test.js
fs.js:982
  return binding.lstat(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
    at Error (native)
    at Object.fs.lstatSync (fs.js:982:18)
    at Object.realpathSync (fs.js:1647:19)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:453:25)
    at Function.Module._load (module.js:403:25)
    at Module.runMain (module.js:590:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

C:\code>node c:\code\test.js
fs.js:982
  return binding.lstat(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, lstat 'c:\ContainerMappedDirectories'
    at Error (native)
    at Object.fs.lstatSync (fs.js:982:18)
    at Object.realpathSync (fs.js:1647:19)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:453:25)
    at Function.Module._load (module.js:403:25)
    at Module.runMain (module.js:590:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

C:\code>node --preserve-symlinks test.js
fs.js:982
  return binding.lstat(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, lstat 'C:\ContainerMappedDirectories'
    at Error (native)
    at Object.fs.lstatSync (fs.js:982:18)
    at Object.realpathSync (fs.js:1647:19)
    at Function.Module._findPath (module.js:167:25)
    at Function.Module._resolveFilename (module.js:453:25)
    at Function.Module._load (module.js:403:25)
    at Module.runMain (module.js:590:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

[StefanScherer]

Just to clarify, if someone else tries this and wants a temporary solution. This only occurs with Node.js 6.x and works fine with Node.js 4.6.0

docker run -it --rm --name my-app -v "${pwd}:C:\App" -w "C:\App" -p 3000:3000 -p 9000:9000 stefanscherer/node-windows:4.6.0 powershell

[StefanScherer]

This also affects opening files, eg. TLS certs mapped into a container through a volume. How can we proceed getting this fixed in LTS version of Node.js 6?

[bnoordhuis]

How sure are you it's a node issue and not a docker-on-windows issue? Does the same node binary work outside of the container?

[StefanScherer]

@bnoordhuis I'm sure this is a Node.js problem. I've stripped down the problem as in #7044 to fs.realpathSync()

An internal directory works:

PS C:\> docker run -v "$(pwd):c:\code" stefanscherer/node-windows:7 node -p "fs.realpathSync('c:\windows')"
c:\windows

But a volume directory throws an exception:

PS C:\> docker run -v "$(pwd):c:\code" stefanscherer/node-windows:7 node -p "fs.realpathSync('c:\code')"
fs.js:893
  return binding.lstat(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, lstat 'c:\ContainerMappedDirectories'
    at Object.fs.lstatSync (fs.js:893:18)
    at Object.realpathSync (fs.js:1527:21)
    at [eval]:1:4
    at ContextifyScript.Script.runInThisContext (vm.js:25:33)
    at Object.exports.runInThisContext (vm.js:77:17)
    at Object.<anonymous> ([eval]-wrapper:6:22)
    at Module._compile (module.js:573:32)
    at bootstrap_node.js:345:29
    at _combinedTickCallback (internal/process/next_tick.js:67:7)
    at process._tickCallback (internal/process/next_tick.js:98:9)

Tested with both Node.js 6.9.1 and 7.0.0.

[bnoordhuis]

cc @nodejs/platform-windows @nodejs/docker