Caching from a read-only base folder is impossible due to safe_copy copying the permissions bits

[RaitoBezarius]

I'm using Nix to install emscripten (2.0.27) and the folder where it is installed is read-only (and immutable), therefore, whenever I try to use the cache setup to ~/.cache/emscripten, it fails as: safe_copy uses shutil.copy rather than shutil.copyfile.

The issue is rather similar to mkdocs/mkdocs#1292 ; making it impossible to run embuilder build SYSTEM and compiles something.

Would that be possible to use shutil.copyfile or ensure that copied files have +w when caching is performed?

Thanks!

[sbc100]

So you are using a writable location (~/.cache/emscripten) for your cache by setting EM_CACHE (or CACHE in the config file).. but you are still getting some kind of permission denied error?

Can you explain a little more? What file are not getting the correct +w permission exactly and what is the resulting error? Can you show the actual error message you get?

[sbc100]

Normally embuilder installs file that the user just built with the compiler (e.g. libc.a) so they should be writable? Perhaps the issue is with the system headers installed with install_system_headers? Even so I would be surprised if this generated an error the first time it was run, but maybe it generates an error when run a second time and the files are overridden? Is that what is happening?

[RaitoBezarius]

So you are using a writable location (~/.cache/emscripten) for your cache by setting EM_CACHE (or CACHE in the config file).. but you are still getting some kind of permission denied error?

Can you explain a little more? What file are not getting the correct +w permission exactly and what is the resulting error? Can you show the actual error message you get?

Yes, but it's not the writable location that matters. As the copy mechanism is copying the permission bits, the source files are read-only, therefore, the destination files are also read-only.

So basically, all files lose their +w permission each time they get recopied, and then, it becomes impossible to update them.

Here's the actual error message:

  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 308, in get_path
    return shared.Cache.get_lib(self.get_filename(), self.build)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/cache.py", line 137, in get_lib
    return self.get(name, *args, **kwargs)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/cache.py", line 165, in get
    creator(cachename)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 388, in build
    create_lib(out_filename, self.build_objects(build_dir))
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 381, in build_objects
    run_build_commands(commands)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 88, in run_build_commands
    ensure_sysroot()
  File "/nix/store/i1m8r7mv8h47wr850cdsxksy22lv6gsz-python3-3.9.6/lib/python3.9/contextlib.py", line 79, in inner
    return func(*args, **kwds)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 2032, in ensure_sysroot
    shared.Cache.get('sysroot_install.stamp', install_system_headers, what='system headers')
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/cache.py", line 165, in get
    creator(cachename)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhv
k-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 2011, in install_system_headers
    copytree_exist_ok(src, dest)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 1987, in copytree_exist_ok
    copytree_exist_ok(srcname, dstname)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/system_libs.py", line 1989, in copytree_exist_ok
    shared.safe_copy(srcname, dstname)
  File "/nix/store/9fycg3wb2i4fbyrwx9890wcz96vywhvk-emscripten-2.0.27/share/emscripten/tools/shared.py", line 765, in safe_copy
    shutil.copy(src, dst)
  File "/nix/store/i1m8r7mv8h47wr850cdsxksy22lv6gsz-python3-3.9.6/lib/python3.9/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/nix/store/i1m8r7mv8h47wr850cdsxksy22lv6gsz-python3-3.9.6/lib/python3.9/shutil.py", line 264, in copyfile
    with open(src, 'rb') as fsrc, open(dst, 'wb') as fdst:
PermissionError: [Errno 13] Permission denied: '/home/raito/.cache/emscripten/sysroot/include/bits/errno.h'

[RaitoBezarius]

Normally embuilder installs file that the user just built with the compiler (e.g. libc.a) so they should be writable? Perhaps the issue is with the system headers installed with install_system_headers? Even so I would be surprised if this generated an error the first time it was run, but maybe it generates an error when run a second time and the files are overridden? Is that what is happening?

I believe that some headers are not only copied once but multiple times or there's something that updates them (al/AL.h seems to be a culprit).

What's happening is that, the first time (al/AL.h), I get an error, the second time I get another error (include/bits/errno.h). If I refresh the perms +w, I get back to the first error (al/AL.h).

EDIT: BTW, the files Nix built initially with the compiler are writable during the build process in the sandbox, but at the end, they get moved to the Nix store and becomes read-only and immutable.

[sbc100]

Thanks! Should be a simple fix.