`Base.cp` seems to have a bug when copying large files

[sriharshakandala]

Issue

Base.cp seems to have a bug when copying large files (~17 GB).

For example,

cp ifs_oper_T1279_2016080100.nc atmos_dyamond_summer/ifs_oper_T1279_2016080100.nc

works fine, but,

Base.cp("ifs_oper_T1279_2016080100.nc", "atmos_dyamond_summer/ifs_oper_T1279_2016080100.nc")

tries to create a huge file that eventually uses up all the space on the hard drive before crashing.

However, using

run(`cp ifs_oper_T1279_2016080100.nc atmos_dyamond_summer/ifs_oper_T1279_2016080100.nc`)

instead works correctly.

A more detailed description of the issue can be found CliMA/ClimaArtifacts#64 and CliMA/ClimaArtifacts#69

[stevengj]

Base.cp calls sendfile, which calls jl_fs_sendfile which calls uv_fs_sendfile from libuv, which calls an OS routine. What operating system are you on?

[fredrikekre]

Possibly related to #30723 and/or #39868

[sriharshakandala]

Base.cp calls sendfile, which calls jl_fs_sendfile which calls uv_fs_sendfile from libuv, which calls an OS routine. What operating system are you on?

This is on macOS Sequoia 15.0.1

[stevengj]

It seems like we shouldn't be calling uv_fs_sendfile from cp, but should instead be calling the higher-level function uv_fs_copyfile.

uv_fs_sendfile calls uv_fs_sendfile under the hood on Unices, but does a bunch of other things too. (For one thing, sendfile takes a Csize_t as a length to copy, but on 32-bit systems this means that large files have to be copied in chunks.)

See also #27295 (review)