Adding files preserving mode or mtime results in data corruption

[kstuart]

Linux ce937568e283 5.17.5-arch1-1 #1 SMP PREEMPT Wed, 27 Apr 2022 20:56:11 +0000 x86_64 Linux

js-ipfs version: 0.15.2-0078aad85f92d7bc8e3a22eb9ef8efd257b439b5
interface-ipfs-core version: ^0.155.0
ipfs-http-client version: 1.0.0
Repo version: 12
System version: x64/linux
Node.js version: v16.4.2
Commit: 0078aad

jsipfs run in daemon offline mode

Steps to reproduce with --preserve-mtime (empty file)

touch -d@1652299617 /tmp/empty_file
jsipfs add --preserve-mtime /tmp/empty_file
added QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna empty_file

jsipfs files stat /ipfs/QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna
QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna
Size: 6
CumulativeSize: 22
ChildBlocks: 0
Type: file
Mode: -rw-r--r--
Mtime: May 11, 2022, 08:06:57 PM UTC

jsipfs get QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna
Saving file(s) QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna

xxd QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna
00000000: 0a04 0802 1800 ......

Expected:

The file 'QmcrCRhLSqsroGBX2PyBtdCrKWBDxjgPpATLg3KxMqywna' to have size 0 and contain no data.

---

Steps to reproduce with --preserve-mode (file with content)

echo -n "A" > /tmp/test_file
touch -d@1652299617 /tmp/test_file
jsipfs add --preserve-mode /tmp/test_file
added QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH test_file

jsipfs files stat /ipfs/QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH
QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH
Size: 9
CumulativeSize: 17
ChildBlocks: 0
Type: file
Mode: -rw-r--r--
Mtime:

jsipfs get QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH
Saving file(s) QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH

xxd QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH
00000000: 0a07 0802 1201 4118 01 ......A..

xxd /tmp/test_file
00000000: 41 A

Expected:

The file 'QmW198BawsGUvwKQN8hxUHKdCNSS3fYf7pMmPQujUAJnBH' to have size 1 and its content to exactly match /tmp/test_file.

[BigLep]

2022-06-10 conversation: there are two things to look into:

1. Why js-ipfs is returning a non-zero size
2. Why the CIDs are different

Maintainers agree this needs to get looked into. There isn't an estimate yet on when this will get picked up though given other priorities. Is this blocking for you @kstuart on your work?

[github-actions]

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.

[kstuart]

@BigLep no, this doesn't block me. After further investigation I've updated the issue, note that preserving mode or mtime will add erroneous file data to the node, essentially a corruption of the original file.

[whizzzkid]

js-ipfs is being deprecated in favor of Helia. You can follow the migration plan here #4336 and read the migration guide.

Please feel to reopen with any comments before 2023-06-05. We will do a final pass on reopened issues afterward (see #4336).

This issue might be resolved in Helia, if not, please let us know!