radiff2: support comparing files larger than available memory

[davispuh]

Currently when you attempt to compare files larger than available memory you'll get a very helpful error message

$ radiff2 bigfile1 bigfile2
radiff2: Cannot open (null)

It happens because r_file_slurp at radiff2.c#286 tries to allocate memory for whole file which fails and so it returns NULL

[radare]

how big are those files?

On 04/09/2015 02:01 AM, Dāvis wrote:

Currently when you attempt to compare files larger than available memory you'll get a very helpful error message

$ radiff2 bigfile1 bigfile2
radiff2: Cannot open (null)

It happens because r_file_slurp at radiff2.c#286 tries to allocate memory for whole file which fails and so it returns NULL

---

Reply to this email directly or view it on GitHub:
#2338

[davispuh]

~8 GiB for each file, but I've just 8 GiB RAM and 1 GiB swap, now I increased swap and seems it's working, no error yet.
I would also suggest to improve that error as without looking at code it's not possible to tell what's wrong :P

[jvoisin]

Fix it with hardware :)

[davispuh]

this time yeah, but what about comparing 10 TiB files?

[jvoisin]

Ask this guy.

More seriously, I diffing file in anything else than non-trivial way will require a lot of thinking :)

[radare]

Plain byte diffing should not allocate the whole bin in memory. This can be fixed with a loop. But i didnt looked how its done now.

On 09 Apr 2015, at 15:32, jvoisin notifications@github.com wrote:

Fix it with hardware :)

—
Reply to this email directly or view it on GitHub.

[radare]

Iirc this was alreay done in r2 cmp command, so it should be a 5min fix, if no one does it for me i will take care of this when i have some time

On 09 Apr 2015, at 18:18, jvoisin notifications@github.com wrote:

Ask this guy.

More seriously, I diffing file in anything else than non-trivial way will require a lot of thinking :)

—
Reply to this email directly or view it on GitHub.

[igittigitthub]

I wanted to compare two files of 46G but my system only has 32G memory.

My output:

radiff2: Cannot open FILENAME

My version:

radiff2 2.3.0 0 @ linux-x86-64 git.2.3.0
commit: HEAD build: 2018-02-23__17:22:33

A more detailed output would be nice. Maybe something like "Not enough memory" :)