[HELP NEEDED] Did I make reallymine faster?

[andlabs]

I don't know if this will make anything faster for other people, but I split decryption itself across all CPUs. You can try it on the concurrent-decryption branch. A synthetic 2.4GB file decrypted in 3.5 minutes with this vs 11.5 minutes for the last binary release, but I didn't test much further than that. If it does make things faster somehow for everyone though, then I'll be surprised, but I can't know unless someone would be willing to try it. Thanks!

UPDATE I've now made some binaries available. You'll need to use the decrypt command to see the results.

Some benchmarks on a 400MB test file:
OS X Sierra 64-bit
original - average of 1m40s
concurrent - average of 10s
Linux 64-bit in a VM
original - average of 40m (???)
concurrent - average of 12s-18s

http://andlabs.lostsig.com/reallymine-concurrent-decryption-darwin64
3.1MB, md5sum 6a8acd865b9c8efb55d87ab8d5994b40

http://andlabs.lostsig.com/reallymine-concurrent-decryption-linux64
3.2MB, md5sum 5e2c82fe1c4a38b41ac5cdf5887261f5

[andlabs]

Also if you want me to provide a binary I could do so later.

[stynoo]

Could you provide a binary plz (Ubuntu 64bit if possible)? I can't seem to figure out how convince go to compile from the other branch. (Or I figured it out and it is still only working on one core.)

What I did notice is that the linux nfts driver introduces HUGE overhead and cpu-waits when decrypting so filesystem choice seems to be relevant as well when ppl report slow decrypts. I'll run some tests on that as well.

[puyuan]

I tried concurrent-decryption branch on WD Book Essential, Initio chip.
Command is: reallymine decrypt /dev/sdc /target_path_to_img.img
Its blazing fast, unfortunately nothing is decrypted.

I compared to the decryption on master branch. In the decrypted version, the hexdump -C -n 10000
would give me some ASCII text like "Invalid partition table", but in the concurrent version, it seems everything is still encrypted and showing a very different hex dump. Pretty much no readable ASCII.

I would love to use the concurrent version at the speed it was copying. Would love to see this fixed.

[andlabs]

@stynoo I can do so later, either today or tomorrow.

@puyuan Is the "decrypted" data the same as the encrypted data? What happens if you extract those early bytes into its own file and decrypt that?

[puyuan]

@andlabs

I did some more test. In the concurrent-decryption branch the
reallymine decrypt /dev/sdc supposedly_decrypt.img
simply makes a copy of the disk without decrypting. I can decrypt it again with
reallymine decryptfile supposedly_decrypt.img out.img dek steps
So the workaround now is:

reallymine getdek /dev/sdc
reallymine decryptfile /dev/sdc outfile "dek_from_above" "steps_from_above"

[andlabs]

Of course that will still do a non-concurrent decryption. So this is interesting; will have to look into it.