Windows executable?

[themaddoctor]

@andlabs Do you think you could add a Windows exe file to the dowloads? I get a lot of requests for help from people who don't know how to use linux, and I send them here sometimes. I'd make one myself, but the newest version of Windows that I have is XP.

[m4sterful]

I made a 32 bit executable version that I used for testing decryption with this app off a Windows 7 box. Decrypted two drives without issue.

The only bad news is that it will not to the best of my knowledge open Windows IO directly - //?/device/ - that means you have to image the drive onto a separate drive first (r-studio or any number of open source programs can do that), and then target the image.

reallymine.zip

[andlabs]

On Windows you will need to use forward slashes for those paths to work — that is, \\?\Device, not //?/device/. IIRC the \\ prevents the automatic translation of / to \ that Windows normally does.

I can provide official binaries later.

[m4sterful]

sorry yes, even with \?\Device I was not able to get it working. Just spun it up to try:

[themaddoctor]

Thank you. I'll let people know about it.

[andlabs]

You may need to use \\.\PhysicalDrive##, as described here. According to this, the command

wmic diskdrive list

should list all the physical drives. I don't know, though; I don't have a physical Windows machine to test this on yet (once I back up my previous laptop's Linux partition I would, but IDK when that will happen).

I wonder how I'm going to add knowledge of how to open drives on Windows to reallymine's documentation.

[m4sterful]

Yeah I'd not known of the MS article but R-Studio indicates that under 'OS Object' so I tried that during initial testing. I brought it up to give another try, and clearly we're on the right track with that: I get the following errors, which would seem to indicate it can open the handle to the drive but is not able to properly read it as a block device.

C:\Users\User>H:\programming\go\bin\reallymine.exe decrypt \.\PhysicalDrive1 c:
\test.bin
error running decrypt: seek \.\PhysicalDrive1: Incorrect function.

I'll try a few things and see if I can get success...

[m4sterful]

Heads up I verified I did put in two backslashes for \., I think the comment system wrote them out.

[m4sterful]

So, I cracked out golang and my trusty old Delphi to compare. The following shows a working example in Delphi:

I tried to write essentially the exact same thing in golang, and I seem to get the exact same error as reallymine gives: 'parameter is incorrect'. I looked over the source code for the windows system calls at https://golang.org/src/syscall/syscall_windows.go and I don't see anything wrong or different, no I'm really not certain why this wouldn't work.

[andlabs]

I guess we will have to see what syscall.Open() and syscall.Read() translate to. Are you not able to call syscall.CreateFile() and syscall.ReadFile()?

[m4sterful]

As I say I traced it to syscall_windows.go which indicates syscall.open and syscall.read are just helper functions on syscall.CreateFile() and syscall.Readfile().

CreateFile() appears to work as expected, coming back with what appears to be an appropriate handle. readfile, however...

Here's the definition of readfile in syscall -
ReadFile(handle Handle, buf []byte, done *uint32, overlapped *Overlapped) (err error)

Here's the definition of readfile in Windows MSDN:

BOOL WINAPI ReadFile(
  _In_        HANDLE       hFile,
  _Out_       LPVOID       lpBuffer,
  _In_        DWORD        nNumberOfBytesToRead,
  _Out_opt_   LPDWORD      lpNumberOfBytesRead,
  _Inout_opt_ LPOVERLAPPED lpOverlapped
);

Note that the number of expressions differs - And here's what I did for a direct call attempt - tried different values for the 'done' value in case I could make it work:

func main() {
    disk := "\\\\.\\PhysicalDrive1"
    var fd syscall.Handle
	var numread int
    var err error
	var pathp *uint16
	fmt.Print("Call Syscall.Open, disk ")
	fmt.Println(disk)
    fd, err = syscall.Open(disk, syscall.O_RDONLY, 0)
	pathp, err = syscall.UTF16PtrFromString(disk)
	fd, err = syscall.CreateFile(pathp, syscall.GENERIC_READ, syscall.FILE_SHARE_READ | syscall.FILE_SHARE_WRITE, nil, syscall.OPEN_EXISTING, syscall.FILE_ATTRIBUTE_NORMAL, 0)
	fmt.Print("Windows provided handle: ")
	fmt.Println(fd)

    if err != nil {
        fmt.Print(err.Error(), "\n")
        return
    }

	fmt.Println("Attempting to read from drive");
    buffer := make([]byte, 10, 100)
	var done uint32
	done = 100
	err = syscall.ReadFile(fd, buffer, &done, nil)

    if err != nil {
        fmt.Print(err.Error(), "\n")
    }

    fmt.Printf("Numbytes read: %d\n", numread)
    fmt.Printf("Buffer: %b\n", buffer)

    err = syscall.Close(fd)

    if err != nil {
        fmt.Println(err.Error(), "\n")
    } else {
		fmt.Println("closed handle successfully.") 
	}
}

Error occurs at Readfile(), which I'm guessing is not working the same way as it does in delphi. Again I'm no golang programmer, so I don't have a debugger or anything else that would let me solve this - maybe there's a better call than readFile..

What I can say is that reallymine uses an io.sectionreader, source available at https://golang.org/src/io/io.go. And the sectionreader reads with readat(), which uses pread(), which ends up using sysutils for io as well.

[andlabs]

io.SectionReader wraps an underlying io.ReaderAt. Everything should funnel down to a ReadFile() call.

Your buffer in the Go program is 100 bytes. Try 2048 bytes like in your Delphi program? reallymine should already be using a multiple of 512 bytes, though, so...

[m4sterful]

That's confirmed, changed line to just buffer := make([]byte, 512) and got the following:

H:\programming\go\src>go run H:\programming\go\src\test.go
Call Syscall.Open, disk \\.\PhysicalDrive1
Windows provided handle: 168
Attempting to read from drive
Numbytes read: 512
Buffer: [110011 11000000 10001110 11010000 10111100 0 1111100 10001110 11000000
10001110 11011000 10111110 0 1111100 10111111 0 110 10111001 0 10 11111100 11110
011 10100100 1010000 1101000 11100 110 11001011 11111011 10111001 100 0 10111101
 10111110 111 10000000 1111110 0 0 1111100 1011 1111 10000101 1110 1 10000011 11
000101 10000 11100010 11110001 11001101 11000 10001000 1010110 0 1010101 1100011
0 1000110 10001 101 11000110 1000110 10000 0 10110100 1000001 10111011 10101010
1010101 11001101 10011 1011101 1110010 1111 10000001 11111011 1010101 10101010 1
110101 1001 11110111 11000001 1 0 1110100 11 11111110 1000110 10000 1100110 1100
000 10000000 1111110 10000 0 1110100 100110 1100110 1101000 0 0 0 0 1100110 1111
1111 1110110 1000 1101000 0 0 1101000 0 1111100 1101000 1 0 1101000 10000 0 1011
0100 1000010 10001010 1010110 0 10001011 11110100 11001101 10011 10011111 100000
11 11000100 10000 10011110 11101011 10100 10111000 1 10 10111011 0 1111100 10001
010 1010110 0 10001010 1110110 1 10001010 1001110 10 10001010 1101110 11 1100110
1 10011 1100110 1100001 1110011 11100 11111110 1001110 10001 1110101 1100 100000
00 1111110 0 10000000 1111 10000100 10001010 0 10110010 10000000 11101011 100001
00 1010101 110010 11100100 10001010 1010110 0 11001101 10011 1011101 11101011 10
011110 10000001 111110 11111110 1111101 1010101 10101010 1110101 1101110 1111111
1 1110110 0 11101000 10001101 0 1110101 10111 11111010 10110000 11010001 1110011
0 1100100 11101000 10000011 0 10110000 11011111 11100110 1100000 11101000 111110
0 0 10110000 11111111 11100110 1100100 11101000 1110101 0 11111011 10111000 0 10
111011 11001101 11010 1100110 100011 11000000 1110101 111011 1100110 10000001 11
111011 1010100 1000011 1010000 1000001 1110101 110010 10000001 11111001 10 1 111
0010 101100 1100110 1101000 111 10111011 0 0 1100110 1101000 0 10 0 0 1100110 11
01000 1000 0 0 0 1100110 1010011 1100110 1010011 1100110 1010101 1100110 1101000
 0 0 0 0 1100110 1101000 0 1111100 0 0 1100110 1100001 1101000 0 0 111 11001101
11010 1011010 110010 11110110 11101010 0 1111100 0 0 11001101 11000 10100000 101
10111 111 11101011 1000 10100000 10110110 111 11101011 11 10100000 10110101 111
110010 11100100 101 0 111 10001011 11110000 10101100 111100 0 1110100 1001 10111
011 111 0 10110100 1110 11001101 10000 11101011 11110010 11110100 11101011 11111
101 101011 11001001 11100100 1100100 11101011 0 100100 10 11100000 11111000 1001
00 10 11000011 1001001 1101110 1110110 1100001 1101100 1101001 1100100 100000 11
10000 1100001 1110010 1110100 1101001 1110100 1101001 1101111 1101110 100000 111
0100 1100001 1100010 1101100 1100101 0 1000101 1110010 1110010 1101111 1110010 1
00000 1101100 1101111 1100001 1100100 1101001 1101110 1100111 100000 1101111 111
0000 1100101 1110010 1100001 1110100 1101001 1101110 1100111 100000 1110011 1111
001 1110011 1110100 1100101 1101101 0 1001101 1101001 1110011 1110011 1101001 11
01110 1100111 100000 1101111 1110000 1100101 1110010 1100001 1110100 1101001 110
1110 1100111 100000 1110011 1111001 1110011 1110100 1100101 1101101 0 0 0 110001
1 1111011 10011010 1001111 110100 1100111 11110110 0 0 10000000 100000 100001 0
111 11111110 11111111 11111111 0 1000 0 0 0 10000000 11100000 11101000 0 0 0 0 0
 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 0 0 0 1010101 10101010]
closed handle successfully.

I've confirmed the binary matches sec0 of the hard drive I specified. If I have the time I'll try going through the code for reallymine's hard disk IO and see if I can figure out what is causing the issue on that side with the same target input.

If I get really adventurous I might rewrite the program in delphi or c++, as the copy speed on linux and windows is pretty abysmal at the moment.

[andlabs]

@m4sterful I'm working on improving the copy speed; see #38.

That's correct; Windows seems to require disk access to be multiples of the sector size (judging from several random Stack Overflow questions I've seen in the past). But reallymine has always read 50MB at a time because it uses a sector count of 102400 so the buffers should always be 102400*512 bytes long (which is also equivalent to 12800*4096, so even those drives should work)... Unless there's some other limit I don't know about?

[m4sterful]

error occurs on line 29 of disk.go, func Open(). filename is passed as a string and pushed to os.open(), then on 29 a call to OS.Seek(), which fails. It never gets to the point of the decryption loop, at that point you're still just attempting to find the size of the disk...

[m4sterful]

I have confirmed that hacking a static value for realsize resolves the issue of opening a hard disk directly on Windows, that is, commenting out line 29 and setting realsize to the value of the hard disk size in bytes allowed it to continue and begin decryption successfully on a raw drive:

fmt.Print("Seeking")
	// realsize, err := f.Seek(0, io.SeekEnd)
	var realsize int64
	realsize = 2000398934016

I'll figure out a windows safe way of figuring out the disk size later on.

[andlabs]

Hmmm. On Windows, (*os.File).Seek() resolves to a call to SetFilePointer(), which MSDN says should only work on actual files.

Does f.Stat() work instead?

s, err := f.Stat()
if err != nil {
    return errfail(err)
}
realsize := s.Size()

[m4sterful]

Unfortunately no, stat() comes back with a failure as it does not appear to have been made fully Windows compatible.

I tried ricochet2200's go-disk-usage (https://github.com/ricochet2200/go-disk-usage) which is a cross-platform and had a few build issues unrelated to the software, but also I believe GetDiskFreeSpaceExW expects a folder rather than a physical drive name according to MSDN; that's how I've used it previously. Had to get back to paid work. I expect I can take another look later on tomorrow.

The following stackoverflow document describes the method to get disk size in windows (ie that there is no existing cross-platform way in Golang and kernel32.dll's GetDiskFreeSpaceExW must be wrapped)
https://stackoverflow.com/questions/20108520/get-amount-of-free-disk-space-using-go

[andlabs]

GetDiskFreeSpaceExW() is not correct for this. DeviceIoControl() with IOCTL_DISK_GET_LENGTH_INFO is needed (according to several Stack Overflow answers, including by some of the more knowledgeable winapi people there), but I'm not sure whether it can safely be done in pure Go (without cgo).

What do you mean by stat() on Windows not being complete?

[m4sterful]

stat() returned with an invalid file type error: in the windows implementation you can stat a file, not a drive. That's because it uses GetFileAttributesEx(), which does not support volumes, and clearly not raw drives (see https://golang.org/src/os/stat_windows.go line 78).

From MSDN:
When the GetFileAttributesEx function is called on a directory that is a mounted folder, it returns the attributes of the directory, not those of the root directory in the volume that the mounted folder associates with the directory. To obtain the attributes of the associated volume, call GetVolumeNameForVolumeMountPoint to obtain the name of the associated volume. Then use the resulting name in a call to GetFileAttributesEx. The results are the attributes of the root directory on the associated volume.

[m4sterful]

I may have misunderstood, my understanding is that f.stat() works on linux to get the size of a drive as well as just files, does it not? If so, one might have the expectation of similar results on Windows via cross-compatibility, rather than this limitation.

[andlabs]

Yes, but that's because of how Unix and Windows evolved their concept of a file. Go could probably use GetFileType() to figure out what is what, but at that point we would have to go to that issue tracker.