drive push is creating duplicates

[l3iggs]

I'm not sure how this has happened, but duplicate files are showing up in my Google Drive web interface. These files have both the same name and the same checksum.

I'll try to see if I can nail down an easy, step-by-step way to reproduce this. I can say the following things for sure:

• This is in a folder which Google Drive has only seen interaction via the drive push command.
• This occurred with 0.1.5 code from mid-march (not exactly sure which commit, that info has now been overwritten.

Maybe this is related to #56?

[odeke-em]

Interesting. Good thing there are tagged releases so you could bisect the commits (manually or whatever way) between the releases between 0.1.5 and 0.1.6.

[l3iggs]

I have no evidence this isn't still happening.
All I'm saying here is that it was present for certain sometime when drive version returned 0.1.5. (by the way, drive version should probably report the short commit hash).

[odeke-em]

If you could reproduce this, it would be helpful.

[mullenkamp]

I seem to be getting the same problem recently. Earlier today I was using push and no duplicates were created, but all of a sudden it will create a duplicate folder with (1) next to it with all of the same files. I am using the mounted push option (which took me a while to figure out completely). Maybe I'm using it incorrectly...?

Example:
drive push -m /local/dir/transfer/ google/drive/

For my purposes, I simply want to transfer files to my google drive from my local directory (rather than mirror them again in myhome directory).

In addition to the above problem, the push will occationally try to delete files and folders from directories outside of the specified directories from the push command.

Thanks for your help.

[odeke-em]

Okay push -m is a beast of its own for which the destination in the root should be the destination not the file name. Also -m is basically asking for operations outside of the root. It is one of the commands that require you to recognize risk. Please use push mostly :)

[mullenkamp]

Yeah...I had the distinct feeling that the push -m is a bit flakey. Thanks for the clarification. I'll keep playing around with it.

[l3iggs]

I've never used the -m switch.

[odeke-em]

@l3iggs any re-spotting of this issue?

[mullenkamp]

I still have this problem. It happens more infrequently now that I'm not using 'push -m', but still frequently enough to cause enough problems. It seems to happen when I'm transferring large amounts of files (>5000 files). It also duplicates folders. I'm using symbolic links of my folders in different locations on my HDD. That could be the problem.
Thanks for your help.

[brandoncurtis]

I am doing exactly the same thing and also getting massive duplication: 'drive push' to move huge amounts of files (10-100GB) that are inside symlinked folders.

[odeke-em]

@brandoncurtis and @mullenkamp please give me your drive version information, but I can guess that is pretty recent because @brandoncurtis joined in not so long ago :) I should investigate this real quick because inspecting huge amounts of files is very hard and bugs like this are disastrous. I think I have a hunch on what could be tripping out and I'll investigate probably tonight or tomorrow evening. If you have the luxury of time, would you mind helping test out this commit d51dc09 with a small test of symlinked dirs that you could have created
e.g

$ cd $GOPATH/src/github.com/odeke-em/drive
$ git reset --hard d51dc091ac9a3a74a1c38d83a5acbd64b3df87c7
$ go get github.com/odeke-em/drive

If this doesn't create duplicate paths then most of the suspicions of symlink resolution creating multiple paths have been confirmed. This probably then originated from the feature request to resolve symlinks fully but then the original manifest was probably not cleaned up. I'll dig more later on.
Thank you.

[brandoncurtis]

I have drive version 0.1.6. I'll put that commit through its paces tonight, on an Apps for Edu account so I don't have to worry about space!

[odeke-em]

Nice, I have one provided by my school too and they are very generous with space. For sure, thank you.

[l3iggs]

@l3iggs any re-spotting of this issue?

I actually haven't been using drive too much lately. So, no.
Also, it might be noteworthy that I've not been using any symlinks or anything funny like that.

[odeke-em]

@mullenkamp and @brandoncurtis please get the latest source or take a look at e071b36. Previously, when using push -m aka mounted push: before full symlink traversal was in, such points were taken note of and added in manually. That commit takes note of that.
As for @l3iggs I don't know how to reproduce your issue otherwise :(

[odeke-em]

Closing this. Please re-open if persists.