merging: tolerate process interruptions during merging

[stefanhengl]

Currently the call to zoekt-merge-index does not tollerate process interruptions, which can potentially lead to duplicate indexes. This could be the root cause for the duplicate indexes we see in production.

With this PR

• merge.Merge returns a temporary compound shard (*.tmp)
• zoekt-merge-index calls merge.Merge and is reponsible to delete the input shards and rename the temporary compound shard returned by merge.Merge.
• zoekt-sourcegraph-indexserver just calls zoekt-merge-index and always expects a consistent state.

A positive side-effect is that both merge and explode now behave the same when called from the command line: both remove the input shards and leave the output shards behind.

Test Plan:

• Unit tests needed only obvious adjustments.
• I ran a local instance of indexserver and verified that merging doesn't leave any temp files behind.

Can be reviewed commit by commit.

[keegancsmith]

LGTM. But I couldn't really find an example where we didn't want to delete everything and then do a rename. Why don't we just always do that and simplify zoekt.Merge interface.

The only example where you don't seem to want to do that is when merging testdata. But can't we just copy those files first to a tmp location anyways?

[stefanhengl]

The only example where you don't seem to want to do that is when merging testdata. But can't we just copy those files first to a tmp location anyways?

extactly, the interface was mostly inspired to give more flexibility during testing. I am fine with copying to temp location, too. Will update the PR.

EDIT: I misread the first time and thought your are talking about the onSuccess callback in zoekt-merge-index's merge. I removed that now, but I won't touch zoekt.Merge because it mirrors nicely what Explode does. If we want to pull the delete logic into the zoekt package we should do it for both and then do it in the same PR.