fix caffe stream

[piiswrong]

@tqchen @HrWangChengdu

[Taco-W]

@piiswrong I still cannot pass example after adding cudaStreamSyn on my machine.

[piiswrong]

@HrWangChengdu "device" works for me but 'local_allreduce_device' doesn't. It's so strange. They should be the same thing

[Taco-W]

Me too. 'Device' could work with or without streamSync. 'local_allreduce_device' cannot work in either setting.

[piiswrong]

@HrWangChengdu Looks like caffeop is fine. It's local_allreduce_device that's broken @tqchen

[piiswrong]

@HrWangChengdu http://docs.nvidia.com/cuda/cuda-driver-api/stream-sync-behavior.html#stream-sync-behavior
looks like this fix is not necessary. Caffe uses the default stream. It blocks all other streams

[tqchen]

should be fixed by now, please check

[Taco-W]

@tqchen It works now!
@piiswrong Yes. Example code could run without this fix. I'll double check this later this week. Thank you.