[Memory]More memory optimization policy

[QiJune]

After add a more optimized level, the image_classification demo memory reduced from 93024256 to 92807168. There is a little benefit.

There are still many die variables not be reused. Most of these are gradient variable. After sgd optimization, these gradient can be released. Maybe we have to delete them with a DeleteOperator.

I add another release memory policy with DeleteOp, and tested on resnet model:

Model	no optimize	reuse memory	release memory	forward memory
Resnet	170590208	92995584(reduce 45.5%)	78004224(reduce 54.3%)	77488128

Release memory policy has almost reached the upper limit(forward memory). If we want to reduce the memory occupation further, there are two ways:

• Look carefully at forward pass, and fuse some small operators, and try to reduce some intermediate result.
• Use re-computation policy, throw some results in forward pass and re-compute it in backward pass

[wangkuiyi]

The code style says that a function name should be a verb-subject phrase, like, optimize_memory, instead of memory_optimize.

Also, it seems that we cannot optimize the memory; what we could is to optimize the usage of the memory.

For this case, does it mean reuse_memory and should we rename level into reuse_tensor_with_the_same_size?

[QiJune]

Yes, you are right. It's mainly reuse memory. And level 0 means that we can reuse tensor with the same size. Level 1 means that we can reuse tensor if current tensor size is the same or less than cache pool tensor.

I will refine these codes accordingly. Thanks!

[dzhwinter]

This PR give an aggressive policy to reuse the memory--do stream synchronize after each operator is launch, so we can delete all the variables if the op is not running.
@QiJune
We will merge this PR since the Image mission deadline is looming. Please give some experiment detail of the effect on speed and complete the issue description. Thanks!