Memory Cost of Deep Nets under Different Allocations

This folder contains a script to show the memory cost of different allocation strategies, discussed in Note on Memory Optimization.

We use inception-bn as an example, with batch size of 32.

How to See the cost

The possible options are gathered together in the Makefile. Type the following command to see the allocation cost. Look for the Final message Total x MB allocated

• make no_optimization
  • Shows the cost without any optimization.
• make with_inplace
  • Shows the cost with inplace optimization.
• make with_sharing
  • Shows the cost with memory allocating algorithm for sharing.
• make with_both
  • Shows the cost of memory allocation with both inplace and sharing optimization.
• make forward_only
  • Shows the cost of when we only want to run forward pass.

Notes

• You can change the symbol in the inception_memcost.py to the net you interested in.
• You will need to install mxnet or type make on the root folder before use the script.
• The estimation is only on space cost of intermediate node.
  • The cost of temporal workspace is not estimated, so you will likely need more memory when running real nets.
• The estimation does real allocation on CPU, the plan is the same on GPU.