MXNet Profiler (#3163)

[itsarbit]

• NNVM Refactor (NNVM Refactor apache/mxnet#3194)

• Init nnvm change

• temp checkin

• Move TShape to NNVM

• Redirect Symbolic API to NNVM

• Add Op Prop Adapter

• Finish migrate in shape infer

• Pass all symbolic test

• temp commit

• enable aux data

• [EXEC] Basic version of exec for forward only

• [EXEC] Enable most optimizations, still wait grad and context

• fix legacy op with latest one

• Update NNVM NodeRef

• Adapt to newer interface

• ALl registry of backop is complete

• temp commit

• Hack finish backward pass

• [EXEC] One day pass

• [EXEC] Pass all operator unittest

• [EXEC] enable model parallel

• Fully pass all legacy tests

• Remove legacy symbolic code

• update news

• Make travis compile

• Fix python3

• Update viz module to new json format

• [NNVM] Imperative Invoke ([NNVM] Imperative Invoke apache/mxnet#3208)

• [Engine] Deduplicate Variable Util

• [NNVM] NNVM Imperative Invoke

• [NNVM] Imperative improve speed

• fix

• fix

• [scala] link libnnvm.a ([scala] link libnnvm.a apache/mxnet#3214)

• [PYTHON] Optional Cython Module for Symbols ([PYTHON] Optional Cython Module for Symbols apache/mxnet#3242)

• [CYTHON] Checkin cython enhancement

• fix lint

• [DOC] Move common doc to base

• [EXEC] Support fcompute ([EXEC] Support fcompute apache/mxnet#3249)

• [EXEC] Support fcompute

• Fix lint

• fix lint

• [OP] Add alias support ([OP] Add alias support apache/mxnet#3261)

• Fix path in setup.py (Fix path in setup.py apache/mxnet#3276)

• Fix path in setup.py

• revert the nnvm version

• [WIP] Element wise op refactor ([WIP] Element wise op refactor apache/mxnet#3245)

• [OPERATOR] Refactor Unary Ops

• [OPERATOR] Refactor Binary Scalar Ops

• Use alias

• update nnvm version (update nnvm version apache/mxnet#3290)

• Fix breaking changes after pull master (Fix breaking changes after pull master apache/mxnet#3291)

• [CYTHON] Cython module for NDArray ([CYTHON] Cython module for NDArray apache/mxnet#3292)

• [NDARRAY] Cython module for ndarray

• More strict tests

• [NNVM] change of attr to set_attr ([NNVM] change of attr to set_attr apache/mxnet#3303)

• Update run_test.sh

• add nnvm cmake with windows (add nnvm cmake with windows apache/mxnet#3255)

• [WIP] binary broadcast wip ([WIP] binary broadcast wip apache/mxnet#3301)

• [WIP] binary broadcast wip

[OPERATOR] Binary Broadcast ops

fix lint

lint

fix

max and min

update submodule

before removing reduce axis

broad cast reduce ops

• update

• fix

• fix warning

• fix

• x (Fix cython build on linux apache/mxnet#3308)

• [IO] Python based ImageIter and Augumenter ([IO] Python based ImageIter and Augumenter apache/mxnet#3227)

• [IO] Python based ImageIter and Augumenter

• fix

• fix

• fix

• [OPT] NNVM Optimizer (Nnvm optimizer apache/mxnet#3314)

• fix cpython in windows (fix cpython in windows apache/mxnet#3309)

• Add Mathematical functions ([OP] Add Mathematical functions  apache/mxnet#3317)

• fix image io

• add hypot degrees radians cosh sinh tanh arcsinh arccosh arctanh ([OP] Add Mathematical functions apache/mxnet#3335)

• add recent examples, collect some missing tutorials (Add recent examples, collect some missing tutorials for Awesome MXNet apache/mxnet#3340)

• Improving docs & utilities for distributed training example. (Improving docs & utilities for distributed training example. apache/mxnet#3341)

• add init dict

• disable SSE for arm hardware e.g. Raspberry Pi (disable SSE for arm hardware e.g. Raspberry Pi apache/mxnet#3346)

• Add channel_ to Shape2D calculation (Add channel_ to Shape2D calculation apache/mxnet#3181)

• Add channel_ to Shape2D calculation

• scalapkg, add example multitask (scalapkg, add example multitask apache/mxnet#3186)

• RNN cell demo with ptb LSTM language model (RNN cell demo with ptb LSTM language model apache/mxnet#3197)

• rnn-cell demo (push to server for testing)

• a running example with cuDNN RNN cell

• Bulk lint fix (Bulk lint fix apache/mxnet#3211)

• [TENSOR] Add FlatTo1D for all elementwise ops ([TENSOR] Add FlatTo1D for all elementwise ops apache/mxnet#3238)

• Fix little bug on context (Fix little bug on context apache/mxnet#3202)

• add PennTreeBank Language Model using lstm model in R (add PennTreeBank Language Model using lstm model in R apache/mxnet#2659)

• Add function 'print_summary' and some revise (Add function 'print_summary' and some revise apache/mxnet#3161)

• Add function 'print_summary' and some revise

Add function 'print_summary' for print detail information of network, and format argument was add in 'plot_network'.
You can use 'print_summary' like:
"""
net = get_symbol(1000)
shape = {'softmax_label': (64, 12), 'data': (64, 3, 224, 224)}
mx.viz.print_summary(net, shape=shape)
"""
If without shape, the number of arguments would be nonsense currently.

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Update visualization.py

• Added my CmakeLists.txt for caffe plugin, etc.

• Revert "fix travis scala test config" (Reenable testing the Julia bindings apache/mxnet#3246)

This reverts parts of commit 3e15f62.
Reenables testing the Julia bindings

• [Scala] Code generation for Symbol ([Scala] Code generation for Symbol apache/mxnet#3217)

[scala] auto-generate Symbol functions

• fix spelling errors (fix spelling errors apache/mxnet#3258)

Also align grammar and punctuation in short descriptions of features

• fix typo in run_test.sh (fix typo in run_test.sh apache/mxnet#3260)

• Copy slice along arbitrary axis (Copy slice along arbitrary axis apache/mxnet#3259)

• rnn-cell demo (push to server for testing)

• a running example with cuDNN RNN cell

• add copyslice along arbitrary axis for NDArray

• copy_slice_to as an ndarray operator

• Python interface to the _copy_slice_to operator

• fix lint error

• Enable concatenation for dim-1 vectors (Enable concatenation for dim-1 vectors apache/mxnet#3264)

• fix PReLU backward computing (fix PReLU backward computing apache/mxnet#3277)

• Add reverse option in Reshape (Add reverse option in Reshape apache/mxnet#3280)

• add scala example, end2end neural-style (add scala example, end2end neural-style apache/mxnet#3267)

add scala example, end2end neural-style

• Improve multi-GPU performance (Improve multi-GPU performance apache/mxnet#3241)

• update kvstore

• update model.py

• bandwith tool

• update readme

• tiny

• fix lint

• fix batch size of dist_device_sync

• fix

• fix perf problem of kvstore when only using a single device

• roll back to previous strategy how to choose update_on_kvsotre

• add an optionl MXNET_ENABLE_GPU_P2P to control whether or not use p2p

• update dmlccore (update dmlccore apache/mxnet#3293)

• Fix newer version of gtest and cpptest (Fix newer version of gtest and cpptest apache/mxnet#3294)

• when set use_global_stats then do not use cudnn (when set use_global_stats then do not use cudnn apache/mxnet#3289)

• when set use_global_stats then do not use cudnn

• fix batch norm with use_global_stats

• Fix req+reserve_space in cudnn_rnn ([OP] Fix req and reserve_space in cudnn_rnn apache/mxnet#3274)

Fix req

Fix reserve_space

Allocate reserve_space using Storage

• add cudnn off option in Convolution (add cudnn off option in Convolution apache/mxnet#3270)

• add support for building on power (add support for building on power apache/mxnet#3302)

• add recent examples, collect some missing tutorials (Add recent examples, collect some missing tutorials for Awesome MXNet apache/mxnet#3340)

• CMake for caffe plugin

• Fix metric & im2rec.py

• [Scala] Nnvm ops for NDArray & Symbol ([Scala] Nnvm ops for NDArray & Symbol apache/mxnet#3361)

• [scala] nnvm op support

• [scala] remove unused codes

• fix scala native code style

• [R] Fix the R interface ([R] Fix the R interface apache/mxnet#3334)

• [R] Fix the R interface. remove man

• Fix BN legacy issue

• Locate compiled library on Windows (Locate compiled library on Windows apache/mxnet#3369)

• Fix metric & im2rec.py (Fix metric & im2rec.py & image io apache/mxnet#3375)

image io fix

• Update legacy op FBackwardInGradIndex (Update legacy op FBackwardInGradIndex apache/mxnet#3376)

• Update legacy op FBackwardInGradIndex

• fix test

• Fix for LRN Layer (Fix for LRN Layer apache/mxnet#3366)

• fixed cpu forward bug

• added out_data[lrn_enum::kOut] as backward req.

• removed lint

• removed duplicate out_data[lrn_enum::kTmpNorm],

• removed inplace option

• add backward index

• include some special functions (include some special functions apache/mxnet#3337)

• gamma
• gammaln
• log1p
• expm1

• fix kv build (Nnvm apache/mxnet#3385)

• initial profiler branch based on dmlc/mxnet:nnvm

• [profiler] add profiler & modify engine API

• [profiler] add USE_PROFILER compile flag & modify code for changed engine api

• [profiler] add c_api interface & modify graph_executor

• [profiler] add python api

• [profiler] typo & lint error

• [profiler] reduce overhead & add PROFIELR_MESSAGE_FUNCNAME macro

• [profiler] remove profiling argument from PushSync/PushAsync

• [profiler] refactor profiler.h/.cc

• [profiler] improve readability

• [profiler] typo && add TODO comment

• [profiler] fix ndarray op name & add WaitForVar back

• [profiler] add example/profiler/profiler_ndarray.py

• [profiler] fix memleak by using op->name

• [profiler] fix lint

• [profiler] fix lint