enable hidden arg

[piiswrong]

refactor lr_mult, wd_mult, ctx_group

revert test

update submodule

[piiswrong]

@tqchen

[tqchen]

seems no changes in here?

[tqchen]

oh i see it now

[tqchen]

Need to enable attributes in https://github.com/dmlc/mxnet-memonger for backing memory monger

[piiswrong]

Refactor it and merge into mxnet?

[tqchen]

to make sure things won;t break, can you push a PR to https://github.com/dmlc/mxnet-memonger to change the attribute force_mirroring to __force_mirror__?

[piiswrong]

only julia failed. please review again and merge

[tqchen]

I am a bit worried about this procedure. Direct checking things in here can make things slow. We will need a speed test under cython mode to confirm this do not affect the symbol composition speed.

[piiswrong]

do you have scripts for this?

[piiswrong]

it's only a loop over 3 elements, so hopefully won't be a big deal

[tqchen]

Can you use something like a global map structure? so it is O(1)?

[tqchen]

I am fine merging it if it will get eventually deleted after evolution.

For speed testing, simply do elementwise add composition and measure ops /sec

[tqchen]

See my comment about potential speed issues.

[piiswrong]

@tqchen jxie@jxie-gpu:/scratch/mxnet$ MXNET_ENFORCE_CYTHON=1 python -m timeit -s "import mxnet as mx; a = mx.sym.Variable('A'); b = mx.sym.Variable('B')" "c = 10*a + 5/b"
10000 loops, best of 3: 22.1 usec per loop

[tqchen]

a historical way is to do something like

import mxnet as mx
import time

a = mx.sym.Variable('a')
b = mx.sym.Variable('b')
start = time.time()
nrep = 100
nstep = 10000
for i in range(nrep):
    for j in range(nstep):
       a = a + b
end = time.time()
print("%g ops/sec" % nrep* nstep/(end-start))
```

[tqchen]

Need to measure the speed under cython build, both before and after the change

[piiswrong]

before the change:
jxie@jxie-gpu:/scratch/mxnet$ MXNET_ENFORCE_CYTHON=1 python -m timeit -s "import mxnet as mx; a = mx.sym.Variable('A'); b = mx.sym.Variable('B')" "c = 10*a + 5/b"
10000 loops, best of 3: 20.2 usec per loop

so the difference is 10%. Doesn't seem to matter much

[tqchen]

will the set of kHiddenArgs grow later, or is it only for deprecation warning purposes? If it is, we can make sure they get removed in next major version

[piiswrong]

its the difference between typing Activation(lr_mult=xx) or Activation(lr_mult=xx)

[tqchen]

I see, I am fine with merging it if you can run a python test script I posted. I hope we do not do such automatic conversion in the backend for more arguments, but instead ask user to directly write underscore for front-end attributes(or have some function so help doing so)

[piiswrong]

84245.5 ops/sec