[src] Speed fix to online decoding (thanks: David van Leeuwen)

[danpovey]

This is a fix to a problem I discovered due to this kaldi-help conversation:

https://groups.google.com/d/msgid/kaldi-help/d362c3b1-24af-41c5-8ead-7a9cea212681%40googlegroups.com

It turns out that the call to ComputeSimpleNnetContext() is slow after you call
ModifyNnetIvectorPeriod(), because the Modulus() of the network increases
(and as it happens it causes a bunch of unnecessary computation because
that relates to the ivectors, not the main input). This patch should fix the slowness
by computing the context before ModifyNnetIvectorPeriod().

[davidavdav]

These are the updated results:

model	old setup time	new setup time	real time factor
nnet2	1.1 s	0.9 s	0.232
nnet3 tdnn	9.7 s	1.1 s	0.093
neet3 tdnn lstm	13.9 s	2.2 s	0.245

Where new setup time is measure using this PR.

Thanks for fixing this!

[davidavdav]

Please note that the setup times above include a runtime

      SetBatchnormTestMode(true, &am_nnet.GetNnet());
      SetDropoutTestMode(true, &am_nnet.GetNnet());
      CollapseModel(CollapseModelConfig(), &am_nnet.GetNnet());

but I haven't timed yet if leaving that out (and moving that to model preparation time) makes much of a difference. I assume the operation is idem-potent.

[danpovey]

yes it's idempotent.