I am using the C++ implementation of the Mxnet library. And I noticed that for some reason, the moving_mean and moving_variances are not updated, no matter the momentum I choose.
The process that I am using is a full forward pass, in training mode (the forward parameter is set to true). Then a backward pass. Then, I update all the parameters with the optimizer. It seems that this operation doesn’t apply on the
moving_mean nor the
moving_var. Thus, I don’t know - am I missing a step to update the
I set the
fix_gamma parameter to false, so I was expecting a full update of the
BatchNorm during the training.
The only differences of values that I get are basically numerical approximation in the order of 1.0e-9.
Also, if I could have a hint on how to operate if I set the
output_mean_var to true? The output doesn’t seem to be accepted by the following activation layer.