Weird result when run with the autograd.record


I have some experiences with tensorflow and is new to mxnet.

I define a hybrid net which is composed of a 3D convolution net (SampleNet) and a pre-trained PSP net.

I use the following codes to fixed the parameters in the PSP net.

def net(input)
 trainer = gluon.Trainer(self.sample_net.collect_params(),'sgd',{'learning_rate':0.0001})
 with autograd.record():
        sample_result = SampleNet(input)
        predict_result = PSPNet(sample_result)

the problem is, when input an image (left, as the result of sample_result) when i output the predicted_result, i get very noisy output (Middle). and if i commented the autograd.record(), the predict_result seems decent (right). So what’s happened?


Any suggestion or debug direction would be appreciated.

The command autograd.record puts your network in training mode, which means that layers such as dropout, BatchNorm will behave differently than in inference mode. By default, the output of a network is in autograd.predict_mode. Therefore when you comment out the line with autograd.record your network will use summary statistics (for BatchNorm) and average of the activations (dropout, instead of killing a random sample of them). Hope this helps.