How to manually set parameters of a module?

Discussion
1

I am asking a probably stupid question about the Module API, though I am unable to figure that out for two days.
I want to manually change the parameters of a network during training and don’t know how to do this. Specifically, I am writing a method to perform spectral normalization, which divides all weights in a network by some value.
I have found in documents that the parameters of a module are stored in “self._exec_group.param_arrays” and “self._kvstore”, and I need to change both of them. But I don’t know how to replace the values in a “kvstore” object, whose updater has been set as an optimizer. Does anyone have idea about the problem? Is there any alternative way for achieving my purpose?

（I am using Module API because I am adapting a program from github, which uses Module API, and I am reluctant to rewrite it using Gluon or other deep learning frameworks.）

2

Hi

Could you not use mod.setParams() https://mxnet.incubator.apache.org/api/python/module/module.html#mxnet.module.BaseModule.set_params.

You can use mod.getParams() to get the parameters, perform your normalization then set them back using mod.setParams

1 Like
3

Sorry, it doesn’t work. The parameters are resumed as soon as I call mod.update(). It seems that the parameters stored in kvstore are not changed.

4

Did you manage to work out a way to do this? I am also interested. :slight_smile:

5

Hey @HaydenFaulkner, I would suggest trying to rewrite your model in Gluon, that would make it trivial to update the parameters and have better control on the flow of your data. Happy to assist you converting your Module code to Gluon, and hopefully convert you from Module to being a Gluon fan :smile:

If that’s not an option, feel free to post your code here

6

Thanks @ThomasDelteil! Sadly Gluon isn’t an option as I’m modifying an existing codebase. However I found that calling set_params({k: v}, allow_missing=True) on the module updated the params how I desired.