I noticed that the default data type (i.e., the one used when creating arrays without specifying the
dtype argument) in MXNet is
float32 when I found my loss became
nan after several hundred iterations. Now I want to use
float64 in stead of
float32 to see the precision of the data type was responsible for the
nan. However, it seems to me that that the only way to specify
float64 as the data type for every array is to explicitly add
dtype='float64' to every function call that creates an array from scratch. I wonder if there is a way to replace the default data type.
This reminds me of a similar question Shortcut for enabling GPU concerning context rather than dtype. The two questions put together are simply ‘Is there any way to change the default settings (
dtype) for newly created arrays?’
UPD: I did not find a way to specify the data type of the convolutional layer parameters.
Embedding both have
dtype in their constructors, but
Conv2D does not.
Below are excerpts from implementations of
Conv2D where the parameters are defined:
self.weight = self.params.get('weight', shape=(units, in_units), init=weight_initializer, dtype=dtype, allow_deferred_init=True) if use_bias: self.bias = self.params.get('bias', shape=(units,), init=bias_initializer, dtype=dtype, allow_deferred_init=True) else: self.bias = None
_Conv, the superclass of
self.weight = self.params.get('weight', shape=wshapes, init=weight_initializer, allow_deferred_init=True) if use_bias: self.bias = self.params.get('bias', shape=wshapes, init=bias_initializer, allow_deferred_init=True) else: self.bias = None
Note that the
Dense specifies the data type for its parameters, but
Conv2D does not, which in turn means that the parameters can only be of type
Parameter has this default setting when
dtype is not specified.
UPD2: I found that
Block.cast is available: (https://mxnet.incubator.apache.org/api/python/gluon/gluon.html#mxnet.gluon.Block.cast)