Hi!
I noticed that the default data type (i.e., the one used when creating arrays without specifying the dtype argument) in MXNet is float32 when I found my loss became nan after several hundred iterations. Now I want to use float64 in stead of float32 to see the precision of the data type was responsible for the nan. However, it seems to me that that the only way to specify float64 as the data type for every array is to explicitly add dtype='float64' to every function call that creates an array from scratch. I wonder if there is a way to replace the default data type.
This reminds me of a similar question Shortcut for enabling GPU concerning context rather than dtype. The two questions put together are simply ‘Is there any way to change the default settings (ctx and dtype) for newly created arrays?’
Thanks!
UPD: I did not find a way to specify the data type of the convolutional layer parameters. Dense and Embedding both have dtype in their constructors, but Conv2D does not.
Below are excerpts from implementations of Dense and Conv2D where the parameters are defined:
Dense:
self.weight = self.params.get('weight', shape=(units, in_units),
init=weight_initializer, dtype=dtype,
allow_deferred_init=True)
if use_bias:
self.bias = self.params.get('bias', shape=(units,),
init=bias_initializer, dtype=dtype,
allow_deferred_init=True)
else:
self.bias = None
Conv2D (actually _Conv, the superclass of Conv2D):
self.weight = self.params.get('weight', shape=wshapes[1],
init=weight_initializer,
allow_deferred_init=True)
if use_bias:
self.bias = self.params.get('bias', shape=wshapes[2],
init=bias_initializer,
allow_deferred_init=True)
else:
self.bias = None
Note that the Dense specifies the data type for its parameters, but Conv2D does not, which in turn means that the parameters can only be of type float32, since Parameter has this default setting when dtype is not specified.
UPD2: I found that Block.cast is available: (https://mxnet.incubator.apache.org/api/python/gluon/gluon.html#mxnet.gluon.Block.cast)