Parameter initialization error in customization of conv2D kernel with ndarray tensor

I insert a customized variant of conv2D into the stem of a net with initialization of filter kernel and then following error occurs.

All Parameters must be initialized on the same set of contexts, but Parameter sonicnet0_firstconv_weight is initialized on [gpu(0), gpu(1)] while previous Parameters are initialized on [cpu(0)].

class MeanShift(nn.Conv2D):
    def __init__(
        self, rgb_range,
        rgb_mean=(0.4488, 0.4371, 0.4040), rgb_std=(1.0, 1.0, 1.0), sign=-1):
        super(MeanShift, self).__init__(3,kernel_size=1,in_channels=3)
        std = nd.array(rgb_std)
#         conv = mx.gluon.nn.Conv2D(channels=1, kernel_size=(3,3), padding=(1,1))
#         conv.initialize()
#         conv.weight.set_data(weight)
        self.weight.set_data(nd.eye(3).reshape(3, 3, 1, 1) / std.reshape(3, 1, 1, 1))
        self.bias.set_data(sign * rgb_range * nd.array(rgb_mean) / std)

as loading model on two GPU, specifically how could I do for the purpose of initializing the block.
I would like to could use net.initialize() without cover the parameters before.