Multiple output layers and multiple losses handling

Discussion
1

Hi, I wanted a network that can take in data and have 2 sets of predictions. My code is written in Gluon. In my custom block, I simply added another Dense layer, and return the two results.
For example, in my custom block,

self.out1 = gluon.nn.Dense(5)
self.out2 = gluon.nn.Dense(19)

and in forward:

x1 = self.out1(x)
x2 = self.out2(x)
return x1, x2

I created 2 loss functions with softmax loss like this:

loss_func_1 = gluon.loss.SoftmaxCrossEntropyLoss()
loss_func_2 = gluon.loss.SoftmaxCrossEntropyLoss()

And in back propagation:

loss = loss_1 + loss_2
loss.backward()

This is not giving me meaningful result. Can anyone tell me the right way to handle the multiple outputs and back propagation?

2

The codes seem fine with me. However, you may need a hyperparameter to balance the two objectives and tune the hyperparameter until you are happy with the results.

3

Thanks for your reply! My problem is actually solved.