How can I compute the accuracy for a multi-lable dataset?

I am wondering how should I get the accuracy for a multi-lable classification task?
Can someone please provide a toy example?


one option is to treat is as a case of computing the accuracy for each label that that your input is assigned to.

if you have 3 labels a, b, c represented as the columns in the example below.

>>> labels = [mxnet.ndarray([[1, 0, 1], [0, 1, 1], [1, 1, 1]])]
>>> outputs = [mxnet.nd.array([[0.8, 0.005, 0.3], [0.02, 0.97, 0.8], [0.6, 0.6, 0.75]])) 

for each row in outputs you can calculate the accuracy per label, using some threshold on your output.
so you have 8/9 accuracy rate

alternatively you can calculate accuracy per entry and force an exact match. in the example above you’d have 2/3 accuracy rate instead.

which of this to do depends on your tasks and whether you care about getting all applicable labels for a particular input or getting non-exhaustive but applicable labels to your inputs. to do either of this, you can create custom metrics in mxnet using mxnet.metric.create.