How to get individual class accuracy in image-classification on validation set

Discussion
1

I work for practical deployment of cnn model, and I want to check a specific desired class accuracy of my deep-learning model (image-classification) on my validation set. Model is trained on a subset of IMAGENET-2012 dataset. A simple template code like score.py for this might be useful for me.

Thanks

2

When you’re interested in an individual class, precision and recall may be better metrics to look at. You can, however, define accuracy as the number of correctly classified items (correct being defined as “if label is class A, prediction is class A and if label is not class A, prediction is not class A”) divided by the total number of items. I modified mxnet.metric.Accuracy to implement a SingleClassAccuracy() as an example:


class SingleClassAccuracy(mx.metric.EvalMetric):
    def __init__(self, axis=1, name='single_class_accuracy',
                 output_names=None, label_names=None):
        super(SingleClassAccuracy, self).__init__(
            name, axis=axis,
            output_names=output_names, label_names=label_names)
        self.axis = axis

    def update(self, label, preds):
        """Updates the internal evaluation result.

        Parameters
        ----------
        label : int
            The class index

        preds : list of `NDArray`
            Prediction values for samples. Each prediction is a vector of likelihoods for all classes.
        """
        for pred_label in preds:
            pred_label = nd.argmax(pred_label, axis=self.axis)
            pred_label = pred_label.asnumpy().astype('int32')

            self.sum_metric += (pred_label.flat == label).sum()
            self.num_inst += len(pred_label.flat)

acc = SingleClassAccuracy()
acc.update(2, [nd.array([[0, 0, 1],[0, 0, 1], [0, 1, 0], [1, 0, 0]])])
print(acc.get())
1 Like