How To Visualize Image Classification (ResnetV2)

1

I am using the modelzoo resnetv2 model for training a classification problem. There are some misclassified cases and I would like to see what areas of the image are most strongly related to the misclassification.

Does anything exist within Mxnet or Gluon for determining this? I’ve found some github examples in other frames, such as: https://github.com/metalbubble/CAM.

2

There is an mxnet implement for gradcam available here

1 Like
3

This site might be helpful too:

4

Sorry, this site is more helpful, I think.

5

Try this code.
This is a mxnet code for CAM (not for Gradcam).

import numpy as np
import mxnet as mx
from mxnet import image
from gluoncv import utils
from gluoncv.model_zoo import get_model
from mxnet.gluon.data.vision import transforms
import cv2

url = 'https://raw.githubusercontent.com/dmlc/web-data/master/gluoncv/segmentation/voc_examples/1.jpg'
filename = 'example.jpg'
utils.download(url, filename)

#net = get_model('resnet18_v1', pretrained=True)
net = get_model('resnet18_v2', pretrained=True)

params_list = list(net.collect_params())
weight_softmax = net.collect_params()[params_list[-2]].data().asnumpy()

def returnCAM(feature_conv, weight_softmax, idx):
    # generate the class activation maps upsample to 256x256
    size_upsample = (256, 256)
    bz, nc, h, w = feature_conv.shape
    cam = weight_softmax[idx].dot(feature_conv.reshape((nc, h*w)))
    cam = cam.reshape(h, w)
    cam = cam - np.min(cam)
    cam_img = cam / np.max(cam)
    cam_img = np.uint8(255 * cam_img)
    return cv2.resize(cam_img, size_upsample)
    
transform = transforms.Compose([
    transforms.Resize((224,224)),
    transforms.ToTensor(),
    transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
])

img = image.imread(filename)
img = transform(img)
img = img.expand_dims(0)

output = net(img)

classes = net.classes

idx = mx.nd.argmax(output).asscalar().astype('int32')

features_blob = net.features[:-4](img).asnumpy()

CAMs = returnCAM(features_blob, weight_softmax, idx)

print('output CAM.jpg for the top1 prediction: %s'%classes[idx])
img = cv2.imread(filename)
height, width, _ = img.shape
heatmap = cv2.applyColorMap(cv2.resize(CAMs,(width, height)), cv2.COLORMAP_JET)
result = heatmap * 0.3 + img * 0.5
cv2.imwrite('CAM.jpg', result)

output CAM.jpg for the top1 prediction: beer bottle
CAM

2 Likes
6

Thanks for all the resources! I’ve been working on another problem but will try this out next week!

7

Hi @mikeobr,

maybe I’m wrong, but you wanted to use CAM on a self-trained model, isn’t it?
Do you have any experience on what you have achieved so far?

I have tried to use @dai-ichiro solution, which is working fine for modles out of the modle zoo, but for an exported self-trained networking i run into many issues…

Whould be nice if anyone can give me some advice (or more info/links) here :slight_smile:
Thanks a lot!

8

Hey @RedTo,

I was using ResNet from the model zoo.

I don’t know this stuff very well, but for self-made models I think you’ll need to find the right layer and use that instead of the hardcoded:

net.features[:-4](img).asnumpy()