Fine-tune object detectors on custom dataset

, ,
1

Hi,
I have some question about fine-tuning gluoncv object detectors:
1- Are following two versions equal?
net = gcv.model_zoo.get_model(‘ssd_512_resnet50_v1_voc’, pretrained=True)
net.reset_class(classes)
and
net = gcv.model_zoo.get_model(‘ssd_512_resnet50_v1_custom’, classes=classes, pretrained_base=True, transfer=‘voc’)

Which weights will be used? imagenet or voc?
When I train it, all the weights will change? Or they are frozen?

2- when we set pretrained_base=True and transfer=‘voc’, what happen? Are the weights from imagenet or voc? Which one has priority?

3- Is there any code for faster rcnn code to fine tune the model on custom data set?
Thank you

2

@kargarisaac,
1- yes they are equivalent, see https://github.com/dmlc/gluon-cv/blob/d7553f3e31804ee1113fd1bfcecb009ba25360bb/gluoncv/model_zoo/ssd/ssd.py#L421

In case you haven’t found it yet, this is a tutorial on how to fine-tune the SSD model:
https://gluon-cv.mxnet.io/build/examples_detection/finetune_detection.html

The weights are frozen or not depending on what gets passed to the Trainer object. (or alternatively based on the grad_req of the parameters)

trainer = gluon.Trainer(
    net.collect_params(), 'sgd',
    {'learning_rate': 0.001, 'wd': 0.0005, 'momentum': 0.9})

This gives all parameters to the trainer.

You can use a regex to select only certain parameters:

print(net.collect_params())
ssd1_ (
  Parameter ssd1_mobilenet0_conv0_weight (shape=(32, 3, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm0_gamma (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm0_beta (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv1_weight (shape=(32, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm1_gamma (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm1_beta (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv2_weight (shape=(64, 32, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm2_gamma (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm2_beta (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv3_weight (shape=(64, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm3_gamma (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm3_beta (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv4_weight (shape=(128, 64, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm4_gamma (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm4_beta (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv5_weight (shape=(128, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm5_gamma (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm5_beta (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv6_weight (shape=(128, 128, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm6_gamma (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm6_beta (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv7_weight (shape=(128, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm7_gamma (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm7_beta (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv8_weight (shape=(256, 128, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm8_gamma (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm8_beta (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv9_weight (shape=(256, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm9_gamma (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm9_beta (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv10_weight (shape=(256, 256, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm10_gamma (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm10_beta (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv11_weight (shape=(256, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm11_gamma (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm11_beta (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv12_weight (shape=(512, 256, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm12_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm12_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv13_weight (shape=(512, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm13_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm13_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv14_weight (shape=(512, 512, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm14_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm14_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv15_weight (shape=(512, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm15_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm15_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv16_weight (shape=(512, 512, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm16_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm16_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv17_weight (shape=(512, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm17_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm17_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv18_weight (shape=(512, 512, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm18_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm18_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv19_weight (shape=(512, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm19_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm19_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv20_weight (shape=(512, 512, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm20_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm20_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv21_weight (shape=(512, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm21_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm21_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv22_weight (shape=(512, 512, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm22_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm22_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv23_weight (shape=(512, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm23_gamma (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm23_beta (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv24_weight (shape=(1024, 512, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm24_gamma (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm24_beta (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv25_weight (shape=(1024, 1, 3, 3), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm25_gamma (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm25_beta (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_conv26_weight (shape=(1024, 1024, 1, 1), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm26_gamma (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm26_beta (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_conv0_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn0_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn0_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_conv0_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn0_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn0_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_conv1_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn1_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn1_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_conv1_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn1_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn1_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_conv2_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn2_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn2_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_conv2_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn2_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn2_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_conv3_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn3_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn3_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_conv3_weight (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn3_gamma (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn3_beta (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm0_running_mean (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm0_running_var (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm1_running_mean (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm1_running_var (shape=(32,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm2_running_mean (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm2_running_var (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm3_running_mean (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm3_running_var (shape=(64,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm4_running_mean (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm4_running_var (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm5_running_mean (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm5_running_var (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm6_running_mean (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm6_running_var (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm7_running_mean (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm7_running_var (shape=(128,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm8_running_mean (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm8_running_var (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm9_running_mean (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm9_running_var (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm10_running_mean (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm10_running_var (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm11_running_mean (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm11_running_var (shape=(256,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm12_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm12_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm13_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm13_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm14_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm14_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm15_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm15_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm16_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm16_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm17_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm17_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm18_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm18_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm19_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm19_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm20_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm20_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm21_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm21_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm22_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm22_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm23_running_mean (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm23_running_var (shape=(512,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm24_running_mean (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm24_running_var (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm25_running_mean (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm25_running_var (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm26_running_mean (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_mobilenet0_batchnorm26_running_var (shape=(1024,), dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn0_moving_mean (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn0_moving_var (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn0_moving_mean (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn0_moving_var (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn1_moving_mean (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_trans_bn1_moving_var (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn1_moving_mean (shape=None, dtype=<class 'numpy.float32'>)
  Parameter ssd1_expand_bn1_moving_var (shape=None, dtype=<class 'numpy.float32'>)
...
)

For example this trainer only get the paramaters related to the mobilenet front end. Etc

trainer = gluon.Trainer(
    net.collect_params(".*mobilenet.*"), 'sgd',
    {'learning_rate': 0.001, 'wd': 0.0005, 'momentum': 0.9})

2- This will use pretrained weights from VOC, ImageNet is the dataset for image classification that was used for pretraining the CNN frontend, mobilenet or ResNet for example.

3- You can refer to the training tutorial https://gluon-cv.mxnet.io/build/examples_detection/train_faster_rcnn_voc.html and full training script for inspiration: https://gluon-cv.mxnet.io/_downloads/6549a6e25b7eeee585a8d2737aac64f0/train_faster_rcnn.py

1 Like