Hi @NRauschmayr, thank you very much for replying to my post
I have tried replacing a resnet_v18 model with the 2 links you have given, and could verify in the model printout that the convolutional layers were replaced.
However, with the SSD_resnet50 model i am using, i could not verify that a replacement of the convolutional layers has occurred.
Before Replacing Conv2D with DeformableConvolution in ResNet(truncated for readability)
ResNetV1(
(features): HybridSequential(
(0): Conv2D(3 -> 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=64)
(2): Activation(relu)
(3): MaxPool2D(size=(3, 3), stride=(2, 2), padding=(1, 1), ceil_mode=False, global_pool=False, pool_type=max, layout=NCHW)
)
After Replacing Conv2D with DeformableConvolution in ResNet(truncated for readability)
ResNetV1(
(features): HybridSequential(
(0): DeformableConvolution(None → 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3))
(1): BatchNorm(axis=1, eps=1e-05, momentum=0.9, fix_gamma=False, use_global_stats=False, in_channels=64)
(2): Activation(relu)
(3): MaxPool2D(size=(3, 3), stride=(2, 2), padding=(1, 1), ceil_mode=False, global_pool=False, pool_type=max, layout=NCHW)
Before replacing Conv2D with Deformable Convolution in SSD_resnet50(truncated and compacted for readability):
SSD(
(features): FeatureExpander(
<Symbol group [ssd0_resnetv10_stage3_activation5, ssd0_resnetv10_stage4_activation2, ssd0_expand_reu0, ssd0_expand_reu1, ssd0_expand_reu2, ssd0_expand_reu3]> : 1 -> 6
)
(class_predictors): HybridSequential(
(0): ConvPredictor(
(predictor): Conv2D(1024 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
)
(1): ConvPredictor((predictor): Conv2D(2048 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(2): ConvPredictor((predictor): Conv2D(512 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(3): ConvPredictor((predictor): Conv2D(512 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(4): ConvPredictor((predictor): Conv2D(256 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(5): ConvPredictor((predictor): Conv2D(256 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
)
(box_predictors): HybridSequential(
(0): ConvPredictor((predictor): Conv2D(1024 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(1): ConvPredictor((predictor): Conv2D(2048 -> 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(2): ConvPredictor((predictor): Conv2D(512 -> 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(3): ConvPredictor((predictor): Conv2D(512 -> 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(4): ConvPredictor((predictor): Conv2D(256 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(5): ConvPredictor((predictor): Conv2D(256 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
After replacing Conv2D with Link 1
SSD(
(features): FeatureExpander(
<Symbol group [ssd0_resnetv10_stage3_activation5, ssd0_resnetv10_stage4_activation2, ssd0_expand_reu0, ssd0_expand_reu1, ssd0_expand_reu2, ssd0_expand_reu3]> : 1 -> 6
)
(class_predictors): HybridSequential(
(0): ConvPredictor(
(predictor): Conv2D(1024 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
)
(1): ConvPredictor((predictor): Conv2D(2048 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(2): ConvPredictor((predictor): Conv2D(512 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(3): ConvPredictor((predictor): Conv2D(512 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(4): ConvPredictor((predictor): Conv2D(256 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(5): ConvPredictor((predictor): Conv2D(256 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
)
(box_predictors): HybridSequential(
(0): ConvPredictor((predictor): Conv2D(1024 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(1): ConvPredictor((predictor): Conv2D(2048 -> 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(2): ConvPredictor((predictor): Conv2D(512 -> 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(3): ConvPredictor((predictor): Conv2D(512 -> 24, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(4): ConvPredictor((predictor): Conv2D(256 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(5): ConvPredictor((predictor): Conv2D(256 -> 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
As observed, there is no difference. I cannot verify if the ResNet feature extractor had its convolutional layers replaced.
After replacing with Conv2D with Link 2(truncated and compacted for readability):
features FeatureExpander(
<Symbol group [ssd2_resnetv10_stage3_activation5, ssd2_resnetv10_stage4_activation2, ssd2_expand_reu0, ssd2_expand_reu1, ssd2_expand_reu2, ssd2_expand_reu3]> : 1 -> 6
)
Recursing instead
class_predictors HybridSequential(
(0): ConvPredictor((predictor): Conv2D(1024 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(1): ConvPredictor((predictor): Conv2D(2048 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(2): ConvPredictor((predictor): Conv2D(512 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(3): ConvPredictor((predictor): Conv2D(512 -> 12, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(4): ConvPredictor((predictor): Conv2D(256 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
(5): ConvPredictor((predictor): Conv2D(256 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
)
Recursing instead
0 ConvPredictor((predictor): Conv2D(1024 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)))
Recursing instead
predictor Conv2D(1024 -> 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
replacing layer
---------------------------------------------------------------------------
TypeError
Traceback (most recent call last)
<ipython-input-13-14db52908be1> in <module>()
28 replace_conv2D(layer)
29
---> 30 replace_conv2d(net)
31 #print("After")
32 #print(net)
4 frames
/usr/local/lib/python3.6/dist-packages/mxnet/gluon/block.py in __setattr__(self, name, value)
197 raise TypeError('Changing attribute type for {name} from {type1} to {type2}' \
198 'is not allowed.'.format(
--> 199 name=name, type1=type(existing), type2=type(value)))
200
201 if isinstance(value, Block):
TypeError: Changing attribute type for predictor from <class 'mxnet.gluon.nn.conv_layers.Conv2D'> to
<class 'mxnet.gluon.contrib.cnn.conv_layers.DeformableConvolution'>is not allowed.
It seems that the predictor blocks do not allow replacement, but i am fine with it. However i still cannot confirm if the convolutational layers in the feature extraction block are replaced with their deformable counterparts