Since mx.sym.contrib.SyncBatchNorm is available for several months, it still lacks a detailed example of how to use it. As far as I know, there is no non-gluon user successfully use this API. For module user, any suggestion?
Here is an example of SyncBatchNorm used in gluon-cv. To use the same technique with Module, you simply need to use the sym.SyncBatchNorm variant of the operator and stack it on top of your convolution, similar to how it’s done in a sequential block in Gluon. If you provide your example, I can tell you if it’s done correctly or not. Should be very straight forward, just like using BatchNorm.