I tried running densenet121 from the gluon model zoo for the first time since upgrading to 1.5.0 this is a standard pip3 install, I have not built anything on this computer. If I set ignore_stale_grad=True. It doesn’t work well and theirs a strange imbalance between gpu usage one is running at 80% and the the other at 30%. The data is a standard rec file made from caltech256. I have had this notebook since version 1.4.1 came out and have never had any problems. I should note that I do create a kvstore device and set kv=kvstore in the trainer.