Does mxnet allow to compute higher order differentiation like in tensorflow, pytorch? Because in their official documentation it’s written that only few operations are available for higher order differentiation.
Support for higher order differentiation is limited at the moment, but coverage is improving.
You should check out the
mxnet.autograd.grad function for this.
x = nd.array([1, 2, 3]) x.attach_grad() with autograd.record(): y = nd.exp(x) y_grad = autograd.grad(y, x, create_graph=True, retain_graph=True) y_grad.backward() print(x.grad)
Simplest way to see if operators are supported is to try it, but you can tell by looking at the backend source code if you’re not scared of C++ code. One sign that an operator supports higher order gradients is that the
FGradient attribute is not registered to a backward operator, e.g. exp. And an indicator that an operator doesn’t support higher order gradients is that the
FGradient attribute is registered to a backward operator, and the backward operator itself doesn’t register another
FGradient function, e.g. sigmoid.
Thanks for answering, it solved my problem.