Does mxnet allow to compute higher order differentiation like in tensorflow, pytorch? Because in their official documentation it’s written that only few operations are available for higher order differentiation.

Support for higher order differentiation is limited at the moment, but coverage is improving.

You should check out the `mxnet.autograd.grad`

function for this.

```
x = nd.array([1, 2, 3])
x.attach_grad()
with autograd.record():
y = nd.exp(x)
y_grad = autograd.grad(y, x, create_graph=True, retain_graph=True)[0]
y_grad.backward()
print(x.grad)
```

Simplest way to see if operators are supported is to try it, but you can tell by looking at the backend source code if you’re not scared of C++ code. One sign that an operator supports higher order gradients is that the `FGradient`

attribute is not registered to a backward operator, e.g. exp. And an indicator that an operator doesn’t support higher order gradients is that the `FGradient`

attribute is registered to a backward operator, and the backward operator itself doesn’t register another `FGradient`

function, e.g. sigmoid.

2 Likes

Thanks for answering, it solved my problem.