Hello, I have a problem where I want to use
with a list of heads. For that to work, all entries in heads must be part of a graph built up in autograd.record(), or must have a gradient attached directly.
Is there a way to find out whether this situation holds for an NDArray object? I can track this myself, sure, but my code would be so much cleaner.
To give context:
a = mx.nd.zeros((1,)) a.attach_grad() with autograd.record(): b = 2 * a b.backward()
This works fine. But:
a = mx.nd.zeros((1,)) with autograd.record(): b = 2 * a b.backward()
This fails, and tells me that b cannot be differentiated. I totally understand that, but:
Is there a way to find out whether b is not amenable for backward, except for trying it out?
For me, this is not even a sane option, because I want to use
with a whole list of heads and head_gradients.
First, I thought that
b.grad is not None
does the job. But this does not work. For example:
a = mx.nd.zeros((1,) a.attach_grad() with autograd.record(): b = 2 * a a.grad is None ==> False b.grad is None ==> True
So this test just tells me whether I myself have attached a gradient. I suppose this is due to lazy creating of the grad variable, it exists after I have called b.backward(), but that is too late.
Thanks a lot for enlightment. I just could not find it in the NDArray API docs