Checking if CuDnn is actually used?

Discussion
1

Hi!

Do you know if there is any way to check if an operation (e.g. nd.Convolution) was actually done by the GPU, and was actually done using CuDNN?

Many thanks,
AL.

2

nvprof can help you check what kernels were actually called

3

I will explore nvprof.
Thanks a lot!

4

@zhreshold Could you give hints to show how to check cudnn by nvprof? I am not familiar with it.

Is it possible to check it directly in mxnet? For example, we can do it simply in pytorch

torch.backends.cudnn.version()

Thanks

5

nvprof python (python-scripy).py
will directly profile the performance of your program.

6

A nice lightweight mechanism is to enable CuDNN API logging as well.

Set
CUDNN_LOGINFO_DBG=1

and
CUDNN_LOGDEST_DBG=stdout