Cannot predict with mx.gpu() simply, how can I load my model into GPU with mx.nd.load method[Solved]

my code is quite simple, load a model, predict a image.

img_inputND = mx.nd.array(mxnet_img,ctx=mx.cpu(0))

nd_["Input"]= img_inputND

e_ = sym_.bind(mx.cpu(0), nd_)

out_ = e_.forward()

It’s ok in CPU.

1@

img_inputND = mx.nd.array(mxnet_img,ctx=mx.gpu(0))

nd_["Input"]= img_inputND

e_ = sym_.bind(mx.cpu(0), nd_)

out_ = e_.forward()

mxnet.base.MXNetError: [21:13:59] src/executor/…/common/exec_utils.h:516: Check failed: x == default_ctx Input array is in gpu(0) while binding with ctx=cpu(0). All arguments must be in global context (cpu(0)) unless group2ctx is specified for cross-device graph.

2@

img_inputND = mx.nd.array(mxnet_img,ctx=mx.cpu(0))

nd_["Input"]= img_inputND

e_ = sym_.bind(mx.gpu(0), nd_)

out_ = e_.forward()

mxnet.base.MXNetError: [21:14:33] src/executor/…/common/exec_utils.h:516: Check failed: x == default_ctx Input array is in cpu(0) while binding with ctx=gpu(0). All arguments must be in global context (gpu(0)) unless group2ctx is specified for cross-device graph.

3@Here I changed ctx to be gpu(0) in two places, but the error information tells me one is in cpu context.

img_inputND = mx.nd.array(mxnet_img,ctx=mx.gpu(0))

nd_["Input"]= img_inputND

e_ = sym_.bind(mx.gpu(0), nd_)

out_ = e_.forward()

mxnet.base.MXNetError: [21:15:08] src/executor/…/common/exec_utils.h:516: Check failed: x == default_ctx Input array is in cpu(0) while binding with ctx=gpu(0). All arguments must be in global context (gpu(0)) unless group2ctx is specified for cross-device graph.

My Nvidia-SMI is OK, I’m on ubuntu 16.04 with cuda 9 and my mxnet version is 1.3.1 with gpu.
mx.test.utils.list_gpus() returns [0]
My GPU is OK, If I change another load model method, load_checkpoint or gluon, it’s ok to predict with GPU.

Hi @HyperGroups,

Are you running this code sequentially as you’ve written it above? You’ve defined the new data on GPU but you’ve still got an old reference in nd_ to the data in CPU. So you’re getting the same error as before.

Hi, @thomelane I’ve modified a bit, the codes in four places are the same, there are 4 lines.

(post edited to remove the HTML quote entities)

Where are you initializing nd_? Would you be able to include the code for that?

Thanks!
Vishaal

I load the model of nd_=mx.nd.load(file_nd)
I’ve add the model here
model-json
model-params

@VishaalKapoor

I found the cause, that is I my data in nd_ in not all in GPU mode.
When I manually set my data into GPU mode, then I can run my code with GPU.

I set it by iterate my nd_

I wonder is there official method, like
mx.nd.load(model, ctx=mx.gpu(0))
or
nd_.copy_to_gpu(mx.gpu(0))

Apologies for not replying to your earlier post sooner, I had suspected that nd_ was in RAM and was curious to see it’s intialization code. I should have mentioned that was why I was curious to see the code. Great that you figured it out! :slight_smile:

Vishaal