Check failed: !IsMKLDNNData() We can't generate TBlob for MKLDNN data. Please use Reorder2Default() to generate a new NDArray first

Discussion
1

I’m using Deformable-Convnets (https://github.com/msracver/Deformable-ConvNets) and able to run its demo.py with no problem on the regular MXNet build.

demo.py is at https://github.com/msracver/Deformable-ConvNets/blob/master/rfcn/demo.py

I now built MXNet with MKLDNN=1 and running into this error when I run demo.py

    [23:48:46] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 7411200 bytes with malloc directly
[23:48:51] src/operator/nn/mkldnn/mkldnn_base.cc:72: Allocate 18874368 bytes with malloc directly
Traceback (most recent call last):
  File "rfcn/demo.py", line 129, in <module>
    main()
  File "rfcn/demo.py", line 98, in main
    scores, boxes, data_dict = im_detect(predictor, data_batch, data_names, scales, config)
  File "/Users/pixm/Deformable-ConvNets/rfcn/core/tester.py", line 132, in im_detect
    rois = output['rois_output'].asnumpy()[:, 1:]
  File "/usr/local/lib/python2.7/site-packages/mxnet-1.2.0-py2.7.egg/mxnet/ndarray/ndarray.py", line 1894, in asnumpy
    ctypes.c_size_t(data.size)))
  File "/usr/local/lib/python2.7/site-packages/mxnet-1.2.0-py2.7.egg/mxnet/base.py", line 210, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [23:48:51] src/ndarray/ndarray.cc:757: Check failed: !IsMKLDNNData() We can't generate TBlob for MKLDNN data. Please use Reorder2Default() to generate a new NDArray first

Stack trace returned 10 entries:
[bt] (0) 0   libmxnet.so                         0x0000000113356da5 dmlc::StackTrace() + 261
[bt] (1) 1   libmxnet.so                         0x0000000113356b5f dmlc::LogMessageFatal::~LogMessageFatal() + 47
[bt] (2) 2   libmxnet.so                         0x000000011478b986 mxnet::NDArray::SetTBlob() const + 1190
[bt] (3) 3   libmxnet.so                         0x00000001134ae197 mxnet::op::custom::AllocateNDArrayCopy(mxnet::NDArray**, std::__1::vector<mxnet::NDArray, std::__1::allocator<mxnet::NDArray> > const&, unsigned long, int) + 343
[bt] (4) 4   libmxnet.so                         0x00000001134ac805 mxnet::op::custom::ForwardEx(mxnet::OpStatePtr const&, mxnet::OpContext const&, std::__1::vector<mxnet::NDArray, std::__1::allocator<mxnet::NDArray> > const&, std::__1::vector<mxnet::OpReqType, std::__1::allocator<mxnet::OpReqType> > const&, std::__1::vector<mxnet::NDArray, std::__1::allocator<mxnet::NDArray> > const&) + 261
[bt] (5) 5   libmxnet.so                         0x000000011469329c std::__1::__function::__func<mxnet::exec::GraphExecutor::InitCachedOps()::$_6, std::__1::allocator<mxnet::exec::GraphExecutor::InitCachedOps()::$_6>, void (mxnet::RunContext, mxnet::engine::CallbackOnComplete)>::operator()(mxnet::RunContext&&, mxnet::engine::CallbackOnComplete&&) + 92
[bt] (6) 6   libmxnet.so                         0x000000011465b535 mxnet::engine::ThreadedEngine::ExecuteOprBlock(mxnet::RunContext, mxnet::engine::OprBlock*) + 533
[bt] (7) 7   libmxnet.so                         0x000000011465eba1 void std::__1::__invoke_void_return_wrapper<void>::__call<mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*, bool)::'lambda'()::operator()() const::'lambda'(std::__1::shared_ptr<dmlc::ManualEvent>)&, std::__1::shared_ptr<dmlc::ManualEvent> >(mxnet::engine::ThreadedEnginePerDevice::PushToExecute(mxnet::engine::OprBlock*, bool)::'lambda'()::operator()() const::'lambda'(std::__1::shared_ptr<dmlc::ManualEvent>)&&&, std::__1::shared_ptr<dmlc::ManualEvent>&&) + 145
[bt] (8) 8   libmxnet.so                         0x000000011465c573 void* std::__1::__thread_proxy<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct> >, std::__1::function<void (std::__1::shared_ptr<dmlc::ManualEvent>)>, std::__1::shared_ptr<dmlc::ManualEvent> > >(void*) + 83
[bt] (9) 9   libsystem_pthread.dylib             0x00007fff7b1ff661 _pthread_body + 340

Any help is really appreciated.

2

This issue has been fixed by Zheng Da - https://github.com/apache/incubator-mxnet/pull/11005/files