Can we reuse PredictHandles created from MXPredCreateMultiThread? Is there any catch for this like waiting or freeing something?
@szha @eric-haibin-lin would you be able to comment on this one? See this was added a couple of weeks ago in https://github.com/apache/incubator-mxnet/pull/12456. Cheers!
I think reuse is fine. As far as I remember, a symbol executor is created when a PredictHandle is used for the first time. The only requirement is that the PredictHandle has to be used in its own thread.
If I use 10 different PredictorHandles created by
MXPredCreate() in the main thread and for each PredictorHandle calling MXPredSetInput(), MXPredForward() and MXPredGetOutput() functions to inference in 10 threads, Is it safe?
Thread can get one current available PredictorHandle to inference, therefore, one certain thread may get different PredictorHandle to inference. Is this safe?