I’m hoping for some clarification on lazy evaluation: https://mxnet.incubator.apache.org/versions/master/tutorials/basic/ndarray.html#lazy-evaluation-and-automatic-parallelization
This use is for inference, consider I run 50 samples of size (1, 3,256,256) in a for loop, then later I retrieve the results. Lazy evaluation seems to say nothing is executed until I actually need the output. Is this the equivalent of running a single (50, 3, 256, 256) through the net?
#Dummy pseudocode for sample in samples: net_out = my_net(sample) mx.nd.concat(output, net_out, dim=0) #Operations are lazily evaluated here operation_output = output.asnumpy() # Format samples in size (50,3,256,256) net_out = my_net(samples) # Is this quicker than above? operation_output = output.asnumpy()
To clarify, my question is: Does lazy evaluation mean MXNet is automatically batching input?