Hello all,
I’m trying to verify that my .rec is correctly formatted, since I’m getting errors when using AWS Sagemaker (“Expected number of batches: 19, did not match the number of batches processed: 20. This may happen when some images or annotations are invalid and cannot be parsed. Please check the dataset and ensure it follows the format in the documentation”).
To verify the .rec, I tried running the following code:
import mxnet as mx
data_iter = mx.image.ImageIter(1, (3, 224, 224), path_imgrec=‘data/output/validation.rec’)
for img in data_iter:
print(img)
However, I get the following error:
Traceback (most recent call last): File “inspect_recordio.py”, line 11, in
for img in data_iter:
File /dev/lib/python3.5/site-packages/mxnet/io/io.py", line 228, in next
return self.next()
File /dev/lib/python3.5/site-packages/mxnet/image/image.py", line 1266, in next
i = self._batchify(batch_data, batch_label)
File /dev/lib/python3.5/site-packages/mxnet/image/image.py", line 1243, in _batchify
batch_label[i] = label
File /dev/lib/python3.5/site-packages/mxnet/ndarray/ndarray.py", line 444, in setitem
self._set_nd_basic_indexing(key, value)
File /dev/lib/python3.5/site-packages/mxnet/ndarray/ndarray.py", line 758, in _set_nd_basic_indexing
value_nd = self._prepare_value_nd(value, vshape)
File /dev/lib/python3.5/site-packages/mxnet/ndarray/ndarray.py", line 676, in _prepare_value_nd
value_nd = value_nd.broadcast_to(vshape)
File /dev/lib/python3.5/site-packages/mxnet/ndarray/ndarray.py", line 1743, in broadcast_to
raise ValueError(err_str)
ValueError: operands could not be broadcast together with remapped shapes[original->remapped]: (9,) and requested shape (1,)
My validation.lst looks like this: https://pastebin.com/4rFwdRbW
Could you give me some pointers on how to solve this issue, or how to validate my .rec files?
Thank you!