Writing mxnet.ndarray to disk in recordio format

Discussion
1

I am trying to write a simple serializer/deserializer for writing label arrays to disk. I cannot find an example for writing mxnet ndarrays to disk in recordio format which I can load back using the data iterators.

The code below will not work as record.write() function expects a char buffer. What is the best way to write such a serializer ?

import mxnet as mx

if __name__=='__main__':
    record = mx.recordio.MXRecordIO('/tmp/tmp-recordio.rec', 'w')
    for i in range(50000):
        arr = mx.ndarray.uniform(low=0, high=1, shape=(1,480,80,80))
        record.write(arr)
    record.close()
2

Hi there, way too long after your question, but I had the exact same one, so I am leaving the solution here for reference to other users. You can use pickle to create a string describing the mxnet nd.array:

import pickle 
import mxnet as mx
record = mx.recordio.MXIndexedRecordIO(idx_path='../temp/test_train.idx',uri='../temp/test_train.rec', flag='w')
for idx in range(1000):
    arr = mx.ndarray.uniform(low=0, high=1, shape=(1,480,80,80))
    arr = pickle.dumps(arr)
    record.write_idx(idx,arr)
record.close()

if you want to read the array:

record=mx.recordio.MXIndexedRecordIO(idx_path='../temp/test_train.idx',uri='../temp/test_train.rec', flag='r')

for key in record.keys:
    arr = record.read_idx(key)
    arr = pickle.loads(arr) # restore the string to mx.nd.array
    break # this will break the for loop