Limited bit depth of the recordIO iterator

Discussion
1

Hi, currently I’m working on the medical imaging project and I’m trying to use the recordIO .rec files to pack the data. In this field it is common to have grayscale images with a pretty high dynamic range, therefore most of the data are encoded with 12 or 16 bits per pixel. It would be nice, of course, to preserve the original image bit-depth for certain U-Net applications.

It is possible in general to pack the 16-bit PNG files in the .rec container. The problem is that mxnet ImageRecordIter seems to convert all data into 8-bit format internally throwing away the information I want to preserve. Changing dtype parameter value does not help (does it only affect labels data format?).

At the first glance at


it seems like the problem arises from the hardcoded specification of CV_8U format (equivalent of unsigned int 8) in the line 544:

cv::Mat buf(1, rec.content_size, CV_8U, rec.content);

My questions are:

  1. Am I missing some way of loading 16-bit data from .rec files?
  2. If not, will it be possible to add a support for higher bit-depths into ImageRecordIter?

Thank you in advance.

2

Hi @Pavel,

I had a similar experience and was able to write a Gluon Dataset to do this. My dataset used OpenCV to read in the image in 16-bit and perform rotation augmentations. Use the following to avoid conversion to 8-bit:

cv2.imread(filepath, -1)

When using OpenCV in the dataset, and using muliple workers in the DataLoader I sometimes had issues (depending on the OpenCV functions I used). As a fix for this I used threading instead of multi-processing.

dataloader = mx.gluon.data.DataLoader(dataset,
                                      batch_size,
                                      shuffle=True,
                                      num_workers=8,
                                      thread_pool=True)

Overall the perfomance for my case wansn’t too different to with ImageRecordIter, and was far from being the bottleneck for my training process. I hope that helps in some way!

Cheers, Thom

3

hi @thomelane, do you have any idea how to modify im2rec to parse 16 bit PNG?

4

Hi @thomelane,

thank you for your reply!

Unfortunately, our project is written in R, so no gluon magic is available there. But I might use your trick if we port it to python.

At least it is clear now that there is no way around but creating a custom data loading routine or modifying mxnet core itself. I will try to issue a feature request since 16 bit data support is important for medical imaging community and should be implemented natively.