Can data loader work with different input shape

Gluon
1

I have been trying to load data using dataloader.

However, if I pass different shaped input for each individual record than it returns shape mismatch error. There has been an open issue in the github community regarding this since last 3 months. However, no one has even responded to the issue and Documentation is bit vague on this topic regarding what can work and what cannot.

If anyone can guide me with this it would be great.

If dataloader doesn’t work with different shaped input, that what are the alternative to this? I have heard bucketing but not sure if that is on the right path or not.

2

@aakashpatel can you link the github issue you are referring to?

The DataLoader is very customizable and supports a wide range of use-cases through custom batchifier and sampler.

What do you mean by “different input shape” ? What do you expect your output to be ? Are you talking images or text data ?

The standard DataLoader returns a NDArray of size (N, ...) where N is your batch size. a NDArray is Tensor, you cannot have a tensor with some dimensions shorter than others, think of it as a square that would some rows shorter than others.

Now you have some solutions for different shape input, the simplest one is padding. For example for NLP use-cases you can have a maximum input length after what the input is truncated, and any shorter input can be padded with 0 up to that maximum length. Such padding would happen in a transform function applied to the Dataset that is passed to the Dataloader.

To optimize the speed of training you can use bucketing, to put input of similar length into buckets. Have a look at gluon-nlp implementation of a bucketting data loader.

3

https://github.com/zackchase/mxnet-the-straight-dope/issues/506.

I have different shaped image and bounding box list. should I pad it? I mean padding it doesn’t make sense in this case. So I was wondering what would be an optimal solution.

So are you saying all people training various object detection networks such as Faster-RCNN and SSD use data loaders that have padded data?

4

See my answer here:

1 Like