Out of Memory when using gluoncv.data.COCODetection

I followed the example to load the COCO data sets as follows:

from gluoncv import data, utils
train_dataset = data.COCODetection(splits=['instances_train2017'])
val_dataset = data.COCODetection(splits=['instances_val2017'])

The train_dataset is about 117266 images and labels.
Now I am simply trying to loop through the data-set:

for i in range(len(train_dataset)):
    train_image, train_label = train_dataset[i]

This loop simply eats up all of my 15Gb of RAM and I am not even 60% of data.

How can I make sure that my memory is used properly or how can i fix a batch size etc?