Follow your suggestion, I have tryed to use OpenCV for displaying, but its speed seems not faster than the given Gluoncv function (gcv.utils.viz.plot_bbox).
num_s = [i for i in scores[0] if i>0.6]
np_boxes = bounding_boxes[0].asnumpy()
for j in range(len(num_s)):
boxes = np_boxes[j]
cv2.rectangle(frame, (boxes[0],boxes[1]), (boxes[2],boxes[3]), (0,0,255), 2)
cv2.namedWindow('image', cv2.WINDOW_NORMAL)
cv2.imshow('image', frame)
cv2.waitKey(1)
The above step just draws the bounding boxes, not showing different colors of boxes and scores.