assuming I have a set of record files with their indices, e.g.
train1.rec, train1.idx, train2.rec, train2.idx what is the most efficient way to join them together in a single
I have a large set of large raster files ~ 300 x (3 x 10k x 10k), and I want to process them (extract training chips for segmentation) in a distributed environment, to create fast a single rec file. I can process in parallel the extraction from a single file (with locks) but then am bound in serially processing all 300 images one by one - for a single rec file (am actually trying to hack this, but not there yet). Any ideas?
Thank you very much for your time,
My question relates to question 3141.