Bug report: dataloader.py does not read all slices #35
Description
Apparently, there is a bug in the algorithm behind the function generate_batch_from_files in the file dataloader.py in the 2D directory. It does not read all slices of some files.
Below you see some printout from inside the function generate_batch_from_files for the first few iterations for a batch_size of 320 and num_slices_per_scan of 155.
For this setup, each time that a queue of images is created, three volumes of 155 slices are read resulting in a stack of 465 slices which is larger than the batch size. The stack should be cropped to 320 slices to match the batch size.
When idy=0, the stack of images is cropped from the end, meaning the first 320 slices are kept and the rest are thrown away. When idy=30, the stack of images is cropped from the beginning, meaning that the last 320 slices are kept and the rest are thrown away. Each time a new set of 3 files is read and the leftover from the last step is not used anymore.
If this repository is still active, I would really appreciate your reply.
NUM_QUEUED_IMAGES: 3
idz=0 idx=0 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_089.nii.gz
idz=1 idx=1 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_071.nii.gz
idz=2 idx=2 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_183.nii.gz
Epoch 1/30
idz=0 idx=3 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_046.nii.gz
idz=1 idx=4 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_182.nii.gz
idz=2 idx= 5 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_408.nii.gz
idz=0 idx=7 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_072.nii.gz
idz=1 idx=8 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_145.nii.gz
idz=2 idx=9 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_102.nii.gz
1/187 [..............................] - ETA: 0s - loss: 7.2098 - dice_coef: 1.3537e-04 - soft_dice_coef: 0.0010after yield: idx 10 idy 320
idz=0 idx=10 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_431.nii.gz
idz=1 idx=11 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_233.nii.gz
idz=2 idx=12 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_476.nii.gz
2/187 [..............................] - ETA: 36:50 - loss: 6.6567 - dice_coef: 4.5922e-04 - soft_dice_coef: 0.0021after yield: idx 14 idy 0
idz=0 idx=14 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_055.nii.gz
idz=1 idx=15 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_118.nii.gz
idz=2 idx=16 idy=0
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_037.nii.gz
3/187 [..............................] - ETA: 48:27 - loss: 6.6684 - dice_coef: 3.6537e-04 - soft_dice_coef: 0.0021after yield: idx 17 idy 320
idz=0 idx=17 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_058.nii.gz
idz=1 idx=18 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_143.nii.gz
idz=2 idx=19 idy=320
filename /home/pkhateri/Documents/data/decathlon/Task01_BrainTumour/./labelsTr/BRATS_298.nii.gz