tensorflow - Changing order of Input Image in 3D convolutions

Question

According to the official documentation of tf.keras.layers.Conv3D

5+D tensor with shape: batch_shape + (channels, conv_dim1, conv_dim2, conv_dim3) if data_format='channels_first' or 5+D tensor with shape: batch_shape + (conv_dim1, conv_dim2, conv_dim3, channels) if data_format='channels_last'

. Now the whole idea around channels and batch shape makes sense, but will changing the general order of (conv_dim1, conv_dim2,conv_dim2) as (x,y,z) to say (z,x,y) affect the performance.

Does Conv3D worry about order of x-y-z dimension ?

I was training a U-net segmentation model and upon changing the order of axis I saw difference in performance. (x,y,z) order converges faster as compared to (y,x,z).

I just wanted to make sure what's the correct way..

question from:https://stackoverflow.com/questions/65863738/changing-order-of-input-image-in-3d-convolutions

Answer 1

Although your data seems to be isotropic, it might not be the case ¹:

The provided data are distributed after their pre-processing, i.e., co-registered to the same anatomical template, interpolated to the same resolution (1 mm^3) and skull-stripped.

It might be the case that the underlying source data was not isotropic, and what you are seeing is the effect of interpolation (some dimensions are "smoother" than the others).

Have you considered taking advantage of this "isotropic" property and use "across dimensions" augmentations as proposed by

Liad Pollak Zuckerman, Eyal Naor, George Pisha, Shai Bagon, Michal Irani Across Scales & Across Dimensions: Temporal Super-Resolution using Deep Internal Learning (ECCV 2020).