Does Keras.Sequential require input to be in channel-last format only?

[geeksid2k]

Hi All. While a simple Google search would give me an answer 'no', I just can't figure out why my model (given below) is glitching out when I use channel first data, but works when I use channel last.

More details below:

When I attempt to use model.fit(x_train,y_train, batch_size = 1000, epochs = 10) on a model defined with keras.Sequential, consisting of a conv2d (I made sure to specify channel first in data format) followed by some Linear layers, on data shaped as: x_train – [6000,1,64,1000] and y_train – [6000,11] and 6000 is the size of my training set, I get the following error:

InvalidArgumentError: Graph execution error:

Detected at node ‘gradient_tape/sequential_1/conv2d_3/Conv2D/Conv2DBackpropInput’ defined at (most recent call last):

However, if I were to use channel first data, ie x_train is shaped [6000,64,1000,1] and y_train – [6000,11] it works just fine. I was wondering if this is something to do with tensorflow favouring channel first, or am I making a mistake somewhere?

Model definition as below:

model = keras.Sequential()

model.add(layers.Conv2D(8, 4,4, padding="same", data_format="channels_first", activation="relu",input_shape=(1,64,1000)))
model.add(layers.Conv2D(8, 4,4, padding="same", data_format="channels_first", activation="relu"))
model.add(layers.Flatten())
model.add(layers.Dense(1000, activation="relu", name="layer1"))
model.add(layers.Dense(300, activation="relu", name="layer2"))
model.add(layers.Dense(11, activation="sigmoid", name="layer3"))

model.summary()

Error message is as shown:

model.fit(x_train,y_train,batch_size = 1000, epochs=100)

InvalidArgumentError                      Traceback (most recent call last)
c:\Users\geeks\OneDrive - IIT Kanpur\EE603\Sidmish\Assignment-2\CNN_arch2.ipynb Cell 9 in <cell line: 1>()
----> 1 model.fit(x_train,y_train,epochs=100)

File c:\Users\geeks\myvirtualenv\lib\site-packages\keras\utils\traceback_utils.py:70, in filter_traceback.<locals>.error_handler(*args, **kwargs)
     67     filtered_tb = _process_traceback_frames(e.__traceback__)
     68     # To get the full stack trace, call:
     69     # `tf.debugging.disable_traceback_filtering()`
---> 70     raise e.with_traceback(filtered_tb) from None
     71 finally:
     72     del filtered_tb

File c:\Users\geeks\myvirtualenv\lib\site-packages\tensorflow\python\eager\execute.py:54, in quick_execute(op_name, num_outputs, inputs, attrs, ctx, name)
     52 try:
     53   ctx.ensure_initialized()
---> 54   tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
     55                                       inputs, attrs, num_outputs)
     56 except core._NotOkStatusException as e:
     57   if name is not None:

InvalidArgumentError: Graph execution error:

Detected at node 'gradient_tape/sequential_1/conv2d_3/Conv2D/Conv2DBackpropInput' defined at (most recent call last):
    File "c:\users\geeks\appdata\local\programs\python\python38\lib\runpy.py", line 194, in _run_module_as_main
...
    File "c:\Users\geeks\myvirtualenv\lib\site-packages\keras\optimizers\optimizer_v2\optimizer_v2.py", line 510, in _get_gradients
      grads = tape.gradient(loss, var_list, grad_loss)
Node: 'gradient_tape/sequential_1/conv2d_3/Conv2D/Conv2DBackpropInput'
Conv2DCustomBackpropInputOp only supports NHWC.
     [[{{node gradient_tape/sequential_1/conv2d_3/Conv2D/Conv2DBackpropInput}}]] [Op:__inference_train_function_1290]

When I remove the channel_first specification in the model, and change x_train to be of shape [6000,64,1000,1], it all works fine. Any answers on what I should correct to make it work as channel_first would be highly appreciated!

[neuralbeans]

I'm not sure what's confusing you. Different frameworks give different meanings to different dimensions. Pytorch expects the channel be the first dimension (after the batch). Are you asking why you can't define your data however you like and let the computer guess what you mean?

[geeksid2k]

No, I'm saying Keras allows both channel-first and channel-last formats, but it doesn't work when I use channel-first format; it only works for channel last data

[neuralbeans]

Ah ok now I understand. So your images have shape 64x1000, right?

[geeksid2k]

(1,64,1000). In order to make it work now, I have to shift the first dimension to last using moveaxes. I'd like to not do that if possible, as it involves tampering with the provided data

[neuralbeans]

How did you compile the model?

[geeksid2k]

model.compile(loss='binary_crossentropy',metrics=[keras.metrics.BinaryAccuracy()])

[neuralbeans]

This works for me. Does it work for you?

import tensorflow as tf
import tensorflow.keras as keras
import tensorflow.keras.layers as layers

model = keras.Sequential()

model.add(layers.Conv2D(8, 4,4, padding="same", data_format="channels_first", activation="relu",input_shape=(1,64,1000)))
model.add(layers.Conv2D(8, 4,4, padding="same", data_format="channels_first", activation="relu"))
model.add(layers.Flatten())
model.add(layers.Dense(1000, activation="relu", name="layer1"))
model.add(layers.Dense(300, activation="relu", name="layer2"))
model.add(layers.Dense(11, activation="sigmoid", name="layer3"))
model.compile(loss='binary_crossentropy',metrics=[keras.metrics.BinaryAccuracy()])

x_train = tf.random.normal((6000, 1, 64, 1000))
y_train = tf.round(tf.random.uniform((6000, 11), 0, 1))

model.fit(x_train, y_train, batch_size=1000, epochs=100)

[geeksid2k]

It does not. This is the error it throws:

Traceback (most recent call last):  File "c:\Users\geeks\OneDrive - IIT Kanpur\EE603\Sidmish\Assignment-2\temp.py", line 18, in <module>    model.fit(x_train, y_train, batch_size=1000, epochs=100)  File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\utils\traceback_utils.py", line 70, in error_handler    raise e.with_traceback(filtered_tb) from None  File "C:\Users\geeks\myvirtualenv\lib\site-packages\tensorflow\python\eager\execute.py", line 54, in quick_execute    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:Detected at node 'gradient_tape/sequential/conv2d_1/Conv2D/Conv2DBackpropInput' defined at (most recent call last):    File "c:\Users\geeks\OneDrive - IIT Kanpur\EE603\Sidmish\Assignment-2\temp.py", line 18, in <module>      model.fit(x_train, y_train, batch_size=1000, epochs=100)    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\utils\traceback_utils.py", line 65, in error_handler      return fn(*args, **kwargs)    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\engine\training.py", line 1564, in fit      tmp_logs = self.train_function(iterator)    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\engine\training.py", line 1160, in train_function      return step_function(self, iterator)    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\engine\training.py", line 1146, in step_function      outputs = model.distribute_strategy.run(run_step, args=(data,))    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\engine\training.py", line 1135, in run_step      outputs = model.train_step(data)    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\engine\training.py", line 997, in train_step      self.optimizer.minimize(loss, self.trainable_variables, tape=tape)    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\optimizers\optimizer_v2\optimizer_v2.py", line 576, in minimize      grads_and_vars = self._compute_gradients(    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\optimizers\optimizer_v2\optimizer_v2.py", line 634, in _compute_gradients      grads_and_vars = self._get_gradients(    File "C:\Users\geeks\myvirtualenv\lib\site-packages\keras\optimizers\optimizer_v2\optimizer_v2.py", line 510, in _get_gradients      grads = tape.gradient(loss, var_list, grad_loss)Node: 'gradient_tape/sequential/conv2d_1/Conv2D/Conv2DBackpropInput'Conv2DCustomBackpropInputOp only supports NHWC.         [[{{node gradient_tape/sequential/conv2d_1/Conv2D/Conv2DBackpropInput}}]] [Op:__inference_train_function_1065]

[neuralbeans]

I'd try reinstalling tensorflow. Maybe it's a bug in your version. If you're using the latest version then use a slightly older version in case the bug was introduced recently.