Tensorflow：在尝试训练具有多个输入的 CNN 时，“不允许迭代 `tf.Tensor`”

Question

我正在尝试制作一个 CNN，它需要 3 张图片来进行预测。在神经网络内部，来自 3 个神经网络的预测被连接起来。我很难给它正确的输入。该示例可以轻松复制/粘贴并运行。

问题出现在call() 方法之前，我试图分离输入以将它们发送到不同的神经网络。我尝试了多个作业，我尝试了，zip() 等。

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
from tensorflow import keras as K
import numpy as np
from functools import partial

mnist = K.datasets.cifar10.load_data()

(xtrain, ytrain), (xtest, ytest) = mnist

train_indices = np.random.randint(0, 50_000, (100_000, 3))
test_indices = np.random.randint(0, 10_000, (20_000, 3))

train_inputs = xtrain[train_indices].astype(np.float32)/255
test_inputs = xtest[test_indices].astype(np.float32)/255

train_outputs = np.array(np.sum(ytrain[train_indices], axis=1) % 2 == 0, dtype=np.int32)
test_outputs = np.array(np.sum(ytest[test_indices], axis=1) % 2 == 0, dtype=np.int32)

x = tf.data.Dataset.from_tensor_slices(train_inputs).map(lambda x: tf.expand_dims(x, 1))
y = tf.data.Dataset.from_tensor_slices(train_outputs)

train_ds = tf.data.Dataset.zip((x, y))
test_ds = tf.data.Dataset.from_tensor_slices((test_inputs, test_outputs))


class MultiInputCNN(K.Model):
    def __init__(self):
        super(MultiInputCNN, self).__init__()
        custom_net = partial(K.applications.MobileNetV2,
                             input_shape=(32, 32, 3),
                             include_top=False,
                             weights=None)

        self.net1 = custom_net()
        self.net2 = custom_net()
        self.net3 = custom_net()

        self.concat = K.layers.Concatenate()
        self.pool = K.layers.GlobalAveragePooling2D()
        self.dropout = K.layers.Dropout(.5)
        self.dense = K.layers.Dense(2)

    def call(self, inputs, training=None, **kwargs):
        x, y, z = inputs[0]
        a = self.net1(x)
        b = self.net2(y)
        c = self.net3(z)

        x = self.concat([a, b, c])
        x = self.pool(x)
        x = self.dropout(x)
        x = tf.nn.sigmoid(self.dense(x))
        return x


model = MultiInputCNN()

model(next(iter(train_ds)))

OperatorNotAllowedInGraphError：不允许迭代tf.Tensor：AutoGraph 没有转换此函数。尝试直接用@tf.function 装饰它。

Answer 1

这是我的解决方案...如果我理解正确的话，有 3 个 modilenet（不是 1 个共享权重，在这种情况下，下面的代码也很容易修改）

class MultiInputCNN(K.Model):
    def __init__(self):
        super(MultiInputCNN, self).__init__()

        self.custom_net1 = K.applications.MobileNetV2(
                             input_shape=(32, 32, 3),
                             include_top=False,
                             weights=None)
        
        self.custom_net2 = K.applications.MobileNetV2(
                             input_shape=(32, 32, 3),
                             include_top=False,
                             weights=None)
  
        self.custom_net3 = K.applications.MobileNetV2(
                             input_shape=(32, 32, 3),
                             include_top=False,
                             weights=None)

        self.concat = K.layers.Concatenate()
        self.pool = K.layers.GlobalAveragePooling2D()
        self.dropout = K.layers.Dropout(.5)
        self.dense = K.layers.Dense(2)

    def call(self, inputs, training=None, **kwargs):
        x, y, z = inputs[0]
        net1 = self.custom_net1(x)
        net2 = self.custom_net2(y)
        net3 = self.custom_net3(z)

        x = self.concat([net1, net2, net3])
        x = self.pool(x)
        x = self.dropout(x)
        x = tf.nn.sigmoid(self.dense(x))
        return x


model = MultiInputCNN()

model(next(iter(train_ds)))

这里是运行示例：https://colab.research.google.com/drive/1pYH7wExp_yglf_8vbszzvnuSHP9wZMv3?usp=sharing（我将 n_sample 减少到不会用完 ram）