Keras：如何使用来自另一个张量的信息对张量进行切片？

Question

我正在尝试实现自定义损失函数并遇到了这个问题。自定义损失函数如下所示：

def customLoss(z):
    y_pred = z[0] 
    y_true = z[1]
    features = z[2] 
    ...
    return loss

在我的情况下，y_pred 和 y_true 实际上是灰度图像。 z[2] 中包含的特征由一对位置 (x,y) 组成，我想在其中比较 y_pred 和 y_true。这些位置取决于输入训练样本，因此在定义模型时，它们作为输入传递。所以我的问题是：如何使用张量 features 来索引张量 y_pred 和 y_true？

Answer 1

如果您使用 Tensorflow 作为后端，tf.gather_nd() 可以解决问题（据我所知，Keras 还没有完全等价的）：

from keras import backend as K
import tensorflow as tf

def customLoss(z):
    y_pred = z[0]
    y_true = z[1]
    features = z[2]

    # Gathering values according to 2D indices:
    y_true_feat = tf.gather_nd(y_true, features)
    y_pred_feat = tf.gather_nd(y_pred, features)

    # Computing loss (to be replaced):
    loss = K.abs(y_true_feat - y_pred_feat)
    return loss

# Demonstration:
y_true = K.constant([[[0, 0, 0], [1, 1, 1]], [[2, 2, 2], [3, 3, 3]]])
y_pred = K.constant([[[0, 0, -1], [1, 1, 1]], [[0, 2, 0], [3, 3, 0]]])
coords = K.constant([[0, 1], [1, 0]], dtype="int64")

loss = customLoss([y_pred, y_true, coords])

tf_session = K.get_session()
print(loss.eval(session=tf_session))
# [[ 0.  0.  0.]
#  [ 2.  0.  2.]]

---

注 1： 然而 Keras 有 K.gather()，它只适用于一维索引。如果您只想使用原生 Keras，您仍然可以展平矩阵和索引，以应用此方法：

def customLoss(z):
    y_pred = z[0]
    y_true = z[1]
    features = z[2]

    y_shape = K.shape(y_true)
    y_dims = K.int_shape(y_shape)[0]

    # Reshaping y_pred & y_true from (N, M, ...) to (N*M, ...):
    y_shape_flat = [y_shape[0] * y_shape[1]] + [-1] * (y_dims - 2)
    y_true_flat = K.reshape(y_true, y_shape_flat)
    y_pred_flat = K.reshape(y_pred, y_shape_flat)

    # Transforming accordingly the 2D coordinates in 1D ones:
    features_flat = features[0] * y_shape[1] + features[1]

    # Gathering the values:
    y_true_feat = K.gather(y_true_flat, features_flat)
    y_pred_feat = K.gather(y_pred_flat, features_flat)

    # Computing loss (to be replaced):
    loss = K.abs(y_true_feat - y_pred_feat)
    return loss

注意 2： 要在评论中回答您的问题，可以使用 numpy 方式以 Tensorflow 作为后端进行切片：

x = K.constant([[[0, 1, 2], [3, 4, 5]], [[0, 0, 0], [0, 0, 0]]])
sess = K.get_session()

# When it comes to slicing, TF tensors work as numpy arrays:
slice = x[0, 0:2, 0:3]
print(slice.eval(session=sess))
# [[ 0.  1.  2.]
#  [ 3.  4.  5.]]

# This also works if your indices are tensors (TF will call tf.slice() below):
coords_range_per_dim = K.constant([[0, 2], [0, 3]], dtype="int32")
slice = x[0,
          coords_range_per_dim[0][0]:coords_range_per_dim[0][1],
          coords_range_per_dim[1][0]:coords_range_per_dim[1][1]
         ]
print(slice.eval(session=sess))
# [[ 0.  1.  2.]
#  [ 3.  4.  5.]]