使用 tf 数据集训练 CNN答案

【问题标题】：Training CNN with tfdataset使用 tf 数据集训练 CNN
【发布时间】：2021-03-16 23:33:07
【问题描述】：

我正在尝试使用 TFRecordDataset 训练 CNN（我认为这无关，但这是我的情况）并收到以下错误：

ValueError：维度 0 的切片索引 0 超出范围。对于'{{节点跨步切片}} = 跨步切片[索引=DT_INT32，T=DT_INT32， begin_mask=0，ellipsis_mask=0，end_mask=0，new_axis_mask=0， shrink_axis_mask=1]（形状，strided_slice/stack，strided_slice/stack_1， strided_slice/stack_2)' 具有输入形状：[0]、[1]、[1]、[1] 和计算输入张量：input[1] = , input[2] = , input[3] = .

例如，这是我正在执行的代码：

CNN：

import tensorflow as tf
def get_cnn_model(input_shape=(31, 31, 9), n_outputs=4, convolutions=3, optimizer='adam', seed=26):
    tf.random.set_seed(seed=seed)
    _input = layers.Input(shape=input_shape, name='input')
    x = layers.Conv2D(64, (4, 4), activation='relu', padding='same', name=f'conv_0')(_input)
    x = layers.MaxPooling2D(2)(x)
    for i in range(convolutions - 1):
        x = layers.Conv2D(64, (4, 4), activation='relu', padding='same', name=f'conv_{i + 1}')(x)
        x = layers.MaxPooling2D(2)(x)
    x = layers.Flatten()(x)
    x = layers.Dense(128, activation='relu', name='dense_1')(x)
    x = layers.Dropout(0.35, name='dropout_1')(x)
    x = layers.Dense(128, activation='relu', name='dense_2')(x)
    x = layers.Dropout(0.35, name='dropout_2')(x)
    p = layers.Dense(n_outputs, activation='tanh', name='p')(x)
    v = layers.Dense(1, activation='tanh', name='v')(x)
    cnn_model = Model(inputs=_input, outputs=[v, p])
    losses = {
        "v": 'mean_squared_error',
        "p": keras.losses.BinaryCrossentropy()
    }
    cnn_model.compile(loss=losses, optimizer=optimizer)
    return cnn_model

cnn = get_cnn_model((31, 31, 9), n_outputs=16, convolutions=3, optimizer='adam', seed=26)

这是示例数据集：

import numpy as np
import tensorflow as tf

v = 0.9
p = np.random.randn(16)
state = np.random.randn(31*31*9)

sample = tf.train.Example(
    features = tf.train.Features(
        feature = {
            'v': tf.train.Feature(float_list=tf.train.FloatList(value=[v])),
            'p': tf.train.Feature(float_list=tf.train.FloatList(value = p)),
            's': tf.train.Feature(float_list=tf.train.FloatList(value = state))
        }
    )
)

with tf.io.TFRecordWriter('tf_record_data') as f:
    f.write(sample.SerializeToString())

这是我得到上述错误的训练过程：

def read_tfrecord(example):
    feature_desc = {
        'v': tf.io.FixedLenFeature([], tf.float32),
        'p': tf.io.VarLenFeature(tf.float32),
        's': tf.io.VarLenFeature(tf.float32)
    }
    sample = tf.io.parse_single_example(example, feature_desc)
    x = tf.reshape(tf.sparse.to_dense(parsed['s']), (1,31,31, 9))
    y = {'v':sample['v'], 'p': tf.sparse.to_dense(sample['p'])}
    return x, y

ds = tf.data.TFRecordDataset(['tf_record_data'])
ds = ds.map(read_tfrecord)

cnn.fit(ds)

有趣的是，当我对数据集进行预测时，它确实有效：

import numpy as np
for serialized in tf.data.TFRecordDataset(['tf_record_data']):
    parsed = tf.io.parse_single_example(serialized, feature_desc)
    st= tf.sparse.to_dense(parsed['s'])
    t = tf.reshape(st, (1, 31, 31, 9))
    print(cnn.predict(t))

我该如何解决这个错误？

【问题讨论】：

标签： python tensorflow tensorflow2.0 tensorflow-datasets

【解决方案1】：

我将数据记录的映射更改为以下内容：

def read_tfrecord(example):
    feature_desc = {
       'v': tf.io.FixedLenFeature([], tf.float32),
       'p': tf.io.VarLenFeature(tf.float32),
       's': tf.io.VarLenFeature(tf.float32)
    }
    sample = tf.io.parse_single_example(example, feature_desc)
    x = tf.reshape(tf.sparse.to_dense(parsed['s']), (1,rows,cols, layers))
    p = tf.reshape(tf.sparse.to_dense(parsed['p']), (1, 16))
    v = tf.reshape(sample['v'], (1, 1))

    y = {'v':v, 'p': p}
    return x, y

重塑输出解决了问题

【讨论】：