不兼容的形状：[1020,1,1] 与 [1019,1,1] - Tensorflow

Question

将我的神经网络的输出设置为1时出现问题，它在数据数组中出现问题，指责哪个形状更大，如果我使用periods = len(valuesAnalisys) - 1，一切正常！

期间：

periods = 1

返回：

Imcompatible shapes: [1020,1,1] vs. [1019,1,1]

神经网络：

datecollect = [x[0] for x in dataSet]
servers = [x[1] for x in dataSet]
valuesAnalisys = [float(x[2]) for x in dataSet]

base = np.array(valuesAnalisys)

periods = 1
future_forecast = 1

X = base[0:(len(base) - (len(base) % periods))]
X_batches = X.reshape(-1, periods, 1)

y = base[1:(len(base) - (len(base) % periods)) + future_forecast]
y_batches = y.reshape(-1, periods, 1)

X_test = base[-(periods + future_forecast):]
X_test = X_test[:periods]
X_test = X_test.reshape(-1, periods, 1)
y_test = base[-(periods):]
y_test = y_test.reshape(-1, periods, 1)

tf.reset_default_graph()

appetizer = 1
hidden_neurons = 100
exit_neurons = 1

xph = tf.placeholder(tf.float32, [None, periods, appetizer])
yph = tf.placeholder(tf.float32, [None, periods, exit_neurons])

cell = tf.contrib.rnn.BasicRNNCell(num_units = hidden_neurons, activation = tf.nn.relu)

cell = tf.contrib.rnn.OutputProjectionWrapper(cell, output_size = 1)

exit_rnn, _ = tf.nn.dynamic_rnn(cell, xph, dtype = tf.float32)
calculateError = tf.losses.mean_squared_error(labels = yph, predictions = exit_rnn)
otimizador = tf.train.AdamOptimizer(learning_rate = 0.001)
training = otimizador.minimize(calculateError)

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())

    for epoch in range(2000):
        _, cost = sess.run([training, calculateError], feed_dict = {xph: X_batches, yph: y_batches})
        if epoch % 100 == 0:
            print("[INFO] Epoch: {} - Level Error: {}".format(epoch,cost))

    forecast = sess.run(exit_rnn, feed_dict = {xph: X_test})

y_test.shape
y_test2 = np.ravel(y_test)

final_forecast = np.ravel(forecast)

mae = mean_absolute_error(y_test2, final_forecast)

for (host, forecast, date) in list(zip(servers, final_forecast, datecollect)):
    send.postForecastMemory(host, forecast, cost, date)

输出：

A:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1356, in _do_call
    return fn(*args)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [1020,1,1] vs. [1019,1,1]
         [[{{node gradients/mean_squared_error/SquaredDifference_grad/BroadcastGradientArgs}}]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "teste.py", line 61, in <module>
    _, custo = sess.run([treinamento, erro], feed_dict = {xph: X_batches, yph: y_batches})
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 950, in run
    run_metadata_ptr)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1173, in _run
    feed_dict_tensor, options, run_metadata)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1350, in _do_run
    run_metadata)
  File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1370, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [1020,1,1] vs. [1019,1,1]
         [[node gradients/mean_squared_error/SquaredDifference_grad/BroadcastGradientArgs (defined at teste.py:55) ]]

Click data view

形状：

[INFO] base shape: (1020,)
[INFO] X shape: (1019,)
[INFO] X batches shape: (1, 1019, 1)
[INFO] Y batches shape: (1, 1019, 1)
[INFO] X teste shape: (1, 1019, 1)
[INFO] Y teste shape: (1, 1019, 1)

Answer 1

罪魁祸首似乎是您的 RNN 单元中的固定时间维度。

xph = tf.placeholder(tf.float32, [None, periods, appetizer])
yph = tf.placeholder(tf.float32, [None, periods, exit_neurons])

cell = tf.contrib.rnn.BasicRNNCell(num_units = hidden_neurons, activation = tf.nn.relu)

在这里，在 xph 和 yph 中，您都将时间维度指定为句点。所以，如果你有更长或更短的信号，就会出现错误。

由于您没有指定输入形状或模型摘要，我无法推断出模型层的确切尺寸。所以，使用占位符数字。

有两种可能的修复方法。

1. 不要使用固定的时间维度 = 周期，而是使用无。

xph = tf.placeholder(tf.float32, [None, None, appetizer])
yph = tf.placeholder(tf.float32, [None, None, exit_neurons])

但是，缺点是您必须在每个批次中具有相同长度的信号，或者您可以简单地使用批次大小 = 1 进行训练，而不用担心时间长度。

1. 使用截断/填充来解决长度问题。只需将您的信号传递给预处理函数即可添加/删除额外的时间点。

import numpy as np
def pre_process(x, fixed_len = 1000): # x.shape -> (100, 1000, 1)

    if x.shape[1] >= fixed_len:
       return x[:,:fixed_len,:]
    else:
       z_ph = np.zeros((x.shape[0], fixed_len, x.shape[2]))
       z_ph[:,:x.shape[1],:] = x
       return z_ph

X_batches = pre_process(X_batches, YOU_CHOOSE_THIS_LENGTH) # based on the length of your data
X_test = pre_process(X_test, YOU_CHOOSE_THIS_LENGTH)