在张量流中恢复模型后变量不存在问题

Question

我有两个python文件File1，File2。一个用于生成张量流模型，另一个用于使用模型。与SO 中的问题类似的问题。

File1 如下所示

   def test():
      weights = {'out': tf.Variable(tf.random_normal([n_hidden, vocab_size]), name="weights")}
      biases = {'out': tf.Variable(tf.random_normal([vocab_size]), name="biases")}

      ...
      tf.matmul(outputs[-1], weights['out']) + biases['out']
      ....

       # Initializing the variables
      init = tf.global_variables_initializer()

      saver = tf.train.Saver()

      # Launch the graph
      with tf.Session() as session:
          session.run(init)
          .....
          while step < training_iters:
            _, acc, loss, onehot_pred = session.run([optimizer, accuracy, cost, pred], \
                                                      feed_dict={x: symbols_in_keys, y: symbols_out_onehot})
          .....
          saver.save(session, "resources/model")

文件2：恢复模型如下图

   modelLocation ='resources/model.meta'
    with tf.Session().as_default() as restored_session:        
        saver = tf.train.import_meta_graph(modelLocation, clear_devices=True)
        saver.restore(restored_session, modelLocation[0:len(modelLocation)-5])

        weights_restored_n = tf.get_variable("weights:0")
        biases_restored_n = tf.get_variable("biases:0")
        # weights_restored = tf.get_default_graph().get_tensor_by_name("weights:0")
        # biases_restored = tf.get_default_graph().get_tensor_by_name("biases:0")
        pred = RNN(x, weights_restored_n, biases_restored_n)

我在运行 File2 时遇到的错误

ValueError: Shape of a new variable (weights:0) must be fully defined, but instead was <unknown>.

如果我使用 pred = RNN(x, weights_restored_n, biases_restored_n) 运行文件并评论其他两个，我会收到以下错误

ValueError: Variable rnn/basic_lstm_cell/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

当我检查可用变量时，我看到权重和偏差变量都赢得了恢复的图表。

<tf.Variable 'weights:0' shape=(512, 112) dtype=float32_ref>
<tf.Variable 'biases:0' shape=(112,) dtype=float32_ref>
<tf.Variable 'rnn/basic_lstm_cell/weights:0' shape=(513, 2048) dtype=float32_ref>
<tf.Variable 'rnn/basic_lstm_cell/biases:0' shape=(2048,) dtype=float32_ref>
<tf.Variable 'weights/RMSProp:0' shape=(512, 112) dtype=float32_ref>
<tf.Variable 'weights/RMSProp_1:0' shape=(512, 112) dtype=float32_ref>
<tf.Variable 'biases/RMSProp:0' shape=(112,) dtype=float32_ref>
<tf.Variable 'biases/RMSProp_1:0' shape=(112,) dtype=float32_ref>
<tf.Variable 'rnn/basic_lstm_cell/weights/RMSProp:0' shape=(513, 2048) dtype=float32_ref>
<tf.Variable 'rnn/basic_lstm_cell/weights/RMSProp_1:0' shape=(513, 2048) dtype=float32_ref>
<tf.Variable 'rnn/basic_lstm_cell/biases/RMSProp:0' shape=(2048,) dtype=float32_ref>
<tf.Variable 'rnn/basic_lstm_cell/biases/RMSProp_1:0' shape=(2048,) dtype=float32_ref>

使用这些变量的地方也设置为

rnn_cell = rnn.BasicLSTMCell(n_hidden, reuse=True)

编辑：第二次迭代

with tf.Session() as restored_session:
    modelLocation = resources/model + '.meta'       
    saver = tf.train.import_meta_graph(modelLocation)
    saver.restore(restored_session, resources/model)

    # Checking what variables are present in the restored graph.
    for v in tf.get_default_graph().get_collection("variables"):
        print(v)

    graph = tf.get_default_graph()
    weights_restored = graph.get_tensor_by_name("weights:0")
    biases_restored = graph.get_tensor_by_name("biases:0")
    x_restored = graph.get_tensor_by_name("x:0")

    pred = RNN(x_restored, weights_restored, biases_restored)

Answer 1

如果我没听错的话，您正在尝试重用一个名为“weights:0”的预训练变量，该变量是从保存的 model.meta 图形文件中恢复的。

为此，您需要导入模型及其图形定义并将其设置为默认图形

saver = tf.train.import_meta_graph('resources/model.meta')
graph = tf.get_default_graph()

要获取图表中包含的所有操作的列表，您可以使用get_operations()：

[op.name for op in graph.get_operations()]

在default_graph 的范围内，您可以访问图表的所有操作，在您的情况下，您可以执行以下操作：

with graph.as_default() as default_graph:
  # get the output tensor of an operation
  weights_restored_n = default_graph.get_operation_by_name('weights').outputs[0]
  biases_restored_n = default_graph.get_operation_by_name('biases').outputs[0]
  # ... do some computations ...
  x = tf.get_variable('x') # adds a new tensor to default_graph!
  pred = RNN(x, weights_restored_n, biases_restored_n)

  with tf.Session() as sess:
    # restore values of 'weights:0' etc., instead of initializing
    saver.restore(sess, 'resources/model')
    # run pred operation and feed some data
    sess.run(pred, feed_dict={x:x_train})

我希望您了解如何重用保存的元图和重用经过训练的参数。

注意：tf.get_variable() 向图中添加新张量或在变量范围的意义上重用现有张量，这与从预训练模型恢复张量及其值的情况不同。

编辑：tf.get_tensor_by_name('weights:0') 和 tf.get_operation_by_name('weights').outputs[0] 给出相同的结果