在张量流中使用 Adam 优化器 TWICE

Question

我尝试使用 adam 优化器两次以最小化代码中的不同张量，我尝试使用 GradientDescentOptimizer 两次，这很好，但是当我使用两次 adam 优化器时收到错误消息，我问了另一个问题：tensorflowVariable RNNLM/RNNLM/embedding/Adam_2/ does not exist，但该解决方案在这里不起作用。我也查了页面：https://github.com/tensorflow/tensorflow/issues/6220，但我还是不明白。

这是我的代码，我收到错误消息：ValueError: Variable NN/NN/W/Adam_2/ 不存在，或者不是用 tf.get_variable() 创建的。您的意思是在 VarScope 中设置 reuse=None 吗？

然后我在 tensorflowVariable RNNLM/RNNLM/embedding/Adam_2/ does not exist 尝试了解决方案，但不起作用

import tensorflow as tf

def main():
    optimizer = tf.train.GradientDescentOptimizer(0.005)
    # optimizer = tf.train.AdamOptimizer(0.005)

    with tf.variable_scope('NN') as scope:
        W = tf.get_variable(name='W', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        X = tf.get_variable(name='X', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        y_ = tf.get_variable(name='y_', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        y1 = W + X
        loss_1 = tf.reduce_mean(tf.abs(y_ - y1))


        # train_op1 = tf.train.GradientDescentOptimizer(0.005).minimize(loss_1)
        train_op1 = tf.train.AdamOptimizer(0.005).minimize(loss_1)
        # with tf.variable_scope('opt'):
        #     train_op1 = tf.train.AdamOptimizer(0.005).minimize(loss_1)

        ##############################################################################################
        scope.reuse_variables()

        W2 = tf.get_variable(name='W', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        X2 = tf.get_variable(name='X', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        b = tf.Variable(tf.random_normal(shape=[5, 1], dtype=tf.float32))
        y2 = W2 + X2 + b
        loss_2 = tf.reduce_mean(tf.abs(y_ - y2))

        # train_op2 = tf.train.GradientDescentOptimizer(0.005).minimize(loss_2)
        train_op2 = tf.train.AdamOptimizer(0.005).minimize(loss_2)
        # with tf.variable_scope('opt'):
        #     train_op2 = tf.train.AdamOptimizer(0.005).minimize(loss_2)


if __name__ == '__main__':
    main()

Answer 1

解决此问题的最简单方法是将第二个优化器放在不同的范围内。这样命名不会引起任何混淆。

Answer 2

如果您绝对必须在同一范围内执行此操作， 确保及时定义所有变量。 我必须对它为什么会这样工作做更多的研究，但是优化器设置被锁定在较低级别的图表中，不再可以动态访问。

最小的工作示例：

import tensorflow as tf

def main():
    optimizer = tf.train.GradientDescentOptimizer(0.005)
    # optimizer = tf.train.AdamOptimizer(0.005)

    with tf.variable_scope('NN') as scope:
        assert scope.reuse == False
        W2 = tf.get_variable(name='W', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        X2 = tf.get_variable(name='X', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        y2_ = tf.get_variable(name='y_', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        b = tf.get_variable(name='b', initializer=tf.random_normal(shape=[5, 1], dtype=tf.float32))
        y2 = W2 + X2 + b
        loss_2 = tf.reduce_mean(tf.abs(y2_ - y2))

        # train_op2 = tf.train.GradientDescentOptimizer(0.005).minimize(loss_2)
        train_op2 = tf.train.AdamOptimizer(0.005).minimize(loss_2)


        # with tf.variable_scope('opt'):
        #     train_op1 = tf.train.AdamOptimizer(0.005).minimize(loss_1)

    ##############################################################################################
    with tf.variable_scope('NN', reuse = True) as scope:


        W = tf.get_variable(name='W', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        X = tf.get_variable(name='X', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        y_ = tf.get_variable(name='y_', initializer=tf.random_uniform(dtype=tf.float32, shape=[5, 1]))
        b = tf.get_variable(name='b', initializer=tf.random_normal(shape=[5, 1], dtype=tf.float32))

        y1 = W + X
        loss_1 = tf.reduce_mean(tf.abs(y_ - y1))


        # train_op1 = tf.train.GradientDescentOptimizer(0.005).minimize(loss_1)
        train_op1 = tf.train.AdamOptimizer(0.005).minimize(loss_1)
        # with tf.variable_scope('opt'):
        #     train_op2 = tf.train.AdamOptimizer(0.005).minimize(loss_2)


if __name__ == '__main__':
    main()