访问二维张量中的行答案

【问题标题】：Accessing row in a 2-D tensor访问二维张量中的行
【发布时间】：2017-03-09 12:22:48
【问题描述】：

我有一个非常简单的神经网络的以下代码（这段代码实际上是对一个简单问题的改编）：

import numpy as np
import tensorflow as tf

with tf.device("cpu:0"):
    sess = tf.InteractiveSession()
    nNodes = 3
    inputDim = 1

    rowIdxs = np.zeros([nNodes, nNodes])
    colIdxs = np.zeros([nNodes, nNodes])
    for rowIdx in range(nNodes):
        for colIdx in range(nNodes):
            rowIdxs[rowIdx, colIdx] = rowIdx
            colIdxs[rowIdx, colIdx] = colIdx

    rowIdxs = np.reshape(rowIdxs, [-1])
    colIdxs = np.reshape(colIdxs, [-1])

    # build a matrix with nNodes x nNodes elements
    # with each row i containing the distance from node i to all the other nodes
    distances = np.zeros([nNodes, nNodes])
    for i in range(nNodes):
        for j in range(nNodes):
            distances[i, j] = ((rowIdxs[i] - rowIdxs[j]) ** 2 + (colIdxs[i] - colIdxs[j]) ** 2)
    print('distances=', distances)

    # tensorflow constant from distances matrix
    distances_ = tf.constant(distances, dtype=tf.float32)

    # w corresponds to a weight matrix in a neural network
    w = tf.random_uniform((nNodes, inputDim), 0.0, 1.0)

    # x corresponds to the input to the network
    x = tf.random_uniform((1, inputDim), 0.0, 1.0)

    xx = tf.tile(x, [nNodes,1])
    print('w', w.shape)
    print('x', x.shape)
    print('xx', xx.shape)

    # differences between weights and input vector
    diff = tf.reduce_sum(tf.abs(tf.subtract(xx, w)), 1)
    print('diff.shape', diff.shape)


    # index of the best matching unit
    bmu = tf.arg_min(diff, 0)       

    # Now I need to access the distances from BMU to the other nodes
    slice = tf.slice(distances_, [bmu, 0], [bmu, -1])

    sess.run(tf.global_variables_initializer())
    sess.run(slice)
    print('slice=', slice.eval())
    print('diff', diff.eval())
    print('bmu=', bmu.eval())

基本上，给定输入 x，将其与权重 w 进行比较，然后选择差异最小的节点 BMU。

我对该代码有几个问题：

1.有时它可以正常工作，有时它会引发异常。

当它不起作用时，输出是这样的：

distances= 
 [[ 0.  1.  4.]
 [ 1.  0.  1.]
 [ 4.  1.  0.]]

w (3, 1)
x (1, 1)
xx (3, 1)
diff.shape (3,)

InvalidArgumentError (see above for traceback): Expected size[0] in [0, 1], but got 2
 [[Node: Slice = Slice[Index=DT_INT64, T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Const, Slice/begin, Slice/size)]]

完整的堆栈如下：

Traceback (most recent call last):
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1022, in _do_call
    return fn(*args)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1004, in _run_fn
    status, run_metadata)
  File "D:\python\lib\contextlib.py", line 66, in __exit__
    next(self.gen)
  File "D:\python\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected size[0] in [0, 1], but got 2
     [[Node: Slice = Slice[Index=DT_INT64, T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Const, Slice/begin, Slice/size)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/workspace/LiClipse Workspace/kerasPython/exercises/testIndexing.py", line 44, in <module>
    sess.run(slice)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
    run_metadata_ptr)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 965, in _run
    feed_dict_string, options, run_metadata)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1015, in _do_run
    target_list, options, run_metadata)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Expected size[0] in [0, 1], but got 2
     [[Node: Slice = Slice[Index=DT_INT64, T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Const, Slice/begin, Slice/size)]]

Caused by op 'Slice', defined at:
  File "D:/workspace/LiClipse Workspace/kerasPython/exercises/testIndexing.py", line 39, in <module>
    slice = tf.slice(distances_, [bmu, 0], [bmu, -1])
  File "D:\python\lib\site-packages\tensorflow\python\ops\array_ops.py", line 561, in slice
    return gen_array_ops._slice(input_, begin, size, name=name)
  File "D:\python\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 3053, in _slice
    name=name)
  File "D:\python\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow\python\framework\ops.py", line 2395, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow\python\framework\ops.py", line 1264, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Expected size[0] in [0, 1], but got 2
     [[Node: Slice = Slice[Index=DT_INT64, T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](Const, Slice/begin, Slice/size)]]

什么时候起作用

w (3, 1)
x (1, 1)
xx (3, 1)
diff.shape (3,)
slice= [[ 1.  0.  1.]]
diff [ 0.29777944  0.08669317  0.09722018]
bmu= 0

bmu错了，应该是1，但是切片是正确的。

有时我会明白：

w (3, 1)
x (1, 1)
xx (3, 1)
diff.shape (3,)
slice= []
diff [ 0.33319855  0.12426794  0.49753141]
bmu= 1

bmu 为 1，但切片为空。

2。当我切换到 GPU 时，我有一个异常告诉我我不能使用 bmu 进行索引。从with tf.device("gpu:0"): 开始，我明白了：

InvalidArgumentError (see above for traceback): Cannot assign a device to node 'Slice/size': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
     [[Node: Slice/size = Pack[N=2, T=DT_INT64, axis=0, _device="/device:GPU:0"](ArgMin, Slice/size/1)]]

完整的堆栈跟踪如下：

Traceback (most recent call last):
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1022, in _do_call
    return fn(*args)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1004, in _run_fn
    status, run_metadata)
  File "D:\python\lib\contextlib.py", line 66, in __exit__
    next(self.gen)
  File "D:\python\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 469, in raise_exception_on_not_ok_status
    pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device to node 'Slice/size': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
     [[Node: Slice/size = Pack[N=2, T=DT_INT64, axis=0, _device="/device:GPU:0"](ArgMin, Slice/size/1)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/workspace/LiClipse Workspace/kerasPython/exercises/testIndexing.py", line 45, in <module>
    sess.run(slice)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 767, in run
    run_metadata_ptr)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 965, in _run
    feed_dict_string, options, run_metadata)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1015, in _do_run
    target_list, options, run_metadata)
  File "D:\python\lib\site-packages\tensorflow\python\client\session.py", line 1035, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cannot assign a device to node 'Slice/size': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
     [[Node: Slice/size = Pack[N=2, T=DT_INT64, axis=0, _device="/device:GPU:0"](ArgMin, Slice/size/1)]]

Caused by op 'Slice/size', defined at:
  File "D:/workspace/LiClipse Workspace/kerasPython/exercises/testIndexing.py", line 40, in <module>
    slice = tf.slice(distances_, [bmu, 0], [bmu, -1])
  File "D:\python\lib\site-packages\tensorflow\python\ops\array_ops.py", line 561, in slice
    return gen_array_ops._slice(input_, begin, size, name=name)
  File "D:\python\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 3053, in _slice
    name=name)
  File "D:\python\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 491, in apply_op
    preferred_dtype=default_dtype)
  File "D:\python\lib\site-packages\tensorflow\python\framework\ops.py", line 716, in internal_convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "D:\python\lib\site-packages\tensorflow\python\ops\array_ops.py", line 923, in _autopacking_conversion_function
    return _autopacking_helper(v, inferred_dtype, name or "packed")
  File "D:\python\lib\site-packages\tensorflow\python\ops\array_ops.py", line 886, in _autopacking_helper
    return gen_array_ops._pack(elems_as_tensors, name=scope)
  File "D:\python\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 2041, in _pack
    result = _op_def_lib.apply_op("Pack", values=values, axis=axis, name=name)
  File "D:\python\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 763, in apply_op
    op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow\python\framework\ops.py", line 2395, in create_op
    original_op=self._default_original_op, op_def=op_def)
  File "D:\python\lib\site-packages\tensorflow\python\framework\ops.py", line 1264, in __init__
    self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Cannot assign a device to node 'Slice/size': Could not satisfy explicit device specification '/device:GPU:0' because no supported kernel for GPU devices is available.
     [[Node: Slice/size = Pack[N=2, T=DT_INT64, axis=0, _device="/device:GPU:0"](ArgMin, Slice/size/1)]]

我无法理解发生了什么：我有一个想法，但在文档或其他任何地方都找不到任何参考。可能是我使用了错误的关键字。

有没有人可以帮助我？

【问题讨论】：

有意思，我用eval跟踪w和x的值，它们的值每一步都在变化，也许这就是问题所在
包含完整的堆栈跟踪将有助于诊断这一点。 Pack 是为 GPU 实现的，因此只有在您的 GPU 构建损坏时才会看到此错误。
@xxi “他们每一步都在变化”是什么意思，我该如何避免这个问题？
我已经添加了完整的堆栈跟踪。
我只是在w = tf.random_uniform((nNodes, inputDim), 0.0, 1.0)上打断点，用w.eval()逐行追踪它们的值，它们的值总是在变化，不确定是不是这个问题。

标签： python numpy tensorflow

【解决方案1】：

tf.slice 函数 (see TF1.15 doc) 将切片的size 作为第三个参数，因此slice = tf.slice(distances_, [bmu, 0], [bmu, -1]) 行必须替换为

slice = tf.slice(distances_, [bmu, 0], [1, -1])

仅当bmu 大于distances_ 中行数的一半时才会出现该错误，因为在这种情况下无法提取给定大小的切片。否则计算不会失败，但它不正确，因为您没有提取单行。

【讨论】：