Tensorflow - 计算 RBF 网络的欧几里得距离

【问题标题】：Tensorflow - calculate euclidean distance for RBF NetworkTensorflow - 计算 RBF 网络的欧几里得距离
【发布时间】：2018-04-26 22:56:51
【问题描述】：

我在 Tensorflow 中实施 RBF 网络时遇到问题。我需要计算 x 和质心之间的欧几里得距离（来自 RBF newtork 的定义）。我写了这段代码：

    x_data = tf.placeholder(shape=[None, 3], dtype=tf.float32)
    y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32)

    # Create variables for NN layers
    A1 = tf.Variable(tf.random_normal(shape=[3, first_layer_nodes]))  # input -> first layer nodes
    A2 = tf.Variable(tf.random_normal(shape=[first_layer_nodes, 1]))  # first_layer nodes -> sum node
    c = tf.Variable(tf.random_normal(shape=[first_layer_nodes]))  # centroids

    # Declare NN
    inputs_with_weights = tf.matmul(x_data, A1)
    print(inputs_with_weights)
    # euclid_dist = tf.sqrt(tf.reduce_sum(tf.pow(tf.subtract(inputs_with_weights, c), 2)))
    euclid_dist = tf.norm(inputs_with_weights - c, ord='euclidean')
    print(euclid_dist)
    first_output = tf_gaussian_function(euclid_dist)
    print(first_output)

final_output = tf.matmul(first_output, A2)

但我是这样的问题：

E:\#PROJEKTY\#PROGRAMOWANIE\AI-Project>python Iris.py
2018-04-27 00:49:37.800684: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
<tf.Variable 'Variable_2:0' shape=(1, 1) dtype=float32_ref>
Tensor("MatMul:0", shape=(?, 1), dtype=float32)
Tensor("norm/Squeeze:0", shape=(), dtype=float32)
Tensor("gaussian_function:0", dtype=float32)
Traceback (most recent call last):
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1361, in _do_call
    return fn(*args)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _run_fn
    target_list, status, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Iris.py", line 144, in <module>
    sess.run(train_step, feed_dict={x_data: x_d, y_target: y_d})
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 905, in run
    run_metadata_ptr)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1137, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1355, in _do_run
    options, run_metadata)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\client\session.py", line 1374, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

Caused by op 'MatMul_1', defined at:
  File "Iris.py", line 124, in <module>
    final_output = tf.matmul(first_output, A2)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 2064, in matmul
    a, b, transpose_a=transpose_a, transpose_b=transpose_b, name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 2790, in _mat_mul
    name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3271, in create_op
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 1650, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): In[0] is not a matrix
         [[Node: MatMul_1 = MatMul[T=DT_FLOAT, transpose_a=false, transpose_b=false, _device="/job:localhost/replica:0/task:0/device:CPU:0"](gaussian_function, Variable_1/read)]]

如果我尝试将任何值放在轴上，它就会卡在：

E:\#PROJEKTY\#PROGRAMOWANIE\AI-Project>python Iris.py
2018-04-27 00:53:15.388129: I C:\tf_jenkins\workspace\rel-win\M\windows\PY\36\tensorflow\core\platform\cpu_feature_guard.cc:140] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
<tf.Variable 'Variable_2:0' shape=(1, 1) dtype=float32_ref>
Tensor("MatMul:0", shape=(?, 1), dtype=float32)
Traceback (most recent call last):
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 686, in _call_cpp_shape_fn_impl
    input_tensors_as_shapes, status)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Invalid reduction dimension 2 for input with 2 dimensions. for 'norm/Sum' (op: 'Sum') with input shapes: [?,1], [1] and with computed input tensors: input[1] = <2>.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "Iris.py", line 120, in <module>
    euclid_dist = tf.norm(inputs_with_weights - c, axis = 2, ord='euclidean')
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\linalg_ops.py", line 552, in norm
    tensor * math_ops.conj(tensor), axis, keepdims=True))
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\util\deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\math_ops.py", line 1373, in reduce_sum
    name=name))
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\ops\gen_math_ops.py", line 5436, in _sum
    name=name)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3273, in create_op
    compute_device=compute_device)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 3313, in _create_op_helper
    set_shapes_for_outputs(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2501, in set_shapes_for_outputs
    return _set_shapes_for_outputs(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2474, in _set_shapes_for_outputs
    shapes = shape_func(op)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\ops.py", line 2404, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "C:\Users\Szatku\AppData\Local\Programs\Python\Python36\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Invalid reduction dimension 2 for input with 2 dimensions. for 'norm/Sum' (op: 'Sum') with input shapes: [?,1], [1] and with computed input tensors: input[1] = <2>.

我不知道如何更正代码。有人可以帮助我吗？

编辑：高斯实现：

def gaussian_function(input_layer):
    initial = math.exp(-SC*math.pow(input_layer, 2))
    return initial


np_gaussian_function = np.vectorize(gaussian_function)


def d_gaussian_function(input_layer):
    initial = -2*SC*input_layer * math.exp(-SC * math.pow(input_layer, 2))
    return initial


np_d_gaussian_function = np.vectorize(d_gaussian_function)


def np_d_gaussian_function_32(input_layer):
    return np_d_gaussian_function(input_layer).astype(np.float32)


def tf_d_gaussian_function(input_layer, name=None):
    with ops.name_scope(name, "d_gaussian_function", [input_layer]) as name:
        y = tf.py_func(np_d_gaussian_function_32, [input_layer],[tf.float32], name=name, stateful=False)
    return y[0]


def py_func(func, inp, Tout, stateful=True, name=None, grad=None):
    rnd_name = 'PyFunGrad' + str(np.random.randint(0, 1E+8))

    tf.RegisterGradient(rnd_name)(grad)
    g = tf.get_default_graph()
    with g.gradient_override_map({"PyFunc": rnd_name}):
        return tf.py_func(func, inp, Tout, stateful=stateful, name=name)


def gaussian_function_grad(op, grad):
    input_variable = op.inputs[0]
    n_gr = tf_d_gaussian_function(input_variable)
    return grad * n_gr


def np_gaussian_function_32(input_layer):
    return np_gaussian_function(input_layer).astype(np.float32)


def tf_gaussian_function(input_layer, name=None):
    with ops.name_scope(name, "gaussian_function", [input_layer]) as name:
        y = py_func(np_gaussian_function_32, [input_layer], [tf.float32], name=name, grad=gaussian_function_grad)
    return y[0]
# end of defining activation function

【问题讨论】：

    标签： python tensorflow radial


    【解决方案1】：

    错误表示

    的第一个参数 
    final_output = tf.matmul(first_output, A2)

    不是矩阵。所以first_output 不行。它来自上一行：

    first_output = tf_gaussian_function(euclid_dist)

    所以我们需要检查该函数tf_gaussian_function()，但这不在发布的代码中。我检查了它是否是一个错字，它应该是 tf.gaussian_function() 但没有这样的东西。

    所以请贴出函数tf_gaussian_function()的定义。

    既然我们有了这个，事实证明您正在尝试将函数 math.exp( -SC * math.pow( input_layer, 2 ) )（通常称为 rho 或 ρ）映射到 euclid_dist ;我认为那将是您的径向基函数。旁注：看起来您正在尝试通过实现自己的渐变和所有内容来完成一些繁重的工作，通过np.vectorize() 和所有这些。我建议你实现一些更简单的东西，比如这段代码（未经测试）：

    first_output = tf.exp( -SC * euclid_dist * euclid_dist )

    （我使用x * x 而不是等效的tf.pow( x, 2 )，因为第一个更有效。） 这应该创建自己的渐变并照顾您想要实现的所有目标。但无论如何，你听起来对你的实现很有信心，我发现here 很可能就是这种情况；您还可以仔细检查它是否产生与此单线相同的结果。

    就euclid_dist 而言，tf.norm( x, ord = "euclidean", axis = None ) 正在返回 2nd 范数（或欧几里得范数，也称为平方和的平方根），因此是绝对是一个标量。 （它正是上面注释掉的行所说的：euclid_dist = tf.sqrt(tf.reduce_sum(tf.pow(tf.subtract(inputs_with_weights, c), 2)))。）不确定你想在那里实现什么？如果你只想要距离的平方，那么你应该这样做：

    euclid_dist = ( inputs_with_weights - c ) * ( inputs_with_weights - c )

    但据我了解，在径向基函数网络中，首先对输入应用权重，然后是径向基函数，第三步是欧几里得距离作为成本函数，这可能非常最好是一个标量。

    如果该行是您尝试为您的 RBF 网络实现 规范化，请注意 normalization is generally done 有点不同。因此，如果我正确理解您的想法，您的实现可能看起来更类似于此：

    from __future__ import print_function
import tensorflow as tf
from tensorflow.python.framework import ops
import numpy as np

x_data = tf.placeholder(shape=[1, 3], dtype=tf.float32)
y_target = tf.placeholder(shape=[1, 1], dtype=tf.float32)

# Create variables for NN layers
first_layer_nodes = 5
A1 = tf.Variable(tf.random_normal(shape=[3, first_layer_nodes]))  # input -> first layer nodes
A2 = tf.Variable(tf.random_normal(shape=[first_layer_nodes, 1]))  # first_layer nodes -> sum node
c = tf.Variable(tf.random_normal(shape=[first_layer_nodes]))  # centroids

# Declare NN
SC = 1
def rho( distances ): return tf.exp( -SC * distances * distances )
def norm( x ): return x / tf.reduce_sum( x, axis = -1 )
inputs_with_weights = tf.matmul( x_data, A1 )
print(inputs_with_weights)
distances = inputs_with_weights - c
print( distances )
first_output = norm( rho( distances ) ) # tf_gaussian_function(distances) # 
print(first_output)
final_output = tf.matmul(first_output, A2)

with tf.Session() as sess:
    sess.run( tf.global_variables_initializer() )
    r = sess.run( [ first_output, final_output ], feed_dict = {
                  x_data : np.array( [ [ 1.0, 2, 3 ] ] ) } )
    for v in r:
        print( v )

    输出：

    张量("MatMul_15:0", shape=(1, 5), dtype=float32)
    张量("sub_8:0", shape=(1, 5), dtype=float32)
    张量("div_5:0", shape=(1, 5), dtype=float32)
    [[4.4366708e-03 6.8647589e-04 5.9621310e-01 7.5066246e-06 3.9865622e-01]]
    [[0.31285414]]

    回复评论，您发布此代码片段的位置：

    exp_list = []
for i in range(first_layer_nodes):
    euclid_distance = tf.reduce_sum(tf.square(tf.subtract(x_data, c[i, :])), 1)
    exp_list.append(tf.exp(-SC * euclid_distance))
phi = tf.transpose(tf.stack(exp_list))

    这可以被矢量化，利用 tf.subtract() 上的隐式广播和像这样的高级索引（未经测试）：

    ed = tf.reduce_sum( tf.square( tf.subtract( x_data[ None, ... ], c ) ), 2 )
phi = tf.transpose( tf.exp( -SC * ed ) )

    【讨论】：

    • 我将代码放入帖子的编辑中。用 tf_gaussian_function() 倒是没问题。因为如果输入 input_with_weights 效果很好（但当然没有 RBF 网络）。问题出在 tf.norm 中，它返回我标量并且不知道为每个节点执行正确操作的设置。
    • 我认为：distances = inputs_with_weights - c 不正确 - 欧式距离是：二维情况下的 sqrt((x1 - x2)**2 + (y1 - y2)**2)
    • 你这里没有 Y！没有两个坐标！即使是质心，它们也只有一个坐标！一维的欧式距离实际上是abs(x2 - x1)。
    • 对不起，我不是故意大喊大叫的，我只是在这个答案上花了很多时间，看起来我可能误解了你的问题，这让我很沮丧。 c = tf.Variable(tf.random_normal(shape=[first_layer_nodes])) 让我觉得每个质心只有一个坐标。我弄错了吗？
    • 对不起，这是我的错。我做我的网络不好。现在我正在寻找一些优化：link
    猜你喜欢
    • 1970-01-01
    • 2020-11-29
    • 2018-02-14
    • 2013-04-07
    • 2021-01-31
    • 2015-09-23
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多
    热门标签
    Java Python linux javascript Mysql C# Docker 算法 前端 SpringBoot Redis Vue spring 设计模式 .net core .net kubernetes c++ 数据库 数据结构 大数据 js 机器学习 微服务 Android Go 程序员 面试 JVM ASP.net core 云原生 人工智能 后端 PHP git CSS golang k8s Nginx Django mybatis 深度学习 多线程 React 架构 devops 爬虫 云计算 Spring Boot LeetCode