【问题标题】:Run distributed TensorFlow, UnavailableError: Endpoint read fail运行分布式TensorFlow,UnavailableError: Endpoint read fail
【发布时间】:2018-01-25 12:19:57
【问题描述】:

我是 TensorFlow 新手,没有太多经验。我现在正在尝试分布式 TensorFlow。

按照官方指南,我首先创建了两个服务器。我在两个单独的终端中运行以下代码

import sys
import tensorflow as tf

task_number = int(sys.argv[1])

cluster = tf.train.ClusterSpec({"local": ["localhost:2222", "localhost:2223"]})
server = tf.train.Server(cluster, job_name="local", task_index=task_number)

print("Starting server #{}".format(task_number))

server.start()
server.join()

服务器设置成功

2018-01-25 20:05:37.651802: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:215] Initialize GrpcChannelCache for job local -> {0 -> localhost:2222, 1 -> localhost:2223}
2018-01-25 20:05:37.652881: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:324] Started server with target: grpc://localhost:2222
Starting server #0
2018-01-25 20:05:37.652938: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:328] Server already started (target: grpc://localhost:2222)

然后我运行以下程序

import tensorflow as tf
x = tf.constant(2)

with tf.device("/job:local/task:1"):
    y2 = x - 66

with tf.device("/job:local/task:0"):
    y1 = x + 300
    y = y1 + y2

with tf.Session("grpc://localhost:2223") as sess:
    result = sess.run(y)
    print(result)

然后它给了我以下错误信息

E0125 20:05:49.573488650   10292 ev_epoll1_linux.c:1051]     grpc epoll fd: 5
Traceback (most recent call last):
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1323, in _do_call
    return fn(*args)
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1293, in _run_fn
    self._extend_graph()
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1354, in _extend_graph
    self._session, graph_def.SerializeToString(), status)
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.UnavailableError: Endpoint read failed

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/****/Documents/intern/sample_data/try.py", line 25, in <module>
    result = sess.run(y)
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/home/****/miniconda3/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.UnavailableError: Endpoint read failed

我用谷歌搜索了它,有人认为这可能是代理的问题,所以我禁用了代理,但没有任何改变。

有人知道问题可能是什么吗?非常感谢。

【问题讨论】:

    标签: python tensorflow distributed-computing grpc


    【解决方案1】:

    没关系,问题解决了。这是关于代理的设置。我们需要在服务器和客户端上取消设置代理才能使程序正常工作。

    【讨论】:

      猜你喜欢
      • 2016-10-09
      • 2016-10-10
      • 2016-06-24
      • 2019-07-11
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2017-12-08
      • 1970-01-01
      相关资源
      最近更新 更多