对 Google Cloud ML 的请求超时

Question

我正在执行从 Google App Engine 到 Google Cloud ML 的请求（在线预测）（我没有创建模型），并且不时收到异常“等待来自 URL 的 HTTP 响应时超出了最后期限” 完整的跟踪在这里：

    Deadline exceeded while waiting for HTTP response from URL: https://ml.googleapis.com/v1/projects/project-id/models/my-model/versions/v3:predict?alt=json (/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py:1552)
Traceback (most recent call last):
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
    rv = self.handle_exception(request, response, e)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
    rv = self.router.dispatch(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
    return route.handler_adapter(request, response)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
    return handler.dispatch()
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
    return self.handle_exception(e, self.app.debug)
  File "/base/data/home/runtimes/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
    return method(*args, **kwargs)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/main.py", line 90, in post
    response = predict(batch_obj=batch_data_obj)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/run_cloud_predict.py", line 88, in predict
    response = request.execute()
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/oauth2client/util.py", line 135, in positional_wrapper
    return wrapped(*args, **kwargs)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/googleapiclient/http.py", line 835, in execute
    method=str(self.method), body=self.body, headers=self.headers)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/googleapiclient/http.py", line 162, in _retry_request
    resp, content = http.request(uri, method, *args, **kwargs)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/oauth2client/client.py", line 631, in new_request
    redirections, connection_type)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/httplib2/__init__.py", line 1659, in request
    (response, content) = self._request(conn, authority, uri, request_uri, method, body, headers, redirections, cachekey)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/httplib2/__init__.py", line 1399, in _request
    (response, content) = self._conn_request(conn, request_uri, method, body, headers)
  File "/base/data/home/apps/s~project-id/1.402312581449917691/lib/httplib2/__init__.py", line 1355, in _conn_request
    response = conn.getresponse()
  File "/base/data/home/runtimes/python27/python27_dist/lib/python2.7/gae_override/httplib.py", line 526, in getresponse
    raise HTTPException(str(e))
HTTPException: Deadline exceeded while waiting for HTTP response from URL: https://ml.googleapis.com/v1/projects/project-id/models/my-model/versions/v3:predict?alt=json

现在我知道 Google App Engine 有 60 秒的响应限制，这就是我使用任务队列进行请求的原因。我还尝试了以下事情：

URLFETCH_DEADLINE = 3600
urlfetch.set_default_fetch_deadline(URLFETCH_DEADLINE)
socket.setdefaulttimeout(URLFETCH_DEADLINE)

我正在这样构建api客户端

import httplib2
from googleapiclient import discovery
from oauth2client import service_account

credentials = service_account.ServiceAccountCredentials.from_json_keyfile_name('credentials-file', scopes)
http = httplib2.Http(timeout=36000)
http = credentials.authorize(http)

ml = discovery.build('ml', 'v1', http=http)
request = ml.projects().predict(name=predict_ver_name, body=request_data)

有趣的是，有时超时发生在 70 秒左右（69.9、70、70.1 等），有时在 120 秒左右（119.8、120.1 等），这告诉我，这可能与一些内部 Cloud ML 交易有关。 我正在通过任务队列并行执行几十个请求。成功的响应时间从几秒到 ~110 秒 如果有人有类似的经验或者可以给我建议如何解决这个问题，我只是好奇，即是什么导致了截止日期。

Answer 1

感谢您发布您的经验。 - 有一些启动成本，并且取决于请求的速率，它可能需要启动多个服务器来满足需求。 - 您尝试预测的模型的大小是多少？较大的模型往往具有较高的启动成本。

谢谢。

Answer 2

您可以使用下面的代码在 api 客户端轻松设置超时。

import socket
timeout_in_sec = 60*3 # 3 minutes timeout limit
socket.setdefaulttimeout(timeout_in_sec)

然后你可以像往常一样创建你的 ML 服务对象，它会有延长的超时限制。

ml_service = discovery.build('ml', 'v1')