【发布时间】:2020-07-18 19:03:55
【问题描述】:
每当我尝试部署更新后的 AppEngine node.js 网站(之前是节点 6.14.x,但我已更新到 12.x 作为解决此问题的一部分)时,我都会收到超时错误。在本地或在 Google Cloud Shell 上运行该站点可以立即正常运行,我也可以点击运行状况检查网址。
我通常使用 Google Cloud Shell 来部署我的 prod 环境,但也曾在我的本地机器上进行过尝试并得到了相同的结果。无论哪种情况,几天来我一直收到此错误:
googlecloudsdk.api_lib.app.operations_util.OperationError: Error Response: [4] Timed out waiting for the flex deployment to become network provisioned.
ERROR: (gcloud.app.deploy) Error Response: [4] Timed out waiting for the flex deployment to become network provisioned.
在 Google Cloud Shell 的一次尝试中,我收到以下错误:
Error Response: [4] Your deployment has failed to become healthy in the allotted time and therefore was rolled back. If you believe this was an error, try adjusting the 'app_start_timeout_sec' setting in the 'readiness_check' section.
我尝试将 app_start_timeout_sec 更新为 600 并得到相同的错误 "Error Response: [4] Your deployment has failed to become healthy in the allotted time and therefore was rolled back. If you believe this was an error, try adjusting the 'app_start_timeout_sec' setting in the 'readiness_check' section."
然后我尝试将其更新到 1800 并得到原来的"ERROR: (gcloud.app.deploy) Error Response: [4] Timed out waiting for the app infrastructure to become healthy."
显然,每次部署都需要更长的时间才能失败。
部署中的详细日志记录并没有真正显示出任何有用的信息,但请参见下文。我不确定是否还有其他地方我应该寻找线索,但从四处寻找任何东西都不是问题。
我在 Google 问题跟踪器(例如 https://issuetracker.google.com/issues/73583699)上查看了几张工单,并尝试了一些建议,但我没有发现任何问题(日志中没有关于资源的内容,在设置中注意到关于达到配额的内容。 ) 我在那里开了一张票,但他们把它关了说
“我试图通过制作一些 App Engine 来重现这个问题 部署,但无法获得您遇到的错误消息。 此外,我还没有发现其他报告相同的案例或问题 行为,因此到目前为止,该问题似乎与您的环境无关。”
警告:这显然是我在 2020 年尝试进行生产的第一次部署,所以也许我需要做一些改变?看起来 2019-10-12 是最后一次构建。
那么,关于我可以做些什么来找到错误或如何解决它的任何提示?
感谢您的帮助。
app.yaml
(更新,我也尝试注释掉 readiness_check 部分,但没有区别)。
# [START runtime]
runtime: nodejs
env: flex
automatic_scaling:
min_num_instances: 1
max_num_instances: 5
cool_down_period_sec: 180
cpu_utilization:
target_utilization: 0.8
readiness_check:
path: "/health"
check_interval_sec: 5
timeout_sec: 4
failure_threshold: 2
success_threshold: 2
app_start_timeout_sec: 300
env_variables:
NODE_ENV: 'production'
FIREBASE_SERVICE_PROJECT_ID: 'blah'
FIREBASE_SERVICE_PRIVATE_KEY_ID: 'blah'
FIREBASE_SERVICE_PRIVATE_KEY: "blah"
FIREBASE_SERVICE_CLIENT_EMAIL: 'blah'
FIREBASE_SERVICE_CLIENT_ID: 'blah'
FIREBASE_SERVICE_CLIENT_X509_CERT_URL: 'blah'
GOOGLE_CLOUD_STORAGE_BUCKET: 'blah'
AUTH_SECRET: 'blah'
AUTH_USERNAME: 'blah'
AUTH_PASSWORD: 'blah'
LEAGUE_TABLE_URL: 'blah'
# [END runtime]
详细的“gcloud 应用部署”日志
...
Updating service [default] (this may take several minutes)...⠏DEBUG: Operation [apps/daggers-72eb6/operations/b7ef6a92-1ce7-457a-a85a-ef32ae235cf9] not complete. Waiting to retry.
Updating service [default] (this may take several minutes)...⠏DEBUG: Operation [apps/daggers-72eb6/operations/b7ef6a92-1ce7-457a-a85a-ef32ae235cf9] complete. Result: {
"done": true,
"error": {
"code": 4,
"message": "Timed out waiting for the flex deployment to become network provisioned."
},
"metadata": {
"@type": "type.googleapis.com/google.appengine.v1.OperationMetadataV1",
"insertTime": "2020-07-14T22:46:01.763Z",
"method": "google.appengine.v1.Versions.CreateVersion",
"target": "apps/blah/services/default/versions/20200714t154223",
"user": "blah"
},
"name": "apps/blah/operations/blah"
}
Updating service [default] (this may take several minutes)...failed.
DEBUG: (gcloud.app.deploy) Error Response: [4] Timed out waiting for the flex deployment to become network provisioned.
Traceback (most recent call last):
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/calliope/cli.py", line 983, in Execute
resources = calliope_command.Run(cli=self, args=args)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/calliope/backend.py", line 808, in Run
resources = command_instance.Run(args)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/surface/app/deploy.py", line 117, in Run
default_strategy=flex_image_build_option_default))
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 651, in RunDeploy
ignore_file=args.ignore_file)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/command_lib/app/deploy_util.py", line 437, in Deploy
extra_config_settings)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/appengine_api_client.py", line 208, in DeployService
poller=done_poller)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 314, in WaitForOperation
sleep_ms=retry_interval)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 264, in WaitFor
sleep_ms, _StatusUpdate)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 326, in PollUntilDone
sleep_ms=sleep_ms)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/core/util/retry.py", line 229, in RetryOnResult
if not should_retry(result, state):
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/api_lib/util/waiter.py", line 320, in _IsNotDone
return not poller.IsDone(operation)
File "/Users/ashleydb/Applications/google-cloud-sdk/lib/googlecloudsdk/api_lib/app/operations_util.py", line 183, in IsDone
encoding.MessageToPyValue(operation.error)))
googlecloudsdk.api_lib.app.operations_util.OperationError: Error Response: [4] Timed out waiting for the flex deployment to become network provisioned.
ERROR: (gcloud.app.deploy) Error Response: [4] Timed out waiting for the flex deployment to become network provisioned.
更新
在本地一切正常。以下是我尝试过的事情的列表,但都没有解决部署问题:
- 启用拆分运行状况检查
- 注释掉 app.yaml 的这个 readiness_check 部分
- 更新过时的软件包,包括 node 6->12、webpack 1->4 和最新的 @google-cloud/error-reporting 和 @google-cloud/storage API。
- 将此设置为新区域中的全新 GCP 项目
【问题讨论】:
-
深入研究,我怀疑这可能是问题所在(现在进行健康检查的方式发生了变化):cloud.google.com/appengine/docs/flexible/nodejs/reference/…
-
你有
/health的url处理程序吗? -
是的,我已经拥有那个处理程序很长时间了。我已经对其进行了更新,以确保现在每次都返回 200 个代码来尝试调试它,但没有任何变化。
标签: node.js google-app-engine google-cloud-platform