如何在 Google Cloud Platform 上加速 bazel 构建

Question

我正在使用 gcr.io/cloud-builders/bazel 在 google 容器构建上构建我的图像。

从日志来看，它似乎大部分时间都在为 bazel 设置工作区。此工作区不会因构建而异，因此我认为可以预先计算并存储在新映像中。

如何在 Google Cloud Platform 上加快这些 bazel 构建速度？

Already have image (with digest): gcr.io/cloud-builders/bazel
Extracting Bazel installation...
........................
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: com_github_golang_protobuf : new_go_repository is deprecated. Please migrate to go_repository soon.
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: com_github_golang_glog : new_go_repository is deprecated. Please migrate to go_repository soon.
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: org_golang_google_grpc : new_go_repository is deprecated. Please migrate to go_repository soon.
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: org_golang_x_net : new_go_repository is deprecated. Please migrate to go_repository soon.
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: com_github_gogo_protobuf : new_go_repository is deprecated. Please migrate to go_repository soon.
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: com_github_grpc_ecosystem_grpc_gateway : new_go_repository is deprecated. Please migrate to go_repository soon.
WARNING: /builder/home/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/external/io_bazel_rules_go/go/def.bzl:137:3: DEPRECATED: org_golang_google_genproto : new_go_repository is deprecated. Please migrate to go_repository soon.
Analyzing: target //:docker (4 packages loaded)
Analyzing: target //:docker (5 packages loaded)
Analyzing: target //:docker (6 packages loaded)
Analyzing: target //:docker (76 packages loaded)
Analyzing: target //:docker (78 packages loaded)
Analyzing: target //:docker (78 packages loaded)
Analyzing: target //:docker (79 packages loaded)
Analyzing: target //:docker (79 packages loaded)
Analyzing: target //:docker (80 packages loaded)
Analyzing: target //:docker (160 packages loaded)
Analyzing: target //:docker (160 packages loaded)
Analyzing: target //:docker (160 packages loaded)
Analyzing: target //:docker (160 packages loaded)
Analyzing: target //:docker (174 packages loaded)
Analyzing: target //:docker (174 packages loaded)
Analyzing: target //:docker (211 packages loaded)
INFO: Analysed target //:docker (286 packages loaded).
INFO: Found 1 target...
[0 / 1] BazelWorkspaceStatusAction stable-status.txt
[17 / 19] SHA256 external/golang/image/002.tar.gz.nogz.sha256
[42 / 281] no action running
[51 / 290] no action running
[63 / 302] no action running
[80 / 319] no action running
[101 / 340] no action running
[118 / 358] no action running
[138 / 378] no action running
[168 / 407] Compiling external/com_github_google_protobuf/src/google/protobuf/compiler/csharp/csharp_message_field.cc [for host]; 0s linux-sandbox
[200 / 439] Compiling external/com_github_google_protobuf/src/google/protobuf/extension_set.cc [for host]; 1s linux-sandbox
INFO: From ProtoCompile assets/assets/assets.pb.gw.go:
assets/assets/assets.proto: warning: Import recipes.proto but not used.
INFO: From ProtoCompile assets/assets/assets.swagger.json:
assets/assets/assets.proto: warning: Import recipes.proto but not used.
INFO: From ProtoCompile assets/assets/assets.pb.go:
assets/assets/assets.proto: warning: Import recipes.proto but not used.
[522 / 697] GoCompile vendor/github.com/gorilla/mux/~lib~/go_default_library.o; 0s linux-sandbox
Target //:docker up-to-date:
bazel-bin/docker-layer.tar
INFO: Elapsed time: 331.343s, Critical Path: 13.73s
INFO: Build completed successfully, 737 total actions

编辑：添加 cloudbuild 文件

cloudbuild.yaml：

steps:
- name: gcr.io/cloud-builders/bazel
  args: ['run', '//:docker']
- name: gcr.io/cloud-builders/docker
  args: ['tag', 'bazel:docker', '${_IMAGE_TAG}']
images: ['${_IMAGE_TAG}']

Answer 1

Container Builder 目前不提供此功能作为内置功能，但您可以自己完成。

1. 将一个或多个volumes 添加到输出您希望在以后的构建中恢复的信息的构建步骤中。 （见documentation on volumes。）
2. 如有必要，请修改这些构建步骤，以将要保留的数据输出到卷提供的路径中。
3. 在后续构建步骤中，使用gsutil build step 将卷中的所有数据保存到 GCS 存储桶中。 （如果您愿意，当然可以使用其他一些实用程序将其保存到其他地方。）

在以后的构建中，使用gsutil 将数据恢复到所需的卷，并将卷挂载到后续的构建步骤中。

Answer 2

现在使用Bazel Remote Caching 有一种更简单的方法。

Create a Cloud Storage bucket，然后在您的.bazelrc 中设置以下内容：

build --remote_cache=https://storage.googleapis.com/<your bucket name>
build --google_default_credentials

Bazel 会自动在您的 Cloud Storage 存储分区中缓存操作，并且它们会在构建之间保持不变。您可以在存储桶上set automatic lifecycle management 删除一定天数后的操作。