--test_env 和 --test_arg 对 bazel 缓存的影响

Question

我天真地将一些可变的测试元数据传递给一些py_test 目标，以将该元数据注入一些稍后上传到云的测试结果工件中。我在bazel test 调用中使用--test_env 或--test_arg 值。

这种可变数据是否会对测试结果的缓存方式产生负面影响，以至于背靠背运行相同的测试会有效地干扰 bazel 缓存？

Answer 1

命令行输入

命令行输入确实会干扰缓存命中。考虑以下一组执行

构建文件

py_test(
    name = "test_args",
    srcs = ["test_args.py"],
    deps = [
        ":conftest",
        "@pytest",
    ],
)

py_library(
    name = "conftest",
    srcs = ["conftest.py"],
    deps = [
        "@pytest",
    ],
)

测试模块

import sys
import pytest

def test_pass():
    assert True

def test_arg_in(request):
    assert request.config.getoption("--metadata")

if __name__ == "__main__":
    args = sys.argv[1:]
    ret_code = pytest.main("--log-level=ERROR"] + args)

    # Exit with return code
    sys.exit(ret_code)

第一次执行

$ bazel test //bazel_check:test_args --test_arg --metadata=abc
INFO: Analyzed target //bazel_check:test_args (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.40s
INFO: Critical path 0.57s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.72s (preparation 0.12s, execution 0.60s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_args                                                  PASSED in 0.4s

Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions

第二次执行：相同的参数值，缓存命中！

$ bazel test //bazel_check:test_args --test_arg --metadata=abc
INFO: Analyzed target //bazel_check:test_args (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 1 process: 1 internal (100.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.00s
INFO: Critical path 0.47s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.61s (preparation 0.12s, execution 0.49s)
INFO: Build completed successfully, 1 total action
//bazel_check:test_args                                         (cached) PASSED in 0.4s

Executed 0 out of 1 test: 1 test passes.
INFO: Build completed successfully, 1 total action

第三次执行：新的参数值，没有缓存命中

$ bazel test //bazel_check:test_args --test_arg --metadata=kk
INFO: Analyzed target //bazel_check:test_args (0 packages loaded, 93 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.30s
INFO: Critical path 0.54s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.71s (preparation 0.14s, execution 0.57s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_args                                                  PASSED in 0.3s

Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions

第四次执行：重复使用与前两次运行相同的参数

有趣的是，尽管结果被较早地缓存，但没有缓存命中。不知何故，它没有持续存在。

$ bazel test //bazel_check:test_args --test_arg --metadata=abc
INFO: Analyzed target //bazel_check:test_args (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
INFO: 2 processes: 1 internal (50.00%), 1 local (50.00%).
INFO: Cache hit rate for remote actions: -- (0 / 0)
INFO: Total action wall time 0.34s
INFO: Critical path 0.50s (setup 0.00s, action wall time 0.00s)
INFO: Elapsed time 0.71s (preparation 0.17s, execution 0.55s)
INFO: Build completed successfully, 2 total actions
//bazel_check:test_args                                                  PASSED in 0.3s

Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 2 total actions

环境输入