为 python3 构建张量流/服务失败

Question

系统信息

OS Platform and Distribution:  Linux  CentOS7.2
TensorFlow Serving installed from (source or binary):  Source
TensorFlow Serving version:  r1.9
Python Version: 3.6.2

描述问题

centos7.2默认安装python2.7，我想为python3构建tensorflow-serving，所以我用pyenv安装了一个python3.6.2，在pyenv virtualenv中，我克隆了tensorflow/serving r1.9 , 以 bazel 从源代码构建它。但最后它失败了，因为“import mok”错误。我在 serving/tools/bazel.rc 文件中看到，它使用 /usr/bin/python 作为 python 运行器，所以我将文件更改为我的 python3.6.2，但又失败了。有错误 " _mod = imp.load_module('_pywrap_tensorflow_internal', fp, 路径名, 描述) 导入错误：/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/ _pywrap_tensorflow_internal.so：未定义符号：_Py_FalseStruct"

复制的确切步骤

我把bazel.rc文件改成如下

build:cuda --crosstool_top=@local_config_cuda//crosstool:toolchain
build:cuda --define=using_cuda=true --define=using_cuda_nvcc=true

build --action_env PYTHON_BIN_PATH="/home/pyenv/.pyenv/versions/tensorflow-serving/bin/python"
build --define PYTHON_BIN_PATH=/home/pyenv/.pyenv/versions/tensorflow-serving/bin/python   # tensorflow-serving is a virtualenv from python3.6.2 installed by pyenv 

build --spawn_strategy=standalone --genrule_strategy=standalone
test --spawn_strategy=standalone --genrule_strategy=standalone
run --spawn_strategy=standalone --genrule_strategy=standalone

build --define=grpc_no_ares=true
build --define with_hdfs_support=true  # support HDFS

# TODO(b/69809703): Remove once no longer required for TensorFlow to build.
build --copt=-DGEMMLOWP_ALLOW_SLOW_SCALAR_FALLBACK

源代码/日志

构建也失败了，错误日志如下

ERROR: 

/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/external/org_tensorflow/tensorflow/BUILD:541:1: Executing genrule @org_tensorflow//tensorflow:python_api_gen failed (Exit 1): bash failed: error executing command 
  (cd /home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving && \
  exec env - \
    PATH=/home/pyenv/.pyenv/plugins/pyenv-virtualenv/shims:/home/pyenv/.pyenv/shims:/home/pyenv/.pyenv/bin:/usr/local/ffmpeg/bin/:/opt/jdk1.8.0_112/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/home/pyenv/.local/bin:/home/pyenv/bin \
    PYTHON_BIN_PATH=/home/pyenv/.pyenv/versions/tensorflow-serving/bin/python \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api --root_init_template=external/org_tensorflow/tensorflow/api_template.__init__.py --apidir=bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/app/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/bitwise/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/compat/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/data/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/distributions/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/distributions/bijectors/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/errors/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/estimator/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/estimator/export/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/estimator/inputs/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/feature_column/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/gfile/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/graph_util/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/image/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/initializers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/activations/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/densenet/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/inception_resnet_v2/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/inception_v3/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/mobilenet/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/nasnet/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/resnet50/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/vgg16/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/vgg19/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/applications/xception/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/backend/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/callbacks/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/constraints/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/boston_housing/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/cifar10/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/cifar100/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/fashion_mnist/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/imdb/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/mnist/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/datasets/reuters/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/estimator/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/initializers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/layers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/losses/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/metrics/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/models/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/optimizers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/preprocessing/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/preprocessing/image/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/preprocessing/sequence/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/preprocessing/text/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/regularizers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/utils/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/wrappers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/keras/wrappers/scikit_learn/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/layers/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/linalg/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/logging/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/losses/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/manip/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/math/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/metrics/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/nn/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/nn/rnn_cell/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/profiler/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/python_io/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/resource_loader/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/strings/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/builder/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/constants/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/loader/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/main_op/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/signature_constants/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/signature_def_utils/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/tag_constants/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/saved_model/utils/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/sets/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/sparse/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/spectral/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/summary/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/sysconfig/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/test/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/train/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/train/queue_runner/__init__.py bazel-out/k8-opt/genfiles/external/org_tensorflow/tensorflow/user_ops/__init__.py')
Traceback (most recent call last):
  File "/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/tf_serving/../org_tensorflow/tensorflow/tools/api/generator/create_python_api.py", line 27, in <module>
    from tensorflow.python.util import tf_decorator
  File "/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 49, in <module>
    from tensorflow.python import pywrap_tensorflow
  File "/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
    raise ImportError(msg)
ImportError: Traceback (most recent call last):
  File "/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
    from tensorflow.python.pywrap_tensorflow_internal import *
  File "/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
    _pywrap_tensorflow_internal = swig_import_helper()
  File "/home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
    _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
ImportError: /home/pyenv/.cache/bazel/_bazel_pyenv/3e832003f40725f2f414b6fe95aca127/execroot/tf_serving/bazel-out/host/bin/external/org_tensorflow/tensorflow/tools/api/generator/create_python_api.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so: undefined symbol: _Py_FalseStruct

那我如何在 CentOS7 上构建带有 python3.6.2 支持的 tensorflow-serving r1.9？

Answer 1

将 tools/bazel.rc 改成 config python path 可以解决这个问题。 但这对于 tensorflow-serving r1.10 或 r1.11 来说似乎不是这样