在docker中从requirement.txt构建图像时出错答案

【问题标题】：Error when building image from requirement.txt in docker在docker中从requirement.txt构建图像时出错
【发布时间】：2020-08-12 05:39:42
【问题描述】：

我正在尝试构建一个使用 Python 的 gensim 库版本 3.8.3 的 Docker 应用程序，该库是通过 pip 从 requirements.txt 文件安装的。

但是，Docker 在尝试做的时候似乎遇到了麻烦运行 pip install -r requirements.txt

我的 Requirement.txt 供参考 -

boto==2.49.0
boto3==1.14.33
botocore==1.17.33
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
Cython==0.29.14
docutils==0.15.2
Flask==1.1.2
gensim==3.8.3
idna==2.10
itsdangerous==1.1.0
Jinja2==2.11.2
jmespath==0.10.0
MarkupSafe==1.1.1
numpy==1.19.1
python-dateutil==2.8.1
requests==2.24.0
s3transfer==0.3.3
scipy==1.5.2
six==1.15.0
smart-open==2.1.0
urllib3==1.25.10
Werkzeug==1.0.1

码头文件

FROM python:3.8.2-alpine
WORKDIR /project
ADD . /project
RUN set -x && apk add --no-cache build-base && apk add --no-cache libexecinfo-dev
RUN pip install --upgrade pip
RUN pip install -r requirements.txt
CMD ["python","similarity.py"]

错误：

(venv) C:\Users\verma\PycharmProjects\flaskTest>docker image build -t similarity-flask-api  .
Sending build context to Docker daemon  302.7MB
Step 1/7 : FROM python:3.8.2-alpine
 ---> 6c32e2504283
Step 2/7 : WORKDIR /project
 ---> Using cache
 ---> 554b6bda89ad
Step 3/7 : ADD . /project
 ---> d085a645ecb1
Step 4/7 : RUN set -x && apk add --no-cache build-base && apk add --no-cache libexecinfo-dev
 ---> Running in e7117c1e18ff
+ apk add --no-cache build-base
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/18) Installing libgcc (9.2.0-r4)
(2/18) Installing libstdc++ (9.2.0-r4)
(3/18) Installing binutils (2.33.1-r0)
(4/18) Installing libmagic (5.37-r1)
(5/18) Installing file (5.37-r1)
(6/18) Installing gmp (6.1.2-r1)
(7/18) Installing isl (0.18-r0)
(8/18) Installing libgomp (9.2.0-r4)
(9/18) Installing libatomic (9.2.0-r4)
(10/18) Installing mpfr4 (4.0.2-r1)
(11/18) Installing mpc1 (1.1.0-r1)
(12/18) Installing gcc (9.2.0-r4)
(13/18) Installing musl-dev (1.1.24-r2)
(14/18) Installing libc-dev (0.7.2-r0)
(15/18) Installing g++ (9.2.0-r4)
(16/18) Installing make (4.2.1-r2)
(17/18) Installing fortify-headers (1.1-r0)
(18/18) Installing build-base (0.5-r1)
Executing busybox-1.31.1-r9.trigger
OK: 182 MiB in 52 packages
+ apk add --no-cache libexecinfo-dev
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
(1/2) Installing libexecinfo (1.1-r1)
(2/2) Installing libexecinfo-dev (1.1-r1)
OK: 183 MiB in 54 packages
Removing intermediate container e7117c1e18ff
 ---> 9e7a97f8bddc
Step 5/7 : RUN pip install --upgrade pip
 ---> Running in 0286591e9e70
Collecting pip
  Downloading pip-20.2.1-py2.py3-none-any.whl (1.5 MB)
Installing collected packages: pip
  Attempting uninstall: pip
    Found existing installation: pip 20.1
    Uninstalling pip-20.1:
      Successfully uninstalled pip-20.1
Successfully installed pip-20.2.1
Removing intermediate container 0286591e9e70
 ---> ca837786d695
Step 6/7 : RUN pip install -r requirements.txt
 ---> Running in 7f124c100c0b
Collecting boto==2.49.0
  Downloading boto-2.49.0-py2.py3-none-any.whl (1.4 MB)
Collecting boto3==1.14.33
  Downloading boto3-1.14.33-py2.py3-none-any.whl (129 kB)
Collecting botocore==1.17.33
  Downloading botocore-1.17.33-py2.py3-none-any.whl (6.5 MB)
Collecting certifi==2020.6.20
  Downloading certifi-2020.6.20-py2.py3-none-any.whl (156 kB)
Collecting chardet==3.0.4
  Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting click==7.1.2
  Downloading click-7.1.2-py2.py3-none-any.whl (82 kB)
Collecting Cython==0.29.14
  Downloading Cython-0.29.14.tar.gz (2.1 MB)
Collecting docutils==0.15.2
  Downloading docutils-0.15.2-py3-none-any.whl (547 kB)
Collecting Flask==1.1.2
  Downloading Flask-1.1.2-py2.py3-none-any.whl (94 kB)
Collecting gensim==3.8.3
  Downloading gensim-3.8.3.tar.gz (23.4 MB)
Collecting idna==2.10
  Downloading idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting itsdangerous==1.1.0
  Downloading itsdangerous-1.1.0-py2.py3-none-any.whl (16 kB)
Collecting Jinja2==2.11.2
  Downloading Jinja2-2.11.2-py2.py3-none-any.whl (125 kB)
Collecting jmespath==0.10.0
  Downloading jmespath-0.10.0-py2.py3-none-any.whl (24 kB)
Collecting MarkupSafe==1.1.1
  Downloading MarkupSafe-1.1.1.tar.gz (19 kB)
Processing /root/.cache/pip/wheels/df/b2/64/111c431ca7f7d49afb42126b7351fe1a4894803d75026360de/numpy-1.19.1-cp38-cp38-linux_x86_64.whl
Collecting python-dateutil==2.8.1
  Downloading python_dateutil-2.8.1-py2.py3-none-any.whl (227 kB)
Collecting requests==2.24.0
  Downloading requests-2.24.0-py2.py3-none-any.whl (61 kB)
Collecting s3transfer==0.3.3
  Downloading s3transfer-0.3.3-py2.py3-none-any.whl (69 kB)
Collecting scipy==1.5.2
  Downloading scipy-1.5.2.tar.gz (25.4 MB)
  Installing build dependencies: started
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: still running...
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
    Preparing wheel metadata: started
    Preparing wheel metadata: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpoyjzx5wb
         cwd: /tmp/pip-install-r078skp_/scipy
    Complete output (139 lines):
    lapack_opt_info:
    lapack_mkl_info:
    customize UnixCCompiler
      libraries mkl_rt not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE

    openblas_lapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE

    openblas_clapack_info:
    customize UnixCCompiler
    customize UnixCCompiler
      libraries openblas,lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE

    flame_info:
    customize UnixCCompiler
      libraries flame not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE

    atlas_3_10_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries tatlas,tatlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_3_10_threads_info'>
      NOT AVAILABLE

    atlas_3_10_info:
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries satlas,satlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_3_10_info'>
      NOT AVAILABLE

    atlas_threads_info:
    Setting PTATLAS=ATLAS
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries ptf77blas,ptcblas,atlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_threads_info'>
      NOT AVAILABLE

    atlas_info:
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/local/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib
    customize UnixCCompiler
      libraries lapack_atlas not found in /usr/lib/
    customize UnixCCompiler
      libraries f77blas,cblas,atlas not found in /usr/lib/
    <class 'numpy.distutils.system_info.atlas_info'>
      NOT AVAILABLE

    accelerate_info:
      NOT AVAILABLE

    lapack_info:
    customize UnixCCompiler
      libraries lapack not found in ['/usr/local/lib', '/usr/lib', '/usr/lib/']
      NOT AVAILABLE

    lapack_src_info:
      NOT AVAILABLE

      NOT AVAILABLE

    setup.py:460: UserWarning: Unrecognized setuptools command ('dist_info --egg-base /tmp/pip-modern-metadata-ujofw06w'), proceeding with generating Cython sources
and expanding templates
      warnings.warn("Unrecognized setuptools command ('{}'), proceeding with "
    Running from SciPy source directory.
    /tmp/pip-build-env-mw61mr08/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) libraries not found.
        Directories to search for the libraries can be specified in the
        numpy/distutils/site.cfg file (section [lapack]) or by setting
        the LAPACK environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
    /tmp/pip-build-env-mw61mr08/overlay/lib/python3.8/site-packages/numpy/distutils/system_info.py:1712: UserWarning:
        Lapack (http://www.netlib.org/lapack/) sources not found.
        Directories to search for the sources can be specified in the
        numpy/distutils/site.cfg file (section [lapack_src]) or by setting
        the LAPACK_SRC environment variable.
      if getattr(self, '_calc_info_{}'.format(lapack))():
    Traceback (most recent call last):
      File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
        main()
      File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "/usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 133, in prepare_metadata_for_build_wheel
        return hook(metadata_directory, config_settings)
      File "/tmp/pip-build-env-mw61mr08/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 157, in prepare_metadata_for_build_wheel
        self.run_setup()
      File "/tmp/pip-build-env-mw61mr08/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 248, in run_setup
        super(_BuildMetaLegacyBackend,
      File "/tmp/pip-build-env-mw61mr08/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 142, in run_setup
        exec(compile(code, __file__, 'exec'), locals())
      File "setup.py", line 583, in <module>
        setup_package()
      File "setup.py", line 579, in setup_package
        setup(**metadata)
      File "/tmp/pip-build-env-mw61mr08/overlay/lib/python3.8/site-packages/numpy/distutils/core.py", line 137, in setup
        config = configuration()
      File "setup.py", line 477, in configuration
        raise NotFoundError(msg)
    numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/local/bin/python /usr/local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py prepare_metadata_for_bu
ild_wheel /tmp/tmpoyjzx5wb Check the logs for full command output.
The command '/bin/sh -c pip install -r requirements.txt' returned a non-zero code: 1

我试过这个帖子 - Docker unable to install numpy, scipy, or gensim 正如建议的那样，我在 dockerFile 中添加了第 4 行和第 5 行，但它仍然无法正常工作。

【问题讨论】：

标签： python docker gensim

【解决方案1】：

在你提到的帖子中，你安装libc-dev 来编译包...
你没有。

RUN apt-get -y install libc-dev
RUN apt-get -y install build-essential

我在尝试将“alpine”与 Python 一起使用时遇到问题... 所以我们选择"slim-buster"作为Python的docker镜像。

如果可以的话……

1 - 如果可以的话，我会尝试瘦身版
2 - 尝试一个 numpy 就绪的 docker 镜像并安装你的 python 包。

【讨论】：

/bin/sh: apt-get: not found 命令 '/bin/sh -c apt-get -y install libc-dev' 返回了一个非零代码：127 我试过这样做但是得到这个
有问题的图像是alpine 图像，它不使用apt 作为包管理器
正如我告诉你的那样...... Alpine 真的是“干”的形象。你能考虑一下我告诉你的那个选项吗... slim-buster hub.docker.com/_/python
@CarlosLeite 刚刚使用 slim-buster 而不是 alpine，这使我的图像大小从 1.4 GB 减少了约 600 mbs，849 mb。您非常正确地提到 alpine 是一个“干”图像，并且在使用它时我必须在 docker 文件中使用一些额外的层，这会导致更大的图像。 slim-buster 对于我的情况来说似乎是一个更好的选择。谢谢你:)
很高兴它有帮助。很高兴能提供帮助。

【解决方案2】：

要在 alpine 映像上安装 numpy，您通常需要更多的依赖项：

RUN apk update && apk add gfortran build-base openblas-dev libffi-dev

即你所缺少的 openblas-dev。这至少会让numpy 安装

【讨论】：

虽然这可行，但现在我的图像大小是 1.39 GB，你能建议我应该怎么做以最小化大小和构建时间。
我的建议是使用安装了这些依赖项的图像。不幸的是，很多依赖项都是从源代码编译的，因此需要构建时间，所以这是一个失败的开始。对于大小，我建议查看the builder pattern