Heroku Python 二进制库依赖答案

【问题标题】：Heroku Python binary library dependencyHeroku Python 二进制库依赖
【发布时间】：2015-01-15 21:21:30
【问题描述】：

我需要在我的 Heroku Django 项目上运行需要 Python 包 dm.xmlsec.binding 的代码。反过来，这取决于安装二进制 apt 包 libxmlsec1，通常通过 apt-get install。

这就是我在 Ubuntu vagrant box 上本地运行的方式。

部署到 Heroku：方法

我的应用程序中有两个“构建包”。使用heroku-buildpack-multi 启用多个构建包。

由于我的应用程序是 Python 应用程序，因此我的 .buildpacks 内容是：

https://github.com/ddollar/heroku-buildpack-apt
https://github.com/heroku/heroku-buildpack-python

第一个 buildpack 引用heroku-buildpack-apt，需要Aptfile 来指定apt 包；在我们的例子中，这个文件的内容是：

python-setuptools
libxmlsec1
libxmlsec1-dev
swig
python-m2crypto

第二个构建包是“Heroku buildpack for Python apps, powered by pip”。

到目前为止一切顺利。

部署到 Heroku：问题

问题是，虽然apt 包看起来安装成功，但当pip install 实际运行时，似乎没有找到已安装的apt 包。

如果 apt 包安装正确，为什么 Python 在后续的 pip install 中找不到它们？

这是完整的git push heroku日志：

git push heroku
Fetching repository, done.
Counting objects: 12, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (8/8), done.
Writing objects: 100% (9/9), 932 bytes | 0 bytes/s, done.
Total 9 (delta 3), reused 0 (delta 0)

-----> Fetching custom git buildpack... done
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/ddollar/heroku-buildpack-apt
=====> Detected Framework: Apt
-----> Updating apt caches
       Ign http://archive.ubuntu.com trusty InRelease
       Ign http://archive.ubuntu.com trusty-security InRelease
       Ign http://archive.ubuntu.com trusty-updates InRelease
       Hit http://archive.ubuntu.com trusty Release.gpg
       Get:1 http://archive.ubuntu.com trusty-security Release.gpg [933 B]
       Get:2 http://archive.ubuntu.com trusty-updates Release.gpg [933 B]
       Hit http://archive.ubuntu.com trusty Release
       Get:3 http://archive.ubuntu.com trusty-security Release [62.0 kB]
       Get:4 http://archive.ubuntu.com trusty-updates Release [62.0 kB]
       Hit http://archive.ubuntu.com trusty/main amd64 Packages
       Hit http://archive.ubuntu.com trusty/universe amd64 Packages
       Hit http://archive.ubuntu.com trusty/main Translation-en
       Hit http://archive.ubuntu.com trusty/universe Translation-en
       Get:5 http://archive.ubuntu.com trusty-security/main amd64 Packages [153 kB]
       Hit http://archive.ubuntu.com trusty-security/main Translation-en
       Get:6 http://archive.ubuntu.com trusty-updates/main amd64 Packages [356 kB]
       Hit http://archive.ubuntu.com trusty-updates/main Translation-en
       Fetched 635 kB in 2s (247 kB/s)
       Reading package lists...
-----> Fetching .debs for python-setuptools
       Reading package lists...
       Building dependency tree...
       The following extra packages will be installed:
         python-pkg-resources
       Suggested packages:
         python-distribute python-distribute-doc
       The following NEW packages will be installed:
         python-pkg-resources python-setuptools
       0 upgraded, 2 newly installed, 0 to remove and 38 not upgraded.
       Need to get 292 kB of archives.
       After this operation, 1017 kB of additional disk space will be used.
       Get:1 http://archive.ubuntu.com/ubuntu/ trusty/main python-pkg-resources all 3.3-1ubuntu1 [61.9 kB]
       Get:2 http://archive.ubuntu.com/ubuntu/ trusty/main python-setuptools all 3.3-1ubuntu1 [230 kB]
       Fetched 292 kB in 0s (408 kB/s)
       Download complete and in download only mode
-----> Fetching .debs for libxmlsec1
       Reading package lists...
       Building dependency tree...
       The following NEW packages will be installed:
         libxmlsec1
       0 upgraded, 1 newly installed, 0 to remove and 38 not upgraded.
       Need to get 0 B/115 kB of archives.
       After this operation, 436 kB of additional disk space will be used.
       Download complete and in download only mode
-----> Fetching .debs for libxmlsec1-dev
       Reading package lists...
       Building dependency tree...
       The following extra packages will be installed:
         libnspr4 libnspr4-dev libnss3 libnss3-dev libnss3-nssdb libxmlsec1
         libxmlsec1-gcrypt libxmlsec1-gnutls libxmlsec1-nss libxmlsec1-openssl
       The following NEW packages will be installed:
         libnspr4 libnspr4-dev libnss3 libnss3-dev libnss3-nssdb libxmlsec1
         libxmlsec1-dev libxmlsec1-gcrypt libxmlsec1-gnutls libxmlsec1-nss
         libxmlsec1-openssl
       0 upgraded, 11 newly installed, 0 to remove and 38 not upgraded.
       Need to get 0 B/2579 kB of archives.
       After this operation, 14.1 MB of additional disk space will be used.
       Download complete and in download only mode
-----> Fetching .debs for swig
       Reading package lists...
       Building dependency tree...
       The following extra packages will be installed:
         swig2.0
       Suggested packages:
         swig-doc swig-examples swig2.0-examples swig2.0-doc
       The following NEW packages will be installed:
         swig swig2.0
       0 upgraded, 2 newly installed, 0 to remove and 38 not upgraded.
       Need to get 0 B/881 kB of archives.
       After this operation, 4412 kB of additional disk space will be used.
       Download complete and in download only mode
-----> Fetching .debs for python-m2crypto
       Reading package lists...
       Building dependency tree...
       The following NEW packages will be installed:
         python-m2crypto
       0 upgraded, 1 newly installed, 0 to remove and 38 not upgraded.
       Need to get 0 B/156 kB of archives.
       After this operation, 831 kB of additional disk space will be used.
       Download complete and in download only mode
-----> Installing libnspr4-dev_2%3a4.10.7-0ubuntu0.14.04.1_amd64.deb
-----> Installing libnspr4_2%3a4.10.7-0ubuntu0.14.04.1_amd64.deb
-----> Installing libnss3-dev_2%3a3.17.1-0ubuntu0.14.04.1_amd64.deb
-----> Installing libnss3-nssdb_2%3a3.17.1-0ubuntu0.14.04.1_all.deb
-----> Installing libnss3_2%3a3.17.1-0ubuntu0.14.04.1_amd64.deb
-----> Installing libxmlsec1-dev_1.2.18-2ubuntu1_amd64.deb
-----> Installing libxmlsec1-gcrypt_1.2.18-2ubuntu1_amd64.deb
-----> Installing libxmlsec1-gnutls_1.2.18-2ubuntu1_amd64.deb
-----> Installing libxmlsec1-nss_1.2.18-2ubuntu1_amd64.deb
-----> Installing libxmlsec1-openssl_1.2.18-2ubuntu1_amd64.deb
-----> Installing libxmlsec1_1.2.18-2ubuntu1_amd64.deb
-----> Installing python-m2crypto_0.21.1-3ubuntu5_amd64.deb
-----> Installing python-pkg-resources_3.3-1ubuntu1_all.deb
-----> Installing python-setuptools_3.3-1ubuntu1_all.deb
-----> Installing swig2.0_2.0.11-1ubuntu2_amd64.deb
-----> Installing swig_2.0.11-1ubuntu2_amd64.deb
-----> Writing profile script
=====> Downloading Buildpack: https://github.com/heroku/heroku-buildpack-python
=====> Detected Framework: Python
-----> Installing dependencies with pip
       Downloading/unpacking dm.xmlsec.binding==1.3.1 (from -r requirements.txt (line 1))
         Running setup.py (path:/tmp/pip_build_u45022/dm.xmlsec.binding/setup.py) egg_info for package dm.xmlsec.binding
           Error: cannot get XMLSec1 pre-processor and compiler flags; do you have the `libxmlsec1` development package installed?
           Complete output from command python setup.py egg_info:
           Error: cannot get XMLSec1 pre-processor and compiler flags; do you have the `libxmlsec1` development package installed?

       ----------------------------------------
       Cleaning up...
       Command python setup.py egg_info failed with error code 1 in /tmp/pip_build_u45022/dm.xmlsec.binding
       Storing debug log for failure in /app/.pip/pip.log

 !     Push rejected, failed to compile Multipack app

To git@heroku.com:xxxxxxxxxxx.git
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'git@heroku.xxxxxxxxxxx.git'

【问题讨论】：

你有没有运气解决这个 Josvic？
@jb 不。我也没有机会尝试以下答案中提出的解决方案，因为我已经离开了我的问题中描述的问题的项目。
我现在也在解决这个问题，似乎问题是 apt buildpack 通过 dpkg 在/app/.apt 中安装包，但这样做，没有配置告诉包他们在 /app/.apt 而不是 / 。使用 libxmlsec1 的包使用 xmlsec1-config --cflags ，在这种情况下打印出 /usr/include 而不是 /app/.apt/usr/include

标签： python python-2.7 heroku apt

【解决方案1】：

每当 python 中的任何包都存在操作系统依赖项时，我更喜欢在 Heroku 上构建 Docker 映像以托管应用程序。

在您的根目录中创建一个 Docker 文件。一个 Docker 文件如下所示：

FROM gcr.io/google-appengine/python
RUN apt-get update
RUN apt install libxmlsec1
RUN pip install requirements.txt
CMD python main.py

在 docker 文件中写入您想要的所有 apt 命令，并使用名称 Dockerfile 保存它。不要给文件名添加任何扩展名。

创建一个名为 heroku.yml 的文件。将以下代码行粘贴到其中：

build:
  docker:
    web: Dockerfile

使用这些命令将您的应用程序推送到 Heroku：

1. git add heroku.yml

2 git commit -m "添加heroku.yml"

3 heroku stack:set container

4 git push heroku master

【讨论】：

【解决方案2】：

不管怎样，我可以通过以下两件事让 xmlsec1 在 Heroku 上工作：

使用 buildpack 安装 xmlsec1。
使用.profile.d script 将$PATH 配置为指向xmlsec1 二进制文件。

使用 buildpack 安装 xmlsec1

我使用 heroku-buildpack-multi 和 strydercorp/heroku-buildpack-xmlsec buildpack 将 xmlsec1 安装到我的 Heroku 实例中。请注意，此 buildpack 适用于 Cedar14，如果您使用的是标准 Cedar，则需要使用 evenco/heroku-buildpack-xmlsec

这是我的.buildpack 文件的样子：

$ cat .buildpacks 
https://github.com/heroku/heroku-buildpack-python
https://github.com/strydercorp/heroku-buildpack-xmlsec

使用 .profile.d 脚本配置 $PATH

在我的 Heroku 实例上安装 xmlsec1 二进制文件后，我需要配置 $PATH 变量以指向 xmlsec1 二进制文件。这是我曾经这样做的.profile.d 脚本：

$ cat .profile.d/path.sh 
# https://devcenter.heroku.com/articles/profiled

# add xmlsec1 binary and libraries to the appropriate paths
LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/app/.heroku/xmlsec1-1.2/lib/"
PATH="$PATH:/app/.heroku/xmlsec1-1.2/bin/"

【讨论】：

【解决方案3】：

我不知道，但也许答案是this：

在@PadraicCunningham 的惊人帮助下，我设法找到了解决方案。问题原来是 heroku-buildpack-apt 将东西安装在新创建的文件夹 /app/.apt/ 中，而该文件夹不在 PYTHONPATH 中。

所以我在 heroku 上的 PYTHONPATH 中添加了相关文件夹，如下所示：

heroku config:add PYTHONPATH=/app/.apt/usr/lib/python2.7/dist-packages/

【讨论】：

没用。 PYTHONPATH 告诉 python 要查找包，但 pip 似乎没有意识到这一点。