在 pip install 上运行 Makefile

Question

我有一些协议缓冲区定义需要构建到 Python 源代码中，作为pip install 进程的一部分。我在setup.py 中对setuptools.command.install 命令进行了子类化，但我认为它试图在安装包后运行Makefile，因此无法识别源。

我找不到有关 pip 安装过程中发生的情况的信息。任何人都可以解释一下吗？

setup.py：

import subprocess
import sys

from setuptools import setup
from setuptools.command.install import install

class Install(install):
    """Customized setuptools install command - builds protos on install."""
    def run(self):
        protoc_command = ["make", "python"]
        if subprocess.call(protoc_command) != 0:
            sys.exit(-1)
        install.run(self)


setup(
    name='myprotos',
    version='0.0.1',
    description='Protocol Buffers.',
    install_requires=[],
    cmdclass={
        'install': Install,
    }
)

$ pip install -vvv .的输出：

Processing /path/to/myprotos
  Running setup.py (path:/private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build/setup.py) egg_info for package from file:///path/to/myprotos
    Running command python setup.py egg_info
    running egg_info
    creating pip-egg-info/myprotos.egg-info
    writing pip-egg-info/myprotos.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/myprotos.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/myprotos.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/myprotos.egg-info/SOURCES.txt'
    reading manifest file 'pip-egg-info/myprotos.egg-info/SOURCES.txt'
    writing manifest file 'pip-egg-info/myprotos.egg-info/SOURCES.txt'
  Source in /private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build has version 0.0.1, which satisfies requirement myprotos==0.0.1 from file:///path/to/myprotos
Building wheels for collected packages: myprotos
  Running setup.py bdist_wheel for myprotos: started
  Destination directory: /var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/tmpD7dfGKpip-wheel-
  Running command /usr/local/opt/python/bin/python2.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/tmpD7dfGKpip-wheel- --python-tag cp27
  running bdist_wheel
  running build
  installing to build/bdist.macosx-10.12-x86_64/wheel
  running install

  # THIS IS MY MAKEFILE RUNNING
  Grabbing github.com/google/protobuf...
  Building Python protos...
  # MAKEFILE COMPLETE

  running install_egg_info
  running egg_info
  creating myprotos.egg-info
  writing myprotos.egg-info/PKG-INFO
  writing top-level names to myprotos.egg-info/top_level.txt
  writing dependency_links to myprotos.egg-info/dependency_links.txt
  writing manifest file 'myprotos.egg-info/SOURCES.txt'
  reading manifest file 'myprotos.egg-info/SOURCES.txt'
  writing manifest file 'myprotos.egg-info/SOURCES.txt'
  Copying myprotos.egg-info to build/bdist.macosx-10.12-x86_64/wheel/myprotos-0.0.1-py2.7.egg-info
  running install_scripts
  creating build/bdist.macosx-10.12-x86_64/wheel/myprotos-0.0.1.dist-info/WHEEL
  Running setup.py bdist_wheel for myprotos: finished with status 'done'
  Stored in directory: /Users/jds/Library/Caches/pip/wheels/92/0b/37/b5a50146994bc0b6774407139f01d648ba3a9b4853d2719c51
  Removing source in /private/var/folders/3t/4qwkfyr903d0b7db7by2kj6r0000gn/T/pip-jpgCby-build
Successfully built myprotos
Installing collected packages: myprotos
  Found existing installation: myprotos 0.0.1
    Uninstalling myprotos-0.0.1:
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/DESCRIPTION.rst
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/INSTALLER
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/METADATA
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/RECORD
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/WHEEL
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/metadata.json
      Removing file or directory /usr/local/lib/python2.7/site-packages/myprotos-0.0.1.dist-info/top_level.txt
      Successfully uninstalled myprotos-0.0.1

Successfully installed myprotos-0.0.1
Cleaning up...

我的 Makefile 是否应该在生成源文件的过程中尽早运行？例如，在egg_info 运行之前文件是否需要存在？

如果我手动运行 Makefile 然后安装包，那么它就可以工作。

---

更新

这是我的项目的结构：

myprotos
├── Makefile
├── README.md
├── document.proto
├── myprotos         # Generated by Makefile
│   ├── __init__.py  # Generated by Makefile
│   └── proto_pb2.py # Generated by Makefile
└── setup.py 

这是 Makefile 的一部分，它从 Potocol 缓冲区定义生成 Python 源代码：

python: protoc deps
    # the protoc and deps command above just downloads
    # the `protoc` binary to a local bin directory 
    @echo "Building Python protos..."
    @mkdir -p "${PYTHON_OUT}"
    @touch "${PYTHON_OUT}"/__init__.py
    @printf "__all__ = ['proto_pb2']" > "${PYTHON_OUT}"/__init__.py
    @PATH="${LOCAL_BINARY_PATH}:$$PATH" protoc \
        --proto_path="${BASE}" \
        --proto_path="${GOPATH}/src/github.com/google/protobuf/src" \
        --python_out="${PYTHON_OUT}/" \
        ${PROTOS}

Answer 1

好的，这里需要更改三件事：

1. 将Makefile 和document.proto 添加到新文件MANIFEST.in。

    Makefile
    document.proto
   

   如果您这样做，python setup.py sdist 创建的.zip 文件（也上传到 PyPI）将包含这些文件。

2. 您需要在python setup.py build 期间运行您的make 命令，而不是在install 期间。由于您正在生成 Python 代码，因此您需要在此处更改 build_py 命令：

    import sys
    import subprocess
   
    from setuptools import setup
    from setuptools.command.build_py import build_py
   
    class Build(build_py):
        """Customized setuptools build command - builds protos on build."""
        def run(self):
            protoc_command = ["make", "python"]
            if subprocess.call(protoc_command) != 0:
                sys.exit(-1)
            build_py.run(self)
   
   
    setup(
        name='buildtest',
        version='1.0',
        description='Python Distribution Utilities',
        packages=['buildtest'],
        cmdclass={
            'build_py': Build,
        }
    )
   

   如果您的Makefile 生成机器代码，即来自 C 或任何其他编译语言，您应该更改 build_ext 命令：

    import sys
    import subprocess
   
    from setuptools import setup
    from setuptools.command.build_ext import build_ext
   
    class Build(build_ext):
        """Customized setuptools build command - builds protos on build."""
        def run(self):
            protoc_command = ["make", "python"]
            if subprocess.call(protoc_command) != 0:
                sys.exit(-1)
            build_ext.run(self)
   
   
    setup(
        name='buildtest',
        version='1.0',
        description='Python Distribution Utilities',
        packages=['buildtest'],
        has_ext_modules=lambda: True,
        cmdclass={
            'build_ext': Build,
        }
    )
   

3. 最后，您需要通过在setup() 中定义属性packages 来告诉setuptools 在install 上安装生成的包：

    setup(
        ...
        packages=['myprotos']
    )
   

决定运行build_py或build_ext的原因在于两者运行时的情况：

• build_py 在创建必须是跨平台的源分发时也会运行。已编译的扩展通常不是跨平台的，因此您无法在此步骤中编译它们。
• build_ext 仅在您创建特定于平台的二进制发行版时运行。在这里编译成特定平台的机器码就可以了。