我想对此添加我自己的答案,作为对 hoefling 所描述内容的一种补充。
谢谢,hoefling,因为您的回答帮助我走上了正轨,以与我自己的存储库几乎相同的方式编写设置脚本。
序言
编写此答案的主要动机是尝试将缺失的部分“粘合在一起”。 OP 没有说明正在开发的 C/C++ Python 模块的性质;我想先明确一点,以下步骤适用于创建多个 .dll/ .so 文件以及预编译的 *.pyd/so 文件的 C/C++ cmake 构建链一些通用的.py 文件需要放在脚本目录中。
所有这些文件在运行cmake build 命令之后直接生成......很有趣。不建议以这种方式构建 setup.py。
因为 setup.py 意味着您的脚本将成为您的包/库的一部分,并且必须通过库部分声明需要构建的 .dll 文件,并列出源和包含目录,所以没有直观的方法可以告诉 setuptools 在build_ext 中发生的对cmake -b 的一次调用所产生的库、脚本和数据文件都应该放在各自的位置。更糟糕的是,如果您想让 setuptools 跟踪此模块并完全卸载,这意味着用户可以卸载它,并在需要时清除系统中的所有痕迹。
我编写 setup.py 的模块是 bpy,.pyd/.so 相当于将搅拌机构建为 Python 模块,如下所述:
https://wiki.blender.org/wiki//User:Ideasman42/BlenderAsPyModule(更好的指令,但现在是死链接)
http://www.gizmoplex.com/wordpress/compile-blender-as-python-module/(可能是更糟糕的说明,但似乎仍然在线)
您可以在这里查看我在 github 上的存储库:
https://github.com/TylerGubala/blenderpy
这就是我写这个答案的动机,希望能帮助其他人尝试完成类似的事情,而不是丢弃他们的 cmake 构建链,或者更糟糕的是,不得不维护两个独立的构建环境。如果离题,我深表歉意。
那么我该怎么做才能做到这一点?
用我自己的类扩展setuptools.Extension 类,它不包含源或库属性的条目
用我自己的类扩展setuptools.commands.build_ext.build_ext 类,该类有一个自定义方法来执行我必要的构建步骤(git、svn、cmake、cmake --build)
-
用我自己的类扩展distutils.command.install_data.install_data 类(糟糕,distutils... 但是似乎没有 setuputils 等效项),以在 setuptools 记录创建期间标记构建的二进制库(安装-files.txt) 这样
用我自己的类扩展 setuptools.command.install_lib.install_lib 类,这将确保将构建的库从其生成的构建文件夹移动到 setuptools 期望它们所在的文件夹中(在 Windows 上,它会将 @987654341 @bin/Release 文件夹中的文件,而不是 setuptools 期望的位置)
-
使用我自己的类扩展 setuptools.command.install_scripts.install_scripts 类,以便将脚本文件复制到正确的目录(Blender 期望 2.79 或任何目录位于脚本位置)
李>
执行构建步骤后,将这些文件复制到已知目录中,setuptools 将复制到我环境的站点包目录中。此时剩余的 setuptools 和 distutils 类可以接管写入 installed-files.txt 记录,并将完全删除!
示例
这是一个示例,或多或少来自我的存储库,但为了更具体的内容清晰而进行了修剪(您可以随时前往存储库并自己查看)
from distutils.command.install_data import install_data
from setuptools import find_packages, setup, Extension
from setuptools.command.build_ext import build_ext
from setuptools.command.install_lib import install_lib
from setuptools.command.install_scripts import install_scripts
import struct
BITS = struct.calcsize("P") * 8
PACKAGE_NAME = "example"
class CMakeExtension(Extension):
"""
An extension to run the cmake build
This simply overrides the base extension class so that setuptools
doesn't try to build your sources for you
"""
def __init__(self, name, sources=[]):
super().__init__(name = name, sources = sources)
class InstallCMakeLibsData(install_data):
"""
Just a wrapper to get the install data into the egg-info
Listing the installed files in the egg-info guarantees that
all of the package files will be uninstalled when the user
uninstalls your package through pip
"""
def run(self):
"""
Outfiles are the libraries that were built using cmake
"""
# There seems to be no other way to do this; I tried listing the
# libraries during the execution of the InstallCMakeLibs.run() but
# setuptools never tracked them, seems like setuptools wants to
# track the libraries through package data more than anything...
# help would be appriciated
self.outfiles = self.distribution.data_files
class InstallCMakeLibs(install_lib):
"""
Get the libraries from the parent distribution, use those as the outfiles
Skip building anything; everything is already built, forward libraries to
the installation step
"""
def run(self):
"""
Copy libraries from the bin directory and place them as appropriate
"""
self.announce("Moving library files", level=3)
# We have already built the libraries in the previous build_ext step
self.skip_build = True
bin_dir = self.distribution.bin_dir
# Depending on the files that are generated from your cmake
# build chain, you may need to change the below code, such that
# your files are moved to the appropriate location when the installation
# is run
libs = [os.path.join(bin_dir, _lib) for _lib in
os.listdir(bin_dir) if
os.path.isfile(os.path.join(bin_dir, _lib)) and
os.path.splitext(_lib)[1] in [".dll", ".so"]
and not (_lib.startswith("python") or _lib.startswith(PACKAGE_NAME))]
for lib in libs:
shutil.move(lib, os.path.join(self.build_dir,
os.path.basename(lib)))
# Mark the libs for installation, adding them to
# distribution.data_files seems to ensure that setuptools' record
# writer appends them to installed-files.txt in the package's egg-info
#
# Also tried adding the libraries to the distribution.libraries list,
# but that never seemed to add them to the installed-files.txt in the
# egg-info, and the online recommendation seems to be adding libraries
# into eager_resources in the call to setup(), which I think puts them
# in data_files anyways.
#
# What is the best way?
# These are the additional installation files that should be
# included in the package, but are resultant of the cmake build
# step; depending on the files that are generated from your cmake
# build chain, you may need to modify the below code
self.distribution.data_files = [os.path.join(self.install_dir,
os.path.basename(lib))
for lib in libs]
# Must be forced to run after adding the libs to data_files
self.distribution.run_command("install_data")
super().run()
class InstallCMakeScripts(install_scripts):
"""
Install the scripts in the build dir
"""
def run(self):
"""
Copy the required directory to the build directory and super().run()
"""
self.announce("Moving scripts files", level=3)
# Scripts were already built in a previous step
self.skip_build = True
bin_dir = self.distribution.bin_dir
scripts_dirs = [os.path.join(bin_dir, _dir) for _dir in
os.listdir(bin_dir) if
os.path.isdir(os.path.join(bin_dir, _dir))]
for scripts_dir in scripts_dirs:
shutil.move(scripts_dir,
os.path.join(self.build_dir,
os.path.basename(scripts_dir)))
# Mark the scripts for installation, adding them to
# distribution.scripts seems to ensure that the setuptools' record
# writer appends them to installed-files.txt in the package's egg-info
self.distribution.scripts = scripts_dirs
super().run()
class BuildCMakeExt(build_ext):
"""
Builds using cmake instead of the python setuptools implicit build
"""
def run(self):
"""
Perform build_cmake before doing the 'normal' stuff
"""
for extension in self.extensions:
if extension.name == 'example_extension':
self.build_cmake(extension)
super().run()
def build_cmake(self, extension: Extension):
"""
The steps required to build the extension
"""
self.announce("Preparing the build environment", level=3)
build_dir = pathlib.Path(self.build_temp)
extension_path = pathlib.Path(self.get_ext_fullpath(extension.name))
os.makedirs(build_dir, exist_ok=True)
os.makedirs(extension_path.parent.absolute(), exist_ok=True)
# Now that the necessary directories are created, build
self.announce("Configuring cmake project", level=3)
# Change your cmake arguments below as necessary
# Below is just an example set of arguments for building Blender as a Python module
self.spawn(['cmake', '-H'+SOURCE_DIR, '-B'+self.build_temp,
'-DWITH_PLAYER=OFF', '-DWITH_PYTHON_INSTALL=OFF',
'-DWITH_PYTHON_MODULE=ON',
f"-DCMAKE_GENERATOR_PLATFORM=x"
f"{'86' if BITS == 32 else '64'}"])
self.announce("Building binaries", level=3)
self.spawn(["cmake", "--build", self.build_temp, "--target", "INSTALL",
"--config", "Release"])
# Build finished, now copy the files into the copy directory
# The copy directory is the parent directory of the extension (.pyd)
self.announce("Moving built python module", level=3)
bin_dir = os.path.join(build_dir, 'bin', 'Release')
self.distribution.bin_dir = bin_dir
pyd_path = [os.path.join(bin_dir, _pyd) for _pyd in
os.listdir(bin_dir) if
os.path.isfile(os.path.join(bin_dir, _pyd)) and
os.path.splitext(_pyd)[0].startswith(PACKAGE_NAME) and
os.path.splitext(_pyd)[1] in [".pyd", ".so"]][0]
shutil.move(pyd_path, extension_path)
# After build_ext is run, the following commands will run:
#
# install_lib
# install_scripts
#
# These commands are subclassed above to avoid pitfalls that
# setuptools tries to impose when installing these, as it usually
# wants to build those libs and scripts as well or move them to a
# different place. See comments above for additional information
setup(name='my_package',
version='1.0.0a0',
packages=find_packages(),
ext_modules=[CMakeExtension(name="example_extension")],
description='An example cmake extension module',
long_description=open("./README.md", 'r').read(),
long_description_content_type="text/markdown",
keywords="test, cmake, extension",
classifiers=["Intended Audience :: Developers",
"License :: OSI Approved :: "
"GNU Lesser General Public License v3 (LGPLv3)",
"Natural Language :: English",
"Programming Language :: C",
"Programming Language :: C++",
"Programming Language :: Python",
"Programming Language :: Python :: 3.6",
"Programming Language :: Python :: Implementation :: CPython"],
license='GPL-3.0',
cmdclass={
'build_ext': BuildCMakeExt,
'install_data': InstallCMakeLibsData,
'install_lib': InstallCMakeLibs,
'install_scripts': InstallCMakeScripts
}
)
以这种方式编写setup.py 后,构建python 模块就像运行py setup.py 一样简单,它将运行构建并生成输出文件。
建议您为网速慢或不想从源代码构建的用户制作轮子。为此,您需要安装 wheel 包 (py -m pip install wheel) 并通过执行 py setup.py bdist_wheel 生成一个轮子分发,然后像任何其他包一样使用 twine 上传它。