同一命名空间中的包：无法在安装脚本中导入模块答案

【问题标题】：Packages in same namespace: can't import module in setup script同一命名空间中的包：无法在安装脚本中导入模块
【发布时间】：2016-09-30 08:57:54
【问题描述】：

我很好奇下面的情况。假设我有两个名为project_alpha 和project_bravo 的项目，它们都定义了一个顶级命名空间包mymeta。布局：

project_alpha/
   -> mymeta/
        -> __init__.py
        -> project_alpha/
             -> __init__.py
             -> version.py
   -> setup.py

project_bravo/
   -> mymeta/
        -> __init__.py
        -> project_bravo/
             -> __init__.py
             -> version.py
   -> setup.py

mymeta/__init__.pys 都只包含__import__('pkg_resources').declare_namespace(__name__) 行（根据namespace section in setuptools docs）。 version.pys 的内容：

__version_info__ = (0, 9, 9, 'dev0')
__version__ = '.'.join((str(entry) for entry in __version_info__))

project_alpha 的 setup.py 脚本非常简单。声明了命名空间包mymeta，版本取自version模块：

# project_alpha
from setuptools import find_packages, setup
from mymeta.project_alpha.version import __version__

setup(
    name='mymeta.project-alpha',
    version=__version__,
    namespace_packages=['mymeta'],
    packages=find_packages(),
)

project_bravo 的 setup.py 脚本具有相同的结构，但有一点不同：project_bravo 在构建时依赖于 project_alpha：

from setuptools import find_packages, setup
from mymeta.project_bravo.version import __version__

setup(
    name='mymeta.project-bravo',
    version=__version__,
    namespace_packages=['mymeta'],
    setup_requires=['mymeta.project-alpha'],
    packages=find_packages(),
)

在构建project_bravo 时，出现以下错误：

~/project_bravo $ python setup.py sdist 
Traceback (most recent call last):
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 156, in save_modules
    yield saved
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 197, in setup_context
    yield
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 246, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 276, in run
    return func()
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 245, in runner
    _execfile(setup_script, ns)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 47, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-ahmxos98/mymeta.project-alpha-0.9.9.dev0/setup.py", line 6, in <module>
    try:
ImportError: No module named 'mymeta.project_alpha'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 22, in <module>
    packages=find_packages(),
  File "/usr/lib64/python3.5/distutils/core.py", line 108, in setup
    _setup_distribution = dist = klass(attrs)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/dist.py", line 315, in __init__
    self.fetch_build_eggs(attrs['setup_requires'])
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "/tmp/tstenv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 853, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "/tmp/tstenv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1125, in best_match
    return self.obtain(req, installer)
  File "/tmp/tstenv/lib/python3.5/site-packages/pkg_resources/__init__.py", line 1137, in obtain
    return installer(requirement)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/dist.py", line 429, in fetch_build_egg
    return cmd.easy_install(req)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 665, in easy_install
    return self.install_item(spec, dist.location, tmpdir, deps)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 695, in install_item
    dists = self.install_eggs(spec, download, tmpdir)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 876, in install_eggs
    return self.build_and_install(setup_script, setup_base)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1115, in build_and_install
    self.run_setup(setup_script, setup_base, args)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/command/easy_install.py", line 1101, in run_setup
    run_setup(setup_script, args)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 249, in run_setup
    raise
  File "/usr/lib64/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 197, in setup_context
    yield
  File "/usr/lib64/python3.5/contextlib.py", line 77, in __exit__
    self.gen.throw(type, value, traceback)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 168, in save_modules
    saved_exc.resume()
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 143, in resume
    six.reraise(type, exc, self._tb)
  File "/tmp/tstenv/lib/python3.5/site-packages/pkg_resources/_vendor/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 156, in save_modules
    yield saved
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 197, in setup_context
    yield
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 246, in run_setup
    DirectorySandbox(setup_dir).run(runner)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 276, in run
    return func()
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 245, in runner
    _execfile(setup_script, ns)
  File "/tmp/tstenv/lib/python3.5/site-packages/setuptools/sandbox.py", line 47, in _execfile
    exec(code, globals, locals)
  File "/tmp/easy_install-ahmxos98/mymeta.project-alpha-0.9.9.dev0/setup.py", line 6, in <module>
    try:
ImportError: No module named 'mymeta.project_alpha'

很遗憾，我不明白这里的错误。跟进口订单有关系吧？如果我在project_bravo/setup.py 中注释掉mymeta.project_bravo.version 的导入并将version 替换为一些硬编码的字符串，那么构建会突然成功...

编辑：我为这个问题介绍了一个解决方法。我没有尝试直接导入版本，而是exec 版本模块以避免导入问题。当然，这不是一个正确的解决方案，因此没有将其发布为答案，但仍然在这里：

__version__ = None # if the exec fails, leave the version unset, the resulting build version will be 0.0.0
version_script_path = os.path.relpath(os.path.join(os.path.dirname(__file__), 'mymeta', 'project_alpha', 'version.py'))
with open(version_script_path) as version_script:
    exec(version_script.read())

读取并执行版本脚本后，版本被初始化，因此无需从mymeta包中导入任何内容。

【问题讨论】：

我试图重现错误但我不能。在我的系统上一切正常。也许尝试使用新的 virtualenv？在尝试构建mymeta.project_bravo 之前，如何提供mymeta.project_alpha？你在mymeta.project_alpha 中做setup.py install 吗？
啊，我忘了说。我发出python setup.py sdist upload 将project_alpha 上传到私有PyPI 存储库。
嗯，发出python setup.py install 也适用于我，但这会将project_alpha 明确安装到venv，因此在构建project_bravo 时该包已经存在。仅当setuptools 尝试将project_alphas egg 下载到project_bravo/.eggs 目录时才会出现此问题

标签： python setuptools namespace-package

【解决方案1】：

差不多一年后，我再次遇到这个问题，python>=3.3 的解决方案是使用 PEP 420 中指定的隐式命名空间包。项目结构几乎没有改变，只是 pkgutil-style 命名空间包 mymeta 的 __init__.pys 都没有了：

project_alpha/
   -> mymeta/
        -> project_alpha/
             -> __init__.py
             -> version.py
   -> setup.py

project_bravo/
   -> mymeta/
        -> project_bravo/
             -> __init__.py
             -> version.py
   -> setup.py

为了让setuptools 开心，两个设置脚本也需要调整：

...
setup(
    ...
    packages=['mymeta.' + pkg for pkg in find_packages('mymeta')],
)

现在在构建 project-bravo 时导入将被正确解析。

【讨论】：

【解决方案2】：

如 cmets 中所述：命令

$ python setup.py sdist

对于 mymeta.project_bravo 从私人 pypi 存储库下载 mymeta.project_alpha 的鸡蛋。

命名空间包的导入非常微妙。

例如我发现当我在我的环境中定期安装ns.a 和ns.b 并在site-packages 目录的.pth 文件中提供ns.d 的导入路径时，无论如何它都不起作用。但是，当我在 site-packeges/ns 目录中符号链接到 ns.d 时，它将起作用。

如果我不安装任何组件并在 site-packages 目录中的 .pth 中提供所有路径，则一切正常。

如果我定期安装所有组件，一切正常。

当我混合概念时它不会起作用。

因此我怀疑这也可能是这里的问题：导入路径的不同策略。

您可能想尝试将您的__init__.py 文件修改为：

from pkgutil import extend_path

__path__ = extend_path(__path__, __name__)
__import__('pkg_resources').declare_namespace(__name__)

我很遗憾我决定做命名空间包。我再也不会这样做了。

【讨论】：

不幸的是，这没有帮助。不过，感谢您的努力，因为答案解释了错误的根源，这为我进一步挖掘提供了方向！
嗨，如果您仍然对解决方案感兴趣，我找到了一个至少适用于 python 3.3 和更新版本的解决方案 - 隐式命名空间包。详情见我的回答。