mognoengine 和 bson 包不能一起工作答案

【问题标题】：mognoengine and bson package not work togethermognoengine 和 bson 包不能一起工作
【发布时间】：2020-09-27 22:36:57
【问题描述】：

我有一个依赖于mongoengine 的项目，我正在使用一个使用bson 库的库。

它的结构如下。

bsNotify/
├── setup.cfg
├── setup.py
└── src
    └── bsnotify
        ├── __init__.py
        └── resources.py

setup.py

$ cat bsNotify/setup.py
"""Base module setup."""
from setuptools import setup

setup(
    setup_requires=['pbr'],
    pbr=True
)

setup.cfg

$ cat bsNotify/setup.cfg
[metadata]
name = bsNotify
classifiers =
    License :: N/A :: N/A
    Programming Language :: Python :: 3.7

[options]
zip_safe = False
include_package_data = True
python_requires = >= 3.7
install_requires =
    mongoengine
    bson
package_dir=
    =src
packages=find:

[options.packages.find]
where=src

[tool:wheel]
universal = 1

[flake8]
exclude =
    venv,
    .tox,
    .git,
    __pycache__,
    *.pyc,
    *.egg-info,
    .cache,
    .eggs,
max-line-length = 80

[tox]
envlist = py37,unittest,lint

[testenv]
basepython=python3.7
deps =
    ipython
    pylint
    pytest
    pytest-cov
    pytest-xdist
    flake8
    flake8-docstrings

[testenv:unittest]
commands=
    pytest -v -s -n auto -l --cov=bsnotify --cov-report term-missing --cov-report xml --no-cov-on-fail tests/unit

[testenv:lint]
commands=
    flake8 src/bsnotify
    pylint src/bsnotify

src/bsnotify/__init__.py

$ cat bsNotify/src/bsnotify/__init__.py

src/bsnotify/resources.py

$ cat bsNotify/src/bsnotify/resources.py

src 中的文件为空。

当我运行 tox 命令时，它会创建一个用于测试的 virtualenv。

$ tox -v -e unittest --notest
using tox.ini: /Users/myuser/bsNotify/setup.cfg (pid 30954)
using tox-3.14.6 from /Users/myuser/.pyenv/versions/3.7.4/lib/python3.7/site-packages/tox/__init__.py (pid 30954)
GLOB sdist-make: /Users/myuser/bsNotify/setup.py
[30956] /Users/myuser/bsNotify$ /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 setup.py sdist --formats=zip --dist-dir /Users/myuser/bsNotify/.tox/dist >.tox/log/GLOB-0.log
package .tmp/package/1/bsNotify-0.1.2.dev2.zip links to dist/bsNotify-0.1.2.dev2.zip (/Users/myuser/bsNotify/.tox)
unittest cannot reuse: no previous config /Users/myuser/bsNotify/.tox/unittest/.tox-config1
unittest create: /Users/myuser/bsNotify/.tox/unittest
[30992] /Users/myuser/bsNotify/.tox$ /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 -m virtualenv --no-download --python /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7 unittest >unittest/log/unittest-0.log
unittest installdeps: ipython, pylint, pytest, pytest-cov, pytest-xdist, flake8, flake8-docstrings
[30993] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip install ipython pylint pytest pytest-cov pytest-xdist flake8 flake8-docstrings >.tox/unittest/log/unittest-1.log
unittest inst: /Users/myuser/bsNotify/.tox/.tmp/package/1/bsNotify-0.1.2.dev2.zip
write config to /Users/myuser/bsNotify/.tox/unittest/.tox-config1 as '7186e9f46c94b6d9f7dde810ce83f8fe46740d9d42f2863cfd063c4c6f4e4a88 /Users/myuser/.pyenv/versions/3.7.4/bin/python3.7\n3.14.6 0 0 0\n00000000000000000000000000000000 ipython\n00000000000000000000000000000000 pylint\n00000000000000000000000000000000 pytest\n00000000000000000000000000000000 pytest-cov\n00000000000000000000000000000000 pytest-xdist\n00000000000000000000000000000000 flake8\n00000000000000000000000000000000 flake8-docstrings'
[31000] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip install --exists-action w .tox/.tmp/package/1/bsNotify-0.1.2.dev2.zip >.tox/unittest/log/unittest-2.log
[31030] /Users/myuser/bsNotify$ /Users/myuser/bsNotify/.tox/unittest/bin/python -m pip freeze >.tox/unittest/log/unittest-3.log
unittest installed: apipkg==1.5,appnope==0.1.0,astroid==2.4.2,attrs==19.3.0,backcall==0.1.0,bsNotify==0.1.2.dev2,bson==0.5.10,coverage==5.1,decorator==4.4.2,execnet==1.7.1,flake8==3.8.3,flake8-docstrings==1.5.0,importlib-metadata==1.6.1,ipython==7.15.0,ipython-genutils==0.2.0,isort==4.3.21,jedi==0.17.0,lazy-object-proxy==1.4.3,mccabe==0.6.1,mongoengine==0.20.0,more-itertools==8.3.0,packaging==20.4,parso==0.7.0,pexpect==4.8.0,pickleshare==0.7.5,pluggy==0.13.1,prompt-toolkit==3.0.5,ptyprocess==0.6.0,py==1.8.1,pycodestyle==2.6.0,pydocstyle==5.0.2,pyflakes==2.2.0,Pygments==2.6.1,pylint==2.5.3,pymongo==3.10.1,pyparsing==2.4.7,pytest==5.4.3,pytest-cov==2.9.0,pytest-forked==1.1.3,pytest-xdist==1.32.0,python-dateutil==2.8.1,six==1.15.0,snowballstemmer==2.0.0,toml==0.10.1,traitlets==4.3.3,typed-ast==1.4.1,wcwidth==0.2.4,wrapt==1.12.1,zipp==3.1.0
_______________________________________________________ summary _______________________________________________________
  unittest: skipped tests
  congratulations :)

然后我尝试在新创建的virtualenv中访问mongoengine，它给出了错误。

$ .tox/unittest/bin/python -c "from mongoengine import connection"
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/mongoengine/__init__.py", line 2, in <module>
    from mongoengine import connection
  File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/mongoengine/connection.py", line 1, in <module>
    from pymongo import MongoClient, ReadPreference, uri_parser
  File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/pymongo/__init__.py", line 77, in <module>
    from pymongo.collection import ReturnDocument
  File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/pymongo/collection.py", line 20, in <module>
    from bson.code import Code
  File "/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/bson/code.py", line 18, in <module>
    from bson.py3compat import abc, string_type, PY3, text_type
ImportError: cannot import name 'abc' from 'bson.py3compat' (/Users/myuser/bsNotify/.tox/unittest/lib/python3.7/site-packages/bson/py3compat.py)

当我单独使用 mongoengine 和 bson 时，它运行良好。

$ python -m venv .venv
$ .venv/bin/pip install mongoengine
Collecting mongoengine
  Using cached https://files.pythonhosted.org/packages/7d/bd/9a7239b0032157f948c69febdf71dd82cb54fcd2499077300496a3f076c9/mongoengine-0.20.0-py3-none-any.whl
Collecting pymongo<4.0,>=3.4 (from mongoengine)
  Using cached https://files.pythonhosted.org/packages/23/cd/27fbc08f0bd835b4735504a758756e979b42c5bc9ebaac5ed3c2cbffd83f/pymongo-3.10.1-cp37-cp37m-macosx_10_9_x86_64.whl
Installing collected packages: pymongo, mongoengine
Successfully installed mongoengine-0.20.0 pymongo-3.10.1
$ .venv/bin/python -c "from mongoengine import connection"
$ .venv/bin/pip install bson
Collecting bson
  Using cached https://files.pythonhosted.org/packages/4d/53/7c534a38850f2252275d7f949aed2219095e90df1e2d180a9c8ed139e499/bson-0.5.10.tar.gz
Collecting python-dateutil>=2.4.0 (from bson)
  Using cached https://files.pythonhosted.org/packages/d4/70/d60450c3dd48ef87586924207ae8907090de0b306af2bce5d134d78615cb/python_dateutil-2.8.1-py2.py3-none-any.whl
Collecting six>=1.9.0 (from bson)
  Using cached https://files.pythonhosted.org/packages/ee/ff/48bde5c0f013094d729fe4b0316ba2a24774b3ff1c52d924a8a4cb04078a/six-1.15.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, bson
  Running setup.py install for bson ... done
Successfully installed bson-0.5.10 python-dateutil-2.8.1 six-1.15.0
You are using pip version 19.0.3, however version 20.2b1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
$ .venv/bin/python -c "from mongoengine import connection"

它检查两个 virtualenv 之间的pip freeze。我发现的唯一区别是在.tox env 中安装了bsNotify==0.1.2.dev2，而.venv env 中不存在。

是什么让bson 更改文件py3compat.py？

【问题讨论】：

标签： python pymongo mongoengine bson

【解决方案1】：

改一下

install_requires =
    mongoengine
    bson

到这里

install_requires =
    mongoengine
    pymongo

或完全删除（mongoengine 已经需要 pymongo

install_requires =
    mongoengine

requirements=bson 和

是一样的东西

pip install bson

安装此第 3 方包，其中不包括 MongoDB 包中的所有好东西

https://pypi.org/project/bson/

pymongo（官方 MongoDB 驱动程序）包含一个 bson 包。我会将 pymongo 文件夹和 bson 文件夹放入您的站点包中。

bson（第三方 bson 实现）也希望将 bson 文件夹放入您的站点包中。

安装 pymongo 和安装 bson 会导致站点包中的文件夹发生冲突。 PyMongo 只知道如何使用它自己的实现，这就是为什么您看到 pymongo 正在寻找不属于 3rd 方 bson 包的 py3compat 的原因。

根据您的安装顺序，它将安装 mongo 引擎（和 pymongo 作为依赖项），然后安装 bson（覆盖 MongoDB 与 pymongo 打包的 bson 实现）。 Pymongo 调用 py3compat 期望它在那里，但 3td 方没有实现它（以及其他一些东西）。

pip install pymongo

https://pypi.org/project/pymongo/

如 pymongo pypi 页面上所述

不要从 pypi 安装“bson”包。 PyMongo 自带自己的 bson 包；做“easy_install bson”安装第三方与 PyMongo 不兼容的包。

【讨论】：

感谢您的回复，bson 不在我的控制范围内，我正在使用一个第三方库，它使用了 bson 包。我只是在这里展示它，问题是如何发生的。还有一件事，当我们在同一环境中安装mongonengine 和bson 时，为什么它不会引起问题？当我们在requirements 中提到时，如果我们使用pip install mongoengine bson 安装它只会给出错误，否则它工作正常。
我认为这真的只是平局的运气。如果首先安装 pymongo，并且应用程序（或 mongo 引擎或 pymongo）调用了 3rd 方 bson 未实现的函数，您将看到错误。此外，如果首先安装 bson，那么当安装 pymongo 时，bson 包会被覆盖，你会看到它也可以正常工作。您可以在工作实例中查看您的站点包文件夹并查看哪个版本（MongoDB 第一方驱动程序或第三方驱动程序）
是的，如果我将bson 更改为先安装，它工作正常，但试图找到原因，如果单独安装它们可以工作，但是在setup.cfg 中进行时，它不起作用。
这是 pip 和任何其他非策划包管理器的众所周知的问题。如果您能找到从 setup.cfg 中删除“bson”的方法，那么一切都会正常。