在 WSL2 Ubuntu 20.04 Windows 10 上导入 clip_retrieval 时无法从 requests.adapters 导入 HTTPAdapter答案

【问题标题】：Cannot import HTTPAdapter from requests.adapters when importing clip_retrieval on WSL2 Ubuntu 20.04 Windows 10在 WSL2 Ubuntu 20.04 Windows 10 上导入 clip_retrieval 时无法从 requests.adapters 导入 HTTPAdapter
【发布时间】：2022-11-11 06:27:37
【问题描述】：

据我检查所有依赖项都已安装并且在范围内（见下文），我还尝试了对可疑有问题的库（烧瓶、请求）进行卸载安装。到目前为止，我没有使用 venv，但如果这可能是某些库的问题，特别是烧瓶在导入过程中会中断），我不知道这个特定的 Py 版本是否可能是一个问题。

在我第一次尝试的 Windows 上，WSL2 上的错误没有发生，手动导入烧瓶和 HTTPAdapter 运行，即 from flask import Flask 和 from requests.adapters import HTTPAdapter 返回正常。

在 WSL 中，>>> import requests 返回，但 from requests.adapters import HTTPAdapter 是错误的，就像在 clip_retrieval 错误中一样。

（在 Win 上，最后的障碍是（据我所知）BLAS 和 OpenMP 缺少二进制文件/正确安装/连接/构建（from . import _swigfaiss ... ）。我尝试修复它一段时间，甚至手动复制 .dll（libblas.dll）。 dll, libiomp5md.dll, liblapack.dll")，但是错误并没有改变，然后我放弃并尝试了WSL。

[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import clip_retrieval
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/__init__.py", line 3, in <module>
    from .clip_back import clip_back
  File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/clip_back.py", line 5, in <module>
    from flask import Flask, request, make_response
ImportError: cannot import name 'Flask' from 'flask' (/home/tosh/.local/lib/python3.8/site-packages/flask/__init__.py)
Error in sys.excepthook:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 72, in apport_excepthook
    from apport.fileutils import likely_packaged, get_recent_crashes
  File "/usr/lib/python3/dist-packages/apport/__init__.py", line 5, in <module>
    from apport.report import Report
  File "/usr/lib/python3/dist-packages/apport/report.py", line 32, in <module>
    import apport.fileutils
  File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 12, in <module>
    import os, glob, subprocess, os.path, time, pwd, sys, requests_unixsocket
  File "/usr/lib/python3/dist-packages/requests_unixsocket/__init__.py", line 4, in <module>
    from .adapters import UnixAdapter
  File "/usr/lib/python3/dist-packages/requests_unixsocket/adapters.py", line 3, in <module>
    from requests.adapters import HTTPAdapter
ImportError: cannot import name 'HTTPAdapter' from 'requests.adapters' (/home/tosh/.local/lib/python3.8/site-packages/requests/adapters.py)

Original exception was:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/__init__.py", line 3, in <module>
    from .clip_back import clip_back
  File "/usr/local/lib/python3.8/dist-packages/clip_retrieval/clip_back.py", line 5, in <module>
    from flask import Flask, request, make_response
ImportError: cannot import name 'Flask' from 'flask' (/home/tosh/.local/lib/python3.8/site-packages/flask/__init__.py)
>>>

 pip3 list
Package                  Version
------------------------ --------------------
aiohttp                  3.8.3
aiosignal                1.2.0
albumentations           1.3.0
aniso8601                9.0.1
async-timeout            4.0.2
attrs                    19.3.0
autofaiss                2.15.3
Automat                  0.8.0
blinker                  1.4
braceexpand              0.1.7
certifi                  2019.11.28
chardet                  3.0.4
charset-normalizer       2.1.1
click                    8.1.3
clip-anytorch            2.5.0
clip-retrieval           2.35.1
cloud-init               22.3.4
colorama                 0.4.3
command-not-found        0.3
configobj                5.0.6
constantly               15.1.0
cryptography             2.8
cupshelpers              1.0
cycler                   0.11.0
dataclasses              0.6
dbus-python              1.2.16
defer                    1.0.6
Distance                 0.1.3
distro                   1.4.0
distro-info              0.23ubuntu1
docker-pycreds           0.4.0
embedding-reader         1.5.0
entrypoints              0.3
ExifRead-nocycle         3.0.1
faiss-cpu                1.7.2
filelock                 3.8.0
fire                     0.4.0
Flask                    2.2.2
Flask-Cors               3.0.10
Flask-RESTful            0.3.9
fonttools                4.28.5
frozenlist               1.3.1
fsspec                   2022.1.0
ftfy                     6.1.1
gitdb                    4.0.9
GitPython                3.1.29
h5py                     3.7.0
httplib2                 0.14.0
huggingface-hub          0.10.1
hyperlink                19.0.0
idna                     2.8
imageio                  2.22.4
img2dataset              1.33.0
importlib-metadata       5.0.0
incremental              16.10.1
itsdangerous             2.1.2
Jinja2                   3.1.2
joblib                   1.2.0
jsonpatch                1.22
jsonpointer              2.0
jsonschema               3.2.0
keyring                  18.0.1
kiwisolver               1.3.2
language-selector        0.1
launchpadlib             1.10.13
lazr.restfulclient       0.14.2
lazr.uri                 1.0.3
macaroonbakery           1.3.1
MarkupSafe               2.1.1
matplotlib               3.5.1
more-itertools           4.2.0
multidict                6.0.2
multilingual-clip        1.0.10
netifaces                0.10.4
networkx                 2.8.8
nltk                     3.7
numpy                    1.22.0
nvidia-cublas-cu11       11.10.3.66
nvidia-cuda-nvrtc-cu11   11.7.99
nvidia-cuda-runtime-cu11 11.7.99
nvidia-cudnn-cu11        8.5.0.96
oauthlib                 3.1.0
onnx                     1.11.0
open-clip-torch          2.3.1
opencv-python            4.6.0.66
opencv-python-headless   4.6.0.66
packaging                21.3
pandas                   1.5.1
pathtools                0.1.2
pexpect                  4.6.0
Pillow                   9.0.0
pip                      20.0.2
prometheus-client        0.15.0
promise                  2.3
protobuf                 3.19.4
psutil                   5.5.1
pyarrow                  7.0.0
pyasn1                   0.4.2
pyasn1-modules           0.2.1
pybind11                 2.9.1
pycairo                  1.16.2
pycups                   1.9.73
PyGObject                3.36.0
PyHamcrest               1.9.0
PyJWT                    1.7.1
pymacaroons              0.13.0
PyNaCl                   1.3.0
pyOpenSSL                19.0.0
pyparsing                3.0.6
pyRFC3339                1.1
pyrsistent               0.15.5
pyserial                 3.4
python-apt               2.0.0+ubuntu0.20.4.8
python-dateutil          2.8.2
python-debian            0.1.36ubuntu1
pytz                     2022.6
PyWavelets               1.4.1
PyYAML                   5.3.1
qudida                   0.0.4
regex                    2022.10.31
requests                 2.28.1
requests-unixsocket      0.2.0
scikit-image             0.19.3
scikit-learn             1.1.3
scipy                    1.9.3
SecretStorage            2.3.1
sentence-transformers    2.2.2
sentencepiece            0.1.97
sentry-sdk               1.10.1
service-identity         18.1.0
setproctitle             1.3.2
setuptools               60.9.3
shortuuid                1.0.9
simplejson               3.16.0
six                      1.14.0
smmap                    5.0.0
sos                      4.4
ssh-import-id            5.10
systemd-python           234
termcolor                2.1.0
threadpoolctl            3.1.0
tifffile                 2022.10.10
tokenizers               0.13.2
torch                    1.13.0
torchvision              0.14.0
tqdm                     4.64.1
transformers             4.24.0
Twisted                  18.9.0
typing-extensions        4.1.1
ubuntu-advantage-tools   27.11.3
ufw                      0.36
unattended-upgrades      0.1
urllib3                  1.26.11
wadllib                  1.3.3
wandb                    0.12.21
wcwidth                  0.2.5
webdataset               0.1.103
Werkzeug                 2.2.2
wheel                    0.34.2
yarl                     1.8.1
zipp                     1.0.0
zope.interface           4.7.1

https://github.com/rom1504/clip-retrieval/blob/main/requirements.txt

img2dataset>=1.25.5,<2
clip-anytorch>=2.5.0,<3
tqdm>=4.62.3,<5
fire>=0.4.0,<0.5.0
torch>=1.7.1,<2
torchvision>=0.10.1,<2
numpy>=1.19.5,<2
faiss-cpu>=1.7.2,<2
flask>=2.0.3,<3
flask_restful>=0.3.9,<1
flask_cors>=3.0.10,<4
pandas>=1.1.5,<2
pyarrow>=6.0.1,<8
autofaiss>=2.9.6,<3
webdataset>=0.1.103,<0.2
h5py>=3.1.0,<4
prometheus-client>=0.13.1,<1
fsspec==2022.1.0
sentence-transformers>=2.2.0,<3
wandb>=0.12.10,<0.13
open-clip-torch>=2.0.0,<3.0.0
requests>=2.27.1,<3
aiohttp>=3.8.1,<4
multilingual-clip>=1.0.10,<2

编辑：这是通过使用 Anaconda 安装在 Windows 上修复的（感谢 rom1504 建议它应该使用 venv 运行 - 我试图避免使用一个）。然而，另一个错误有待解决。见：https://github.com/rom1504/clip-retrieval/issues/204

【问题讨论】：

标签： python flask python-requests ubuntu-20.04 wsl-2

【解决方案1】：

我设法修复它并在本地运行整个剪辑检索记事本示例并从 Internet 访问本地隧道 Node.js 服务器。这是必须从 WSL 或 Windows 运行的代码：

https://colab.research.google.com/github/rom1504/clip-retrieval/blob/master/notebook/clip-retrieval-getting-started.ipynb

导入现已完成。

>>> import clip_retrieval
>>> dir(clip_retrieval)
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'clip_back', 'clip_end2end', 'clip_filter', 'clip_front', 'clip_index', 'clip_inference', 'ivf_metadata_ordering', 'load_clip']

解决方案：我搬到了 Python 3.10（也许不是严格需要，IDK）并升级了 pip。然后在尝试运行clip-retrieval命令时出现另一个错误，在导入_pustil时，“部分导入”，恰好是安装时发生冲突，可能混合sudo apt-get install和pip3 install，python3.10 -m pip install，因为错误跟踪包括对两个不同目录的引用。

  File "/usr/lib/python3/dist-packages/psutil/__init__.py", line 95, in <module>
    from . import _pslinux as _psplatform
  File "/usr/lib/python3/dist-packages/psutil/_pslinux.py", line 26, in <module>
    from . import _psutil_linux as cext
ImportError: cannot import name '_psutil_linux' from partially initialized module 'psutil' (most likely due to a circular import) (/usr/lib/python3/dist-packages/psutil/__init__.py)

删除和重新安装冲突的库时出现问题，但手动删除 sudo rm -r ... 包含模块 psutil 的文件夹，然后用 pip 重新安装它终于解决了。（有两个，增加了混乱，一个带有 .egg-info sudo rm -r /usr/lib/python3/dist-packages/psutil-5.5.1 sudo rm -r /usr/lib/python3/dist-packages/psutil-5.5.1.egg-info

img2dataset --url_list=myimglist.txt --output_folder=image_folder --thread_count=4 --image_
size=256
Starting the downloading of this file
Sharding file number 1 of 1 called /mnt/z/myimglist.txt
0it [00:00, ?it/s]File sharded in 0 shards
Downloading starting now, check your bandwidth speed (with bwm-ng)your cpu (with htop), and your disk usage (with iotop)!
0it [00:00, ?it/s]

myimglist.txt

https://placekitten.com/200/305
https://placekitten.com/200/304
https://placekitten.com/200/303
https://placekitten.com/200/302
https://placekitten.com/150/235
https://placekitten.com/120/120

调用剪辑检索时，它会下载

 clip-retrieval inference  --input_dataset image_folder --output_folder embedding_folder
The number of samples has been estimated to be 6
Starting the worker
dataset is 12
Starting work on task 0
100%|███████████████████████████████████████| 354M/354M [01:35<00:00, 3.72MiB/s]
/home/xxxx/.local/lib/python3.10/site-packages/clip/clip.py:160: FutureWarning: 'torch.onnx._patch_torch._node_getitem' is deprecated in version 1.13 and will be removed in version 1.14. Please Internally use '_node_get' in symbolic_helper instead..
  if "value" in node.attributeNames() and str(node["value"]).startswith("cuda"):
/home/xxxx/.local/lib/python3.10/site-packages/clip/clip.py:186: FutureWarning: 'torch.onnx._patch_torch._node_getitem' is deprecated in version 1.13 and will be removed in version 1.14. Please Internally use '_node_get' in symbolic_helper instead..
  if inputs[i].node()["value"] == 5:
warming up with batch size 256 on cpu
done warming up in 56.720762968063354s
/home/tosh/.local/lib/python3.10/site-packages/torch/utils/data/dataloader.py:554: UserWarning: This DataLoader will create 8 worker processes in total. Our suggested max number of worker in current system is 4, which is smaller than what this DataLoader is going to create. Please be aware that excessive worker creation might get DataLoader running slow or even freeze, lower the worker number to avoid potential slowness/freeze if necessary.
  warnings.warn(_create_warning_msg(
 sample_per_sec 1 ; sample_count 6

...

视窗

如果该信息对任何人有用，我意识到我正在尝试在 CLI 中导入 clip_retrieval（在 WSL 上我最初也是这样做的），而示例笔记本中的示例用法是将其作为命令调用。 img2dataset 成功：

<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
Starting the downloading of this file
Sharding file number 1 of 1 called z:/list.txt
0it [00:00, ?it/s]File sharded in 1 shards
Downloading starting now, check your bandwidth speed (with bwm-ng)your cpu (with htop), and your disk usage (with iotop)!
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
1it [00:07,  7.66s/it]
worker  - success: 1.000 - failed to download: 0.000 - failed to resize: 0.000 - images per sec: 8 - count: 18
total   - success: 1.000 - failed to download: 0.000 - failed to resize: 0.000 - images per sec: 8 - count: 18

正确调用剪辑检索现在返回另一个错误：JAX 要求，它似乎必须从源代码构建。所以可能它可能只是 JAX，而不是 OpenMP 和 BLAS，IDK。

clip-retrieval
<frozen importlib._bootstrap>:228: RuntimeWarning: scipy._lib.messagestream.MessageStream size changed, may indicate binary incompatibility. Expected 56 from C header, got 64 from PyObject
Traceback (most recent call last):
  File "C:Users	oshbAppDataLocalProgramsPythonPython39libsite-packagesjax_srclib__init__.py", line 38, in <module>
    import jaxlib as jaxlib
ModuleNotFoundError: No module named 'jaxlib'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:Users	oshbAppDataLocalProgramsPythonPython39lib
unpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:Users	oshbAppDataLocalProgramsPythonPython39lib
unpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:Users	oshbAppDataLocalProgramsPythonPython39Scriptsclip-retrieval.exe__main__.py", line 4, in <module>
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagesclip_retrieval__init__.py", line 6, in <module>
    from .clip_inference.main import main as clip_inference
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagesclip_retrievalclip_inference__init__.py", line 3, in <module>
    from .main import main as clip_inference
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagesclip_retrievalclip_inferencemain.py", line 10, in <module>
    from clip_retrieval.clip_inference.distributor import PysparkDistributor, SequentialDistributor
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagesclip_retrievalclip_inferencedistributor.py", line 5, in <module>
    from .worker import worker
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagesclip_retrievalclip_inferenceworker.py", line 15, in <module>
    from clip_retrieval.clip_inference.mapper import ClipMapper
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagesclip_retrievalclip_inferencemapper.py", line 5, in <module>
    from sentence_transformers import SentenceTransformer
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagessentence_transformers__init__.py", line 3, in <module>
    from .datasets import SentencesDataset, ParallelSentencesDataset
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagessentence_transformersdatasets__init__.py", line 3, in <module>
    from .ParallelSentencesDataset import ParallelSentencesDataset
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagessentence_transformersdatasetsParallelSentencesDataset.py", line 4, in <module>
    from .. import SentenceTransformer
  File "C:Users	oshbAppDataRoamingPythonPython39site-packagessentence_transformersSentenceTransformer.py", line 11, in <module>
    import transformers
  File "C:Users	oshbAppDataRoamingPythonPython39site-packages	ransformers__init__.py", line 30, in <module>
    from . import dependency_versions_check
  File "C:Users	oshbAppDataRoamingPythonPython39site-packages	ransformersdependency_versions_check.py", line 17, in <module>
    from .utils.versions import require_version, require_version_core
  File "C:Users	oshbAppDataRoamingPythonPython39site-packages	ransformersutils__init__.py", line 34, in <module>
    from .generic import (
  File "C:Users	oshbAppDataRoamingPythonPython39site-packages	ransformersutilsgeneric.py", line 36, in <module>
    import jax.numpy as jnp
  File "C:Users	oshbAppDataLocalProgramsPythonPython39libsite-packagesjax__init__.py", line 35, in <module>
    from jax import config as _config_module
  File "C:Users	oshbAppDataLocalProgramsPythonPython39libsite-packagesjaxconfig.py", line 17, in <module>
    from jax._src.config import config
  File "C:Users	oshbAppDataLocalProgramsPythonPython39libsite-packagesjax_srcconfig.py", line 29, in <module>
    from jax._src import lib
  File "C:Users	oshbAppDataLocalProgramsPythonPython39libsite-packagesjax_srclib__init__.py", line 40, in <module>
    raise ModuleNotFoundError(
ModuleNotFoundError: jax requires jaxlib to be installed. See https://github.com/google/jax#installation for installation instructions.

【讨论】：