【问题标题】:AttributeError: 'PyxImporter' object has no attribute 'find_spec' with Pandas/BigQueryAttributeError:“PyxImporter”对象在 Pandas/BigQuery 中没有属性“find_spec”
【发布时间】:2017-11-16 17:13:40
【问题描述】:

我们正在尝试使用 Pandas 从 BigQuery 中提取一些数据,并遇到关于导入的绝对庞大的跟踪。据我所知,所有正确的依赖项都已安装。

Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 879, in _find_spec
AttributeError: 'PyxImporter' object has no attribute 'find_spec'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "run_forecasts.py", line 117, in <module>
    run_forecasts()
  File "run_forecasts.py", line 50, in run_forecasts
    a.run()
  File "./budgetforecastmodel/prepare/funnel_entrance.py", line 37, in run
    self._project_id
  File "./budgetforecastmodel/prepare/funnel_entrance.py", line 60, in _read_data
    funnel_entrances = fe_query.run()
  File "./budgetforecastmodel/extract/bigquery/biq_query_base.py", line 41, in run
    self.read_data()
  File "./budgetforecastmodel/extract/bigquery/biq_query_base.py", line 141, in read_data
    self.df = pd.read_gbq(self.query, self.project_id)
  File "./venv/lib/python3.6/site-packages/pandas/io/gbq.py", line 100, in read_gbq
    **kwargs)
  File "./venv/lib/python3.6/site-packages/pandas_gbq/gbq.py", line 906, in read_gbq
    dialect=dialect, auth_local_webserver=auth_local_webserver)
  File "./venv/lib/python3.6/site-packages/pandas_gbq/gbq.py", line 202, in __init__
    self.credentials = self.get_credentials()
  File "./venv/lib/python3.6/site-packages/pandas_gbq/gbq.py", line 214, in get_credentials
    credentials = self.get_application_default_credentials()
  File "./venv/lib/python3.6/site-packages/pandas_gbq/gbq.py", line 243, in get_application_default_credentials
    credentials, _ = google.auth.default(scopes=[self.scope])
  File "./venv/lib/python3.6/site-packages/google/auth/_default.py", line 281, in default
    credentials, project_id = checker()
  File "./venv/lib/python3.6/site-packages/google/auth/_default.py", line 158, in _get_gae_credentials
    from google.auth import app_engine
  File "./venv/lib/python3.6/site-packages/google/auth/app_engine.py", line 32, in <module>
    from google.appengine.api import app_identity
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load
  File "<frozen importlib._bootstrap>", line 946, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 881, in _find_spec
  File "<frozen importlib._bootstrap>", line 855, in _find_spec_legacy
  File "./venv/lib/python3.6/site-packages/pyximport/pyximport.py", line 253, in find_module
    fp, pathname, (ext,mode,ty) = imp.find_module(fullname,package_path)
  File "~/.pyenv/versions/3.6.1/lib/python3.6/imp.py", line 270, in find_module
    "not {}".format(type(path)))
RuntimeError: 'path' must be None or a list, not <class '_frozen_importlib_external._NamespacePath'>

我们的需求文件如下所示:

cachetools==2.0.1
certifi==2017.7.27.1
chardet==3.0.4
et-xmlfile==1.0.1
fbprophet==0.2
google-api-python-client==1.6.4
google-auth==1.1.1
google-auth-httplib2==0.0.2
google-auth-oauthlib==0.1.1
httplib2==0.10.3
idna==2.6
jdcal==1.3
numpy==1.12.1
oauth2client==4.1.2
oauthlib==2.0.4
openpyxl==2.4.8
pandas==0.20.1
pandas-gbq==0.2.0
psycopg2==2.7.3.1
py==1.4.34
pyasn1==0.3.7
pyasn1-modules==0.1.5
pyodbc==4.0.17
pytest==3.2.2
python-dateutil==2.6.0
pytz==2017.2
PyYAML==3.12
requests==2.18.4
requests-oauthlib==0.8.0
rpy2==2.8.5
rsa==3.4.2
six==1.10.0
SQLAlchemy==1.1.9
sqlalchemy-redshift==0.6.0
uritemplate==3.0.0
urllib3==1.22
xlrd==1.1.0

这发生在具有全新虚拟环境的全新克隆上,在 OS X 上运行 Python 3.6.1。在两台机器上重现。老实说,甚至有点不知所措。它以前有效,但有些东西退步导致它失败,但它死亡的方式表明它不完全是因为我们?

从字面上看,任何线索都会有所帮助! :)

【问题讨论】:

  • 据我所见,您正在 appengine 中运行此代码,对吗?可以分享一下代码吗?是标准环境吗?
  • 不,这是独立的并且在本地运行。我们只使用pandas.read_gbq(query, project_id),而后者又使用BigQuery API library。除此之外没有什么特别的。
  • 有趣...不知道为什么,但它似乎使用来自 appengine 的app_identity 进行身份验证(AFAIK 这应该只在 appengine 标准环境中发生)
  • 原来 Pandas/Google BQ/FB 的 Prophet 之间存在一些冲突。 I raised an issue。不确定这是否可以作为答案?
  • 如果确认存在错误,那么我认为肯定存在。希望其他有同样问题的人也能找到这个。

标签: python python-3.x pandas google-bigquery


【解决方案1】:

这是由于 PyStan(Facebook 的 Prophet 使用)与 Google 身份验证库之一(BigQuery API 使用)之间的冲突造成的错误。与 pyximport 的使用有关。

There's an issue on Prophet here 可以跟踪,但找到修复似乎很困难。

【讨论】:

    【解决方案2】:

    按照here 的建议,我可以通过将 Cython 从 0.25.2 升级到 0.29 来解决这个问题。

    【讨论】:

      猜你喜欢
      • 2021-06-27
      • 2017-02-12
      • 2019-07-19
      • 2020-06-14
      • 2015-09-06
      • 2019-07-03
      • 2017-03-14
      • 2019-09-25
      • 2014-01-10
      相关资源
      最近更新 更多