导入 apache_beam 时的 Google App Engine dev_appserver.py IOError答案

【问题标题】：Google App Engine dev_appserver.py IOError when importing apache_beam导入 apache_beam 时的 Google App Engine dev_appserver.py IOError
【发布时间】：2017-08-13 13:18:19
【问题描述】：

我有一个 App Engine 应用程序，它在我做的那一刻就中断了

import apache_beam

我可以确认依赖项已安装在应用程序运行的虚拟环境中。从下面的堆栈跟踪来看，它似乎出于某种原因试图访问 /dev/null，但它失败了。

INFO     2017-08-13 13:03:37,980 stubs.py:50] Sandbox prevented access to file "/home/raul/Downloads/google-cloud-sdk"
INFO     2017-08-13 13:03:37,980 stubs.py:51] If it is a static file, check that `application_readable: true` is set in your app.yaml
INFO     2017-08-13 13:03:37,980 stubs.py:50] Sandbox prevented access to file "/home/raul/Downloads/google-cloud-sdk/platform"
INFO     2017-08-13 13:03:37,980 stubs.py:51] If it is a static file, check that `application_readable: true` is set in your app.yaml
INFO     2017-08-13 13:03:38,260 stubs.py:50] Sandbox prevented access to file "/dev/null"
INFO     2017-08-13 13:03:38,260 stubs.py:51] If it is a static file, check that `application_readable: true` is set in your app.yaml
ERROR    2017-08-13 13:03:38,261 wsgi.py:263] 
Traceback (most recent call last):
  File "/home/raul/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/home/raul/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/home/raul/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
    obj = __import__(path[0])
  File "/home/raul/Documents/football/main.py", line 7, in <module>
    import apache_beam
  File "/home/raul/Documents/football/env/lib/python2.7/site-packages/apache_beam/__init__.py", line 75, in <module>
    import apache_beam.internal.pickler
  File "/home/raul/Documents/football/env/lib/python2.7/site-packages/apache_beam/internal/pickler.py", line 38, in <module>
    import dill
  File "/home/raul/Documents/football/env/lib/python2.7/site-packages/dill/__init__.py", line 27, in <module>
    from .dill import dump, dumps, load, loads, dump_session, load_session, \
  File "/home/raul/Documents/football/env/lib/python2.7/site-packages/dill/dill.py", line 165, in <module>
    FileType = type(open(os.devnull, 'rb', buffering=0))
  File "/home/raul/Downloads/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/stubs.py", line 260, in __init__
    raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: '/dev/null'

任何想法可能导致此问题以及如何解决？

【问题讨论】：

你是部署在谷歌云还是本地环境？原因是，我也在尝试添加一个 cron 来通过 appengine 运行数据流管道，它会对我有所帮助。 :)
我没有部署它，因为沙盒限制，我一开始就无法让它在开发服务器上运行。灵活的环境似乎是这种用例的必经之路

标签： python google-app-engine apache-beam

【解决方案1】：

GAE Python sandbox 对您的应用代码允许执行的操作非常严格。看起来apache-beam 使用的dill 包需要访问沙盒上不允许的特殊文件（本例中为/dev/null 设备文件）。

您要么更改代码以满足所有沙盒要求，要么尝试使用the flexible environment，这更加宽容（但总体而言它是另一种野兽）。请参阅Choosing an App Engine Environment 和Migrating Services from the Standard Environment to the Flexible Environment。

旁注：回溯表明您的应用程序尝试从本地系统安装执行 apache-beam，这在标准环境中也不起作用 - 它需要被供应到您的应用程序中，请参阅Installing a third-party library。

【讨论】：