python 2.7 + RHEL 6.5 + Shapely-1.4.4：内存错误

Question

我在 RHEL 6.5 上的两个部署中遇到了这个问题。无法通过 Google 搜索找到任何答案

基础安装包

(nout)$ sudo rpm -qa | grep geos
geos-devel-3.3.2-1.el6.x86_64
geos-3.3.2-1.el6.x86_64
geos-python-3.3.2-1.el6.x86_64

形状安装

(nout)$ easy_install -Z Shapely-1.4.4.tar.gz 
Processing Shapely-1.4.4.tar.gz
Writing /tmp/easy_install-iLylTY/Shapely-1.4.4/setup.cfg
Running Shapely-1.4.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-iLylTY/Shapely-1.4.4/egg-dist-tmp-xFmed4
Numpy or Cython not available, shapely.vectorized submodule not being built.
shapely/speedups/_speedups.c: In function ‘__pyx_pf_7shapely_8speedups_9_speedups_2geos_linestring_from_py’:
shapely/speedups/_speedups.c:1603: warning: assignment discards qualifiers from pointer target type
shapely/speedups/_speedups.c:2087: warning: assignment discards qualifiers from pointer target type
shapely/speedups/_speedups.c:2553: warning: assignment discards qualifiers from pointer target type
shapely/speedups/_speedups.c: In function ‘__pyx_pf_7shapely_8speedups_9_speedups_4geos_linearring_from_py’:
shapely/speedups/_speedups.c:3118: warning: assignment discards qualifiers from pointer target type
shapely/speedups/_speedups.c:3127: warning: pointer targets in passing argument 3 of ‘GEOSCoordSeq_getSize_r’ differ in signedness
/usr/include/geos_c.h:321: note: expected ‘unsigned int *’ but argument is of type ‘int *’
shapely/speedups/_speedups.c:3614: warning: assignment discards qualifiers from pointer target type
shapely/speedups/_speedups.c:4158: warning: assignment discards qualifiers from pointer target type
shapely/speedups/_speedups.c: At top level:
shapely/speedups/_speedups.c:924: warning: ‘__pyx_f_7shapely_8speedups_9_speedups_get_geos_context_handle’ defined but not used
shapely/speedups/_speedups.c:1010: warning: ‘__pyx_f_7shapely_8speedups_9_speedups_geos_from_prepared’ defined but not used
zip_safe flag not set; analyzing archive contents...
shapely.geos: module references __file__
Adding Shapely 1.4.4 to easy-install.pth file

Python 外壳

(nout)$ python
Python 2.7.8 (default, Sep 12 2014, 14:39:38) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from shapely.geometry import Polygon, Point
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/omd/nocout/nout/lib/python2.7/site-packages/Shapely-1.4.4-py2.7-linux-x86_64.egg/shapely/geometry/__init__.py", line 4, in <module>
    from .base import CAP_STYLE, JOIN_STYLE
  File "/omd/nocout/nout/lib/python2.7/site-packages/Shapely-1.4.4-py2.7-linux-x86_64.egg/shapely/geometry/base.py", line 9, in <module>
    from shapely.coords import CoordinateSequence
  File "/omd/nocout/nout/lib/python2.7/site-packages/Shapely-1.4.4-py2.7-linux-x86_64.egg/shapely/coords.py", line 8, in <module>
    from shapely.geos import lgeos
  File "/omd/nocout/nout/lib/python2.7/site-packages/Shapely-1.4.4-py2.7-linux-x86_64.egg/shapely/geos.py", line 194, in <module>
    error_h = EXCEPTION_HANDLER_FUNCTYPE(error_handler)
MemoryError

Python 外壳

>>> from shapely.ops import transform
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/omd/nocout/nout/lib/python2.7/site-packages/Shapely-1.4.4-py2.7-linux-x86_64.egg/shapely/ops.py", line 13, in <module>
    from shapely.geos import lgeos
  File "/omd/nocout/nout/lib/python2.7/site-packages/Shapely-1.4.4-py2.7-linux-x86_64.egg/shapely/geos.py", line 194, in <module>
    error_h = EXCEPTION_HANDLER_FUNCTYPE(error_handler)
MemoryError

在调用import 时收到MemoryError

我无法理解原因。

Answer 1

对于RHEL + selinux enabled + /tmp as noexec，任何试图在/tmp 中访问和执行的模块都会产生内存错误。

在我的例子中，/tmp 是 noexec 并且 selinux 被强制执行

/tmp 默认为 defaults,noexec,nosuid,nodev。

我设置：查看：https://bugzilla.redhat.com/show_bug.cgi?id=645193#c11

$ getsebool -a | grep httpd_tmp_exec
httpd_tmp_exec --> on

问题仍然存在，如果我理解正确，这是因为：https://bugzilla.redhat.com/show_bug.cgi?id=582009

The libffi library tries to write to /tmp, 
which isn't allowed for the apache user in a default SELinux config   
(https://bugzilla.redhat.com/show_bug.cgi?id=582009). 
In this environment, importing ctypes always throws a MemoryError

所以httpd_tmp_exec --> on 选项对我没有用。

我查看了python lib文件，搜索/tmp声明，我找到了tempfile.py line 147：def _candidate_tempdir_list()

我看到如果有 envnames TMPDIR , TEMP, TMP ，python 将获取临时位置。

设置哪个变量？为此，我进入了libffi 源代码libffi-3.0.6。文件：src/closures.c 第 290 行 open_temp_exec_file_env, "TMPDIR", 0。

TMPDIR 是要设置的候选对象。

我继续安装/opt/app/tmp，设置export TMPDIR=/opt/app/tmp

这一切工作正常，直到我使用python idle，当我继续进行nginx + uwsgi 部署时，我遇到了同样的错误。现在我不确定为什么会发生这种情况，我认为一定是uwsgi 执行问题。所以我先重新安装了uwsgi，然后我修改了/location为nginx.conf并查看：http://nginx.org/en/docs/http/ngx_http_uwsgi_module.html#uwsgi_temp_path设置：uwsgi_temp_path

我重启了所有的服务，nginx，uwsgi。并尝试过。有效。

我也第一次尝试了：（安全）HACK：Python ctypes MemoryError in fcgi process from PIL library

结果令我失望。