【问题标题】:Segmentation Fault with PyArg_ParseTuple when passing string from Python从 Python 传递字符串时 PyArg_ParseTuple 出现分段错误
【发布时间】:2019-12-18 03:23:10
【问题描述】:

我正在尝试编写一个接受 numpy 数组作为输入的 C 扩展。一切正常,除非我将字符串作为参数传递。

#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
#include "../../include/Python.h"
#include "../../include/arrayobject.h"

static PyObject *max(PyObject *self, PyObject *args)
{
    PyArrayObject *arr;
    long i, n, strides;

    if (PyArg_ParseTuple(args, "O!", &PyArray_Type, &arr)){
        /* Get some info about the data. */
        n           = PyArray_DIMS(arr)[0];
        strides     = PyArray_STRIDES(arr)[0];
        void *data0 = PyArray_DATA(arr);
        int typenum = PyArray_TYPE(arr);

        if (typenum == NPY_DOUBLE){
            double max = *(double *)data0;
            for (i=0; i<n; ++i){
                if (*(double *)data0 > max){
                    max = *(double *)data0;
                }
                data0 += strides;
            }
            return Py_BuildValue("d", max);
        }
        else if (typenum == NPY_LONG){
            long max = *(long *)data0;
            for (i=0; i<n; ++i){
                if (*(long *)data0 > max){
                    max = *(long *)data0;
                }
                data0 += strides;
            }
            return Py_BuildValue("l", max);
        }
        else {
            PyErr_Format(
                PyExc_TypeError, "\rInput should be a numpy array of numbers."
            );
            return NULL;
        }
    }
    else{
        PyErr_Format(
            PyExc_TypeError, "\rInput should be a numpy array of numbers."
        );
        return NULL;
    }
}

static PyMethodDef DiffMethods[] =
{
    {"max", max, METH_VARARGS, "Compute the maximum of a numpy array."},
    {NULL, NULL, 0, NULL}
};

static struct PyModuleDef cModPyDem =
    {PyModuleDef_HEAD_INIT, "_math_functions", "", -1, DiffMethods};

PyMODINIT_FUNC PyInit__math_functions(void)
{
    import_array();
    return PyModule_Create(&cModPyDem);
}

然后我运行这个 setup.py 脚本:

def configuration(parent_package=None, top_path=None):
    import numpy
    from numpy.distutils.misc_util import Configuration
    config.add_extension('_math_functions', ['_math_functions.c'])

    return config

if __name__ == "__main__":
    from numpy.distutils.core import setup
    setup(configuration=configuration)

使用这些命令:

python setup.py config --compiler=gnu99 build_ext --inplace
rm -rf build/

而且效果很好。该功能在大多数情况下都有效:

In [1]: import _math_functions as mf

In [2]: import numpy as np

In [3]: x = np.random.randint(-1e3, 1e3, size=100)

In [4]: np.max(x), mf.max(x)
Out[4]: (998, 998)

In [5]: x = np.random.rand(100)

In [6]: np.max(x), mf.max(x)
Out[6]: (0.9962604850115798, 0.9962604850115798)

它还可以处理一些不适当的输入:

In [7]: x = np.array([1,2,"bob"])

In [8]: mf.max(x)
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-8-7ced17af9505> in <module>()
----> 1 mf.max(x)

Input should be a numpy array of numbers.

In [9]: mf.max("bob")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-a656f60cf00d> in <module>()
----> 1 mf.max("bob")

Input should be a numpy array of numbers.

以下输入出现问题:

In [10]: x = np.array("Bob")

In [11]: mf.max(x)
Segmentation fault: 11

编辑:我尝试过的一些事情。 使用:

PyArg_ParseTuple(args, "O", &arr)

相反,这仍然给出了段错误。我还在每一行之前添加了printf("i")(使用 i=1、2、...),所以我确信 segfault 发生在PyArg_ParseTuple

我通读了文档并找到了"O&amp;" 选项,但无法使其正常工作。欢迎任何关于如何正确使用的建议。

我也浏览过这些相关的帖子: PyArg_ParseTuple causing segmentation fault

PyArg_ParseTuple SegFaults in CApi (不知道如何解决这个问题......)

Crash when calling PyArg_ParseTuple on a Numpy array

有关如何正确处理此问题的任何线索?我想要的输出是一个 TypeError 被引发。

谢谢!

【问题讨论】:

    标签: python c string api segmentation-fault


    【解决方案1】:

    您是否尝试过添加调试语句来确定代码中发生分段错误的确切位置?

    假设分段错误发生在这里:

    if (PyArg_ParseTuple(args, "O!", &PyArray_Type, &arr)) {
    

    尝试添加不同的解析机制,如"O",以免假设传递了PyArrayObject 实例;然后尝试使用生成的泛型PyObject* 并检查其类型(请参阅PyObject_TypeCheck)并根据类型控制程序的流程。扩展文档中解释了从此处引发异常的方式,但我认为它类似于:

    PyErr_SetString(PyExc_TypeError, "Input should be a numpy array of numbers.");
    return NULL;
    

    【讨论】:

    • 您好,很抱歉回复延迟。不幸的是,这是我已经尝试过的事情之一。我还研究了"O&amp;" 选项,但无法让它发挥作用。我想 numpy 的 max 函数的源代码就足够了,但我找不到它。我会用我迄今为止尝试过的东西来更新我的帖子。
    【解决方案2】:

    哦,天哪,这个问题根本与字符串无关。如果输入是零维的,PyArray_DIMSPyArray_STRIDES 返回NULL,这就是问题所在。我放了更多的打印语句,程序确实通过了PyArg_ParseTuple。我真的是个傻瓜。这是一个完整的工作示例,我只是为这两个指针添加了一个检查。

    #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
    #include "../../include/Python.h"
    #include "../../include/arrayobject.h"
    
    static PyObject *max(PyObject *self, PyObject *args)
    {
        PyArrayObject *arr;
        npy_int i, n, strides;
        void *data0;
    
        if (PyArg_ParseTuple(args, "O!", &PyArray_Type, &arr)){
    
            // Check to make sure input isn't zero dimensional!
            if ((PyArray_DIMS(arr) == NULL) || (PyArray_STRIDES(arr) == NULL)){
                PyErr_Format(PyExc_TypeError,
                             "Input is zero-dimensional.");
                return NULL;
            }
    
            // Useful information about the data.
            int typenum = PyArray_TYPE(arr);
            n           = PyArray_DIMS(arr)[0];
            strides     = PyArray_STRIDES(arr)[0];
            data0       = PyArray_DATA(arr);
    
            if (typenum == NPY_DOUBLE){
                double max = *(double *)data0;
                for (i=0; i<n; ++i){
                    if (*(double *)data0 > max){
                        max = *(double *)data0;
                    }
                    data0 += strides;
                }
                return Py_BuildValue("d", max);
            }
            else if (typenum == NPY_LONG){
                long max = *(long *)data0;
                for (i=0; i<n; ++i){
                    if (*(long *)data0 > max){
                        max = *(long *)data0;
                    }
                    data0 += strides;
                }
                return Py_BuildValue("l", max);
            }
            else {
                PyErr_Format(PyExc_TypeError,
                             "Input should be a numpy array of numbers.");
                return NULL;
            }
        }
        else{
            PyErr_Format(PyExc_TypeError,
                         "Input should be a numpy array of numbers.");
            return NULL;
        }
    }
    
    static PyMethodDef DiffMethods[] =
    {
        {"max", max, METH_VARARGS, "Compute the maximum of a numpy array."},
        {NULL, NULL, 0, NULL}
    };
    
    static struct PyModuleDef cModPyDem =
        {PyModuleDef_HEAD_INIT, "_math_functions", "", -1, DiffMethods};
    
    PyMODINIT_FUNC PyInit__math_functions(void)
    {
        import_array();
        return PyModule_Create(&cModPyDem);
    }
    

    构建与以前相同。到目前为止,这通过了我对其进行的所有测试:

    In [1]: import numpy as np                                                                                                                                  
    
    In [2]: import _math_functions as mf                                                                                                                        
    
    In [3]: for i in range(1000): 
       ...:     for j in range(10): 
       ...:         x = np.random.rand((i+1)*100) 
       ...:         if ((np.max(x) - mf.max(x)) != 0): 
       ...:             print(i, j) 
       ...:         x = np.random.randint(-1e13*(i+1), 1e13*(i+1), size=1000) 
       ...:         if ((np.max(x) - mf.max(x)) !=0): 
       ...:             print(i, j) 
       ...:                                                                                                                                                     
    # Nothing prints, so np.max and mf.max are spitting out the same answer.
    In [4]: mf.max("Bob")                                                                                                                                       
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-4-bc67f3e1c10d> in <module>
    ----> 1 mf.max("Bob")
    
    TypeError: Input should be a numpy array of numbers.
    
    In [5]: mf.max(np.array(1))                                                                                                                                 
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-5-1cb4380527fa> in <module>
    ----> 1 mf.max(np.array(1))
    
    TypeError: Input is zero-dimensional.
    
    In [6]: mf.max(np.array("Bob"))                                                                                                                             
    ---------------------------------------------------------------------------
    TypeError                                 Traceback (most recent call last)
    <ipython-input-6-47b1925b8c3c> in <module>
    ----> 1 mf.max(np.array("Bob"))
    
    TypeError: Input is zero-dimensional.
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2020-11-27
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多