python genfromtxt 问题

Question

我是 Python 新手……这是我的问题。 对于我在 Python 中测试的优化子例程，我需要解析一个带有数字的 csv 文件。

csv 文件的格式如下：

Support load summary for anchor at node 5,

Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,

Sustained,-3,-2679,120,2012,164,69,,
Operating1,1472,2710,-672,-4520,8743,-2047,,
Maximum,1472,2710,120,2012,8743,69,,
Minimum,-3,-2679,-672,-4520,164,-2047,,

Support load summary for anchor at node 40,

Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,

Sustained,9,-3872,-196,-91,854,-3914,,
Operating1,-2027,-8027,3834,-7573,-9102,-6323,,
Maximum,9,-3872,3834,-91,854,-3914,,
Minimum,-2027,-8027,-196,-7573,-9102,-6323,,

Support load summary for anchor at node 125,

Load combination,FX (N),FY (N),FZ (N),MX (Nm),MY (Nm),MZ (Nm),,

Sustained,-7,-2448,76,264,83,1320,,
Operating1,556,-3771,-3162,-6948,-1367,1272,,
Maximum,556,-2448,76,264,83,1320,,
Minimum,-7,-3771,-3162,-6948,-1367,1272,,

Support load summary for Hanger at node 10,

Load combination,Load (N),,

Sustained,-3668,,
Operating1,-13876,,
Maximum,-3668,,
Minimum,-13876,,

Support load summary for Hanger at node 20B,

Load combination,Load (N),,

Sustained,-14305,,
Operating1,-13359,,
Maximum,-13359,,
Minimum,-14305,,

Support load summary for restraint at node 115B,

Load combination,FX (N),FY (N),FZ (N),,

Sustained,,-5655,,,
Operating1,3696,,
Maximum,,3696,,,
Minimum,,-5655,,,

我的代码主要在以

开头的行上工作

Operating1,
Maximum, 
Minimum,

工作（成本函数）是对这些关键字之一之后的所有数字进行（代数）总和。有时您可以在上面的数据文件中看到，第二列或第三列中只有一个数字。 （见数据文件末尾），有时，像下面的文件片段一样根本没有数字（见下面的 Operating1 行）。

Support load summary for Hanger at node 115B,

Load combination,Load (N),,

Sustained,-5188,,
Operating1,,,
Maximum,,,
Minimum,-5188,,

我正在使用 np.genfromtxt()。效果很好，除非我遇到列中值少于 4 个或有时根本没有值的行。

我在 genfromtxt() 上使用 sum() - 请参阅代码。当只有一个值时，我使用了 float()。当没有时，我试图识别并分配零给总数。我可以为每种情况进行自定义，但我想知道是否有一种通用的、更抽象的方法来读取和汇总不可预测的情况下的数字。

另外，我尝试了“missing_values”和“filling_values”，但它们似乎不起作用。如何计算文件中非零列的数量？

这是目前为止的部分代码：

def optimize(fn, optflag):

    modeltotals = []
    i=0
    csv1 = []
    j = 1 # line # count

    for line in csv.reader(filelist) :
    temp = repr(line)  

    if "Support load summary" in temp :
       csv1.append(line) # just making another list of actionable lines for future use
       if (d): print "\n", line
       continue
    if (optflag == "ope") : # optimize on Operating loads
       if "Operating1" in temp:
          csv1.append(line)
          if (len(line) > 4):
              modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
              if (d): print "Sum of OPE Loads:", modeltotals[i], "\n"
          elif (len(line) > 0 and len(line) <= 4):
              if (d): print "line=", line, "length", len(line)
              line1 = np.genfromtxt(line[1:], delimiter=",")
              if not line1: # meaning if array is empty
                  modeltotals.append(0)
              else:
                  modeltotals.append(np.genfromtxt(line[1:], delimiter=",", missing_values=[0,0,0,0]))
              if (d): print "OPE Max:", modeltotals[i],"\n"

          i +=1
    elif (optflag == "minmax") : #optimize on all loads, min and max.
       #print  "i=", i
       if "Maximum" in temp:
          csv1.append(line)
          if (len(line) > 4):
              modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
              if (d): print "Sum of Maxs:", modeltotals[i]
          elif (len(line) <= 4):
              #line1 = np.genfromtxt(line[1:], delimiter=",", filling_values = 0)
              #modeltotals.append(sum(line1))

              if (d): print "line=", line, "length", len(line)
              line1 = np.genfromtxt(line[1:], delimiter=",")
              print "line1 =", line1
              if not line1: # meaning if array is empty
                  modeltotals.append(0) 
              else:
                  modeltotals.append(np.genfromtxt(line[1:], delimiter=",", filling_values = 0))
              if (d): print "Max:", modeltotals[i]

          i+=1
       elif "Minimum" in temp:
          csv1.append(line)
          if (len(line) > 4):
              #print "#", j, "line", line
              modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
              if (d): print "Sum of Mins:", modeltotals[i]

          elif (len(line) > 0 and len(line) <= 4):
              if (d): print "line=", line, "length", len(line)
              line1 = np.genfromtxt(line[1:], delimiter=",")
              if not line1: # meaning if array is empty
                  modeltotals.append(0)
              else:
                  modeltotals.append(np.genfromtxt(line[1:], delimiter=","))
              if (d): print "Min:", modeltotals[i]

          i +=1
    j+=1
 if len(modeltotals) > 0:
     print modeltotals
     average = float(sum(modeltotals))/len(modeltotals)  #sometimes error here
 else:
     return "000"  # error, seems like no file was analyzed
 if (d):
     print "Current model mean =", average

 del csv1[:]
 return abs(average)

我在不同文件中遇到的几个错误是相似的：

['Support load summary for restraint at node 20B', '']
Traceback (most recent call last):
File "sor4.py", line 190, in <module>
modelmean[filename] = optimize(filename, args.optimizeon)
  File "sor4.py", line 107, in optimize
   modeltotals.append(sum(np.genfromtxt(line[1:], delimiter=",")))
TypeError: iteration over a 0-d array

另一个错误是“无法转换为标量”。

我了解这些错误，但对如何巧妙处理它们的 Python 知之甚少。对不起，很长的帖子；我会更好地更简洁地介绍信息。正如这里的另一位海报所说，我将不胜感激地接受您的回答。谢谢。

Answer 1

我将您的问题简化为以下代码。它检查 nans 和空输入字符串。

from StringIO import StringIO
import numpy as np

def getnumbers(s):
    try:
        res = np.genfromtxt(s, delimiter=",")
        return res[np.where(np.isnan(res), False, True)]
    except IOError as ioe:
        return np.array(0.)

print(sum(getnumbers(StringIO('1., 2., , '))))
print(sum(getnumbers(StringIO(''))))

它给出了结果

3.0
0.0