使用 numpy.genfromtxt 给出 TypeError: Can't convert 'bytes' object to str 隐式

Question

我有一个来自 kaggle.com 的 python 项目。我在读取数据集时遇到问题。它有一个 csv 文件。我们需要读入它并将目标和训练部分放入数组中。

这里是数据集的前3行（目标列是第19列，特征是前18列）：

user    gender  age how_tall_in_meters  weight  body_mass_index x1  
debora  Woman   46  1.62    75  28.6    -3  
debora  Woman   46  1.62    75  28.6    -3  

此处未显示的目标列具有字符串值。

from pandas import read_csv
import numpy as np
from sklearn.linear_model.stochastic_gradient import SGDClassifier
from sklearn import preprocessing
import sklearn.metrics as metrics
from sklearn.cross_validation import train_test_split

#d = pd.read_csv("data.csv", dtype={'A': np.str(), 'B': np.str(), 'S': np.str()})

dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]
target = np.array([x[19] for x in dataset])
train = np.array([x[1:] for x in dataset])

print(target)

我得到的错误是：

Traceback (most recent call last):
  File "C:\Users\Cameron\Desktop\Project - Machine learning\datafilesforproj\SGD_classifier.py", line 12, in <module>
    dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]
  File "C:\Python33\lib\site-packages\numpy\lib\npyio.py", line 1380, in genfromtxt
    first_values = split_line(first_line)
  File "C:\Python33\lib\site-packages\numpy\lib\_iotools.py", line 217, in _delimited_splitter
    line = line.split(self.comments)[0]
TypeError: Can't convert 'bytes' object to str implicitly

Answer 1

您需要将bytes 对象显式解码为str 对象，正如TypeError 所暗示的那样。

# For instance, interpret as UTF-8 (depends on your source)
self.comments = self.comments.decode('utf-8')

Answer 2

代替：

dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]

试试这个：

dataset = np.genfromtxt('C:\\\\..\\\\..\\\train.csv', delimiter=',', dtype='None')[1:]

请注意，您必须使用额外的“\”来转义另一个。

Answer 3

对我有用的是换线

dataset = np.genfromtxt(open('data.csv','r'), delimiter=',', dtype='f8')[1:]

到

dataset = np.genfromtxt('data.csv', delimiter=',', dtype='f8')[1:]

（不幸的是，我不太确定根本问题是什么）

Answer 4

根据https://mail.python.org/pipermail/python-list/2012-April/622487.html 你可能需要

import io
import sys
inpstream = io.open('data.csv','rb')
dataset = np.genfromtxt(inpstream, delimiter=',', dtype='f8')[1:]

在http://docs.scipy.org/doc/numpy/reference/generated/numpy.genfromtxt.html中显示的示例中 用作文件的对象属于StringIO 类。 不过，从函数的规范来看，我猜应该可以传递文件名。

Answer 5

这实际上是 numpy 中的一个错误，参见。 issue #3184.

我将复制我在那里提出的解决方法：

import functools
import io
import numpy as np
import sys

genfromtxt_old = np.genfromtxt
@functools.wraps(genfromtxt_old)
def genfromtxt_py3_fixed(f, encoding="utf-8", *args, **kwargs):
  if isinstance(f, io.TextIOBase):
    if hasattr(f, "buffer") and hasattr(f.buffer, "raw") and \
    isinstance(f.buffer.raw, io.FileIO):
      # Best case: get underlying FileIO stream (binary!) and use that
      fb = f.buffer.raw
      # Reset cursor on the underlying object to match that on wrapper
      fb.seek(f.tell())
      result = genfromtxt_old(fb, *args, **kwargs)
      # Reset cursor on wrapper to match that of the underlying object
      f.seek(fb.tell())
    else:
      # Not very good but works: Put entire contents into BytesIO object,
      # otherwise same ideas as above
      old_cursor_pos = f.tell()
      fb = io.BytesIO(bytes(f.read(), encoding=encoding))
      result = genfromtxt_old(fb, *args, **kwargs)
      f.seek(old_cursor_pos + fb.tell())
  else:
    result = genfromtxt_old(f, *args, **kwargs)
  return result

if sys.version_info >= (3,):
  np.genfromtxt = genfromtxt_py3_fixed

将它放在代码的顶部后，您可以再次使用np.genfromtxt，它应该可以在 Python 3 中正常工作。