Python Pandas：数据下采样

Question

我的数据如下所示：

TEST
2012-05-01 00:00:00.203 OFF 0
2012-05-01 00:00:11.203 OFF 0
2012-05-01 00:00:22.203 ON 1
2012-05-01 00:00:33.203 ON 1
2012-05-01 00:00:44.203 OFF 0
TEST
2012-05-02 00:00:00.203 OFF 0
2012-05-02 00:00:11.203 OFF 0
2012-05-02 00:00:22.203 OFF 0
2012-05-02 00:00:33.203 ON 1
2012-05-02 00:00:44.203 ON 1
2012-05-02 00:00:55.203 OFF 0

最终，我希望能够将这样的数据下采样到各个日期，例如使用均值、最小值、最大值。 我无法让它为我的数据工作并收到此错误：

TypeError: unhashable type: 'list'

也许它与数据框中的日期格式有关，因为索引行如下所示：

[datetime.datetime(2012, 5, 1, 0, 0, 0, 203000)]   OFF  0

谁能帮忙。 到目前为止我的代码是这样的：

import time
import dateutil.parser
from pandas import *
from pandas.core.datetools import *



t0 = time.clock()

filename = "testdata.dat"

index = []
data = []

with open(filename) as f:
    for line in f:
        if not line.startswith('TEST'):
            line_content =  line.split(' ')

            mydatetime =  dateutil.parser.parse(line_content[0] +  " " + line_content[1])

            del line_content[0] # delete the date
            del line_content[0] # delete the time so that only values remain

            index_row = [mydatetime]
            data_row = []
            for item in line_content:
                data_row.append(item)

            index.append(index_row)
            data.append(data_row)


df = DataFrame(data, index = index)
print df.head()
print df.tail()

print
date_from =  index[0] # first datetime entry in data frame
print date_from
date_to =  index[len(index)-1] #last datetime entry in date frame
print date_to

print date_to[0] - date_from[0]
dayly= DateRange(date_from[0], date_to[0], offset=datetools.DateOffset())
print dayly

grouped = df.groupby(dayly.asof)
#print grouped.mean()
#df2 = df.groupby(daily.asof).agg({'2':np_mean})


time2 = time.clock() - t0
print time2

Answer 1

您最好将所有日期时间插值留给pandas，并用干净的输入流来提供它。然后，您可以使用 read_fwf 分隔字段（对于固定宽度的格式化行）。例如：

import pandas
import StringIO

buf = StringIO.StringIO()
buf.write(''.join(line
    for line in open('f.txt')
    if not line.startswith('TEST')))
buf.seek(0)

df = pandas.read_fwf(buf, [(0, 24), (24, 27), (27, 30)],
        index_col=0, names=['switch', 'value'])
print df

输出：

                        switch  value
2012-05-01 00:00:00.203    OFF      0
2012-05-01 00:00:11.203    OFF      0
2012-05-01 00:00:22.203     ON      1
2012-05-01 00:00:33.203     ON      1
2012-05-01 00:00:44.203    OFF      0
2012-05-02 00:00:00.203    OFF      0
2012-05-02 00:00:11.203    OFF      0
2012-05-02 00:00:22.203    OFF      0
2012-05-02 00:00:33.203     ON      1
2012-05-02 00:00:44.203     ON      1
2012-05-02 00:00:55.203    OFF      0

Answer 2

我对@9​​87654321@ 没有任何经验，但从我的代码中可以看出，

df = DataFrame(data, index = index)

和错误，似乎index 不应该是像 python 列表这样的可变对象。也许这会起作用：

df = DataFrame(data, index = tuple(index))

此外，您的 index_row 和 data_row 本身就是列表似乎并不明显，而您将它们附加到 index 和 data 列表中。