int - 将日期时间转换为 Unix 时间纪元时出现字符串类型错误答案

【问题标题】：int - String Type error while converting datetime to Unix Time epochint - 将日期时间转换为 Unix 时间纪元时出现字符串类型错误
【发布时间】：2023-03-30 10:31:01
【问题描述】：

我正在尝试将 Datetime 转换为 Unix 时间纪元，但出现以下错误。

输入：

userid,datetime,latitude,longitude
156,2014-02-01 00:00:00.739166+01,41.8836718276551,12.4877775603346
187,2014-02-01 00:00:01.148457+01,41.9285433333333,12.4690366666667
297,2014-02-01 00:00:01.220066+01,41.8910686119733,12.4927045625339
89,2014-02-01 00:00:01.470854+01,41.7931766914244,12.4321219603157
79,2014-02-01 00:00:01.631136+01,41.90027472,12.46274618
191,2014-02-01 00:00:02.048546+01,41.8523047579646,12.5774065771898
343,2014-02-01 00:00:02.647839+01,41.8921718255185,12.4696996165151
341,2014-02-01 00:00:02.709888+01,41.9102125627332,12.4770004336041
260,2014-02-01 00:00:03.458195+01,41.8658208551143,12.4655221109313

程序：

import pandas as pd
import numpy as np
import io

df = pd.read_csv('input.csv', 
                 #header=None, #no header in csv
                 header=['userid','datetime','latitude','longitude'], #set custom column names
                 parse_dates=['datetime']) #parse columns d, e to datetime

df['datetime'] = df['datetime'].astype(np.int64) // 10**9
#df['e'] = df['e'].astype(np.int64) // 10**9

df.to_csv('output.csv', header=True, index=False)

上述程序在 python 2.7 中运行良好，但不是我已经升级到 python 3.x Anaconda 我无法得到结果

错误：

  File "pandas\parser.pyx", line 519, in pandas.parser.TextReader.__cinit__ (pandas\parser.c:5907)

TypeError: Can't convert 'int' object to str implicitly

编辑：输入文件here

【问题讨论】：

标签： python pandas datetime unix-timestamp

【解决方案1】：

如果 csv 没有标头，则需要参数 names 和 parse_dates 和 [1] - 尝试将第二列解析为 datetime：

import pandas as pd
import numpy as np
from pandas.compat import StringIO

temp=u"""156,2014-02-01 00:00:00.739166+01,41.8836718276551,12.4877775603346
187,1014-02-01 00:00:01.148457+01,41.9285433333333,12.4690366666667
297,2014-02-01 00:00:01.220066+01,41.8910686119733,12.4927045625339
89,2014-02-01 00:00:01.470854+01,41.7931766914244,12.4321219603157
79,2014-02-01 00:00:01.631136+01,41.90027472,12.46274618
191,2014-02-01 00:00:02.048546+01,41.8523047579646,12.5774065771898
343,2014-02-01 00:00:02.647839+01,41.8921718255185,12.4696996165151
341,2014-02-01 00:00:02.709888+01,41.9102125627332,12.4770004336041
260,2014-02-01 00:00:03.458195+01,41.8658208551143,12.4655221109313"""
#after testing replace 'StringIO(temp)' to 'filename.csv'
df = pd.read_csv(StringIO(temp), 
                parse_dates=[1], 
                names=['userid','datetime','latitude','longitude'])
#print (df)

#check dtypes if datetime it is OK
print (df['datetime'].dtypes)
datetime64[ns]

df['datetime'] = df['datetime'].astype(np.int64) // 10**9
print (df)
   userid    datetime   latitude  longitude
0     156  1391209200  41.883672  12.487778
1     187  1391209201  41.928543  12.469037
2     297  1391209201  41.891069  12.492705
3      89  1391209201  41.793177  12.432122
4      79  1391209201  41.900275  12.462746
5     191  1391209202  41.852305  12.577407
6     343  1391209202  41.892172  12.469700
7     341  1391209202  41.910213  12.477000
8     260  1391209203  41.865821  12.465522

另一个可能的问题是错误数据，在我的示例第二行中：

import pandas as pd
from pandas.compat import StringIO

temp=u"""156,2014-02-01 00:00:00.739166+01,41.8836718276551,12.4877775603346
187,1014-02-01 00:00:01.148457+01,41.9285433333333,12.4690366666667
297,2014-02-01 00:00:01.220066+01,41.8910686119733,12.4927045625339
89,2014-02-01 00:00:01.470854+01,41.7931766914244,12.4321219603157
79,2014-02-01 00:00:01.631136+01,41.90027472,12.46274618
191,2014-02-01 00:00:02.048546+01,41.8523047579646,12.5774065771898
343,2014-02-01 00:00:02.647839+01,41.8921718255185,12.4696996165151
341,2014-02-01 00:00:02.709888+01,41.9102125627332,12.4770004336041
260,2014-02-01 00:00:03.458195+01,41.8658208551143,12.4655221109313"""
#after testing replace 'StringIO(temp)' to 'filename.csv'
df = pd.read_csv(StringIO(temp), 
                 parse_dates=[1], 
                 names=['userid','datetime','latitude','longitude'])

#print (df)

#check dtypes - parse failed, get object dtype
print (df['datetime'].dtypes)
object

使用 to_datetime 和参数 errors='coerce' 解析到日期时间 - 它将坏数据替换为 NaT，然后将 NaT 替换为某个值，例如0 (1970-01-01 00:00:00.000000) 与 fillna:

df['datetime'] = pd.to_datetime(df['datetime'], errors='coerce').fillna(0)
print (df)
   userid                   datetime   latitude  longitude
0     156 2014-01-31 23:00:00.739166  41.883672  12.487778
1     187 1970-01-01 00:00:00.000000  41.928543  12.469037
2     297 2014-01-31 23:00:01.220066  41.891069  12.492705
3      89 2014-01-31 23:00:01.470854  41.793177  12.432122
4      79 2014-01-31 23:00:01.631136  41.900275  12.462746
5     191 2014-01-31 23:00:02.048546  41.852305  12.577407
6     343 2014-01-31 23:00:02.647839  41.892172  12.469700
7     341 2014-01-31 23:00:02.709888  41.910213  12.477000
8     260 2014-01-31 23:00:03.458195  41.865821  12.465522


df['datetime'] = df['datetime'].astype(np.int64) // 10**9
print (df)
   userid    datetime   latitude  longitude
0     156  1391209200  41.883672  12.487778
1     187           0  41.928543  12.469037
2     297  1391209201  41.891069  12.492705
3      89  1391209201  41.793177  12.432122
4      79  1391209201  41.900275  12.462746
5     191  1391209202  41.852305  12.577407
6     343  1391209202  41.892172  12.469700
7     341  1391209202  41.910213  12.477000
8     260  1391209203  41.865821  12.465522

编辑：

如果还有标题并且需要替换列名需要header=0添加到read_csv。

【讨论】：

非常感谢.. 这太棒了！但我只能接受一个答案。你认为你能帮我解决这个问题吗：从 2.x 迁移到 3.x 感觉要做很多改变..stackoverflow.com/questions/43970972/…
是的，接受哪个答案由您决定。
在您的第二个问题中 - 代码中的哪一行返回错误？
它与基于 Unix Time Epoch 的数据操作大致相同。

【解决方案2】：

pd.read_csv 中的 header 参数需要一个整数或整数列表，而不是字符串列表。

from io import StringIO
file="""
userid,datetime,latitude,longitude
156,2014-02-01 00:00:00.739166+01,41.8836718276551,12.4877775603346
187,2014-02-01 00:00:01.148457+01,41.9285433333333,12.4690366666667
297,2014-02-01 00:00:01.220066+01,41.8910686119733,12.4927045625339
89,2014-02-01 00:00:01.470854+01,41.7931766914244,12.4321219603157
79,2014-02-01 00:00:01.631136+01,41.90027472,12.46274618
191,2014-02-01 00:00:02.048546+01,41.8523047579646,12.5774065771898
343,2014-02-01 00:00:02.647839+01,41.8921718255185,12.4696996165151
341,2014-02-01 00:00:02.709888+01,41.9102125627332,12.4770004336041
260,2014-02-01 00:00:03.458195+01,41.8658208551143,12.4655221109313"""

让我们试试这个 read_csv 语句：

df = pd.read_csv(StringIO(file),parse_dates=['datetime'])
df['datetime'] = df['datetime'].astype(np.int64) // 10**9

print(df.head())

输出：

   userid    datetime   latitude  longitude
0     156  1391209200  41.883672  12.487778
1     187  1391209201  41.928543  12.469037
2     297  1391209201  41.891069  12.492705
3      89  1391209201  41.793177  12.432122
4      79  1391209201  41.900275  12.462746

【讨论】：

感谢您的回答，但我收到以下错误：ValueError: 'datetime' is not in list 来自 csv 文件的相同输入完全没有变化
你能把 CSV 文件的前三行粘贴到这里吗？
我已将输入文件作为链接提供，您可以从那里下载
应该可以。而且您正在执行与我在这里相同的 pd.read_cvs 刚刚将 StringIO(file) 更改为预期 csv 文件的正确路径？确保 parse_dates=['datetime'] 用括号表示列表。
您的程序中不需要 StringIO，只需 pd.read_csv('input.csv', parse_dates=['datetime'])