【问题标题】:How to fill missing values (date & time wise) with specific in pandas如何用熊猫中的特定值填充缺失值(日期和时间)
【发布时间】:2018-11-10 07:14:08
【问题描述】:

我正在尝试使用 pandas 填充缺失值,但无法获得输出。

输入数据:这里缺少一些行值。

             Date_time  current_demand   Temp_Mean  humidity_Mean
0     2018-05-01 00:00         15951.0  300.904267      49.600000
1     2018-05-01 00:15         16075.0  300.904267      49.600000
2     2018-05-01 00:30         15977.0  300.904267      49.600000
3     2018-05-01 00:45         15945.0  300.837600      50.333333
4     2018-05-01 01:00         15868.0  298.889333      59.133333
5     2018-05-01 01:15         15583.0  298.889333      59.133333
6     2018-05-01 01:30         15470.0  298.756000      59.800000
7     2018-05-01 01:45         15301.0  298.756000      59.800000
8     2018-05-01 02:15         14946.0  298.756000      59.800000
9     2018-05-01 02:30         14736.0  298.756000      59.800000
10    2018-05-01 02:45         14630.0  298.502333      59.000000
11    2018-05-01 03:15         14350.0  298.502333      59.000000

我试过的脚本:

import pandas as pd
import numpy as np


df = pd.read_csv(r'submission.csv', index_col=[1], parse_dates=[1], dayfirst=True)

df['Date_time'] = pd.to_datetime(df['Date_time']).dt.time
start = pd.to_datetime(str(df['Date_time'].min()))
end = pd.to_datetime(str(df['Date_time'].max()))
dates = pd.date_range(start=start, end=end, freq='15Min').time


df1 = pd.pivot_table(df, "current_demand", "Temp_Mean", "humidity_Mean").stack(dropna=False).reset_index(name="current_demand")
df1.loc[df1['current_demand'].isnull(), "Temp_Mean", "Temp_Mean" , "humidity_Mean"] = np.nan

经验。输出:

                Date_time  current_demand   Temp_Mean  humidity_Mean
0     2018-05-01 00:00         15951.0  300.904267      49.600000
1     2018-05-01 00:15         16075.0  300.904267      49.600000
2     2018-05-01 00:30         15977.0  300.904267      49.600000
3     2018-05-01 00:45         15945.0  300.837600      50.333333
4     2018-05-01 01:00         15868.0  298.889333      59.133333
5     2018-05-01 01:15         15583.0  298.889333      59.133333
6     2018-05-01 01:30         15470.0  298.756000      59.800000
7     2018-05-01 01:45         15301.0  298.756000      59.800000
8     2018-05-01 02:00         0        0                 0
9     2018-05-01 02:15         14946.0  298.756000      59.800000
10    2018-05-01 02:30         14736.0  298.756000      59.800000
11    2018-05-01 02:45         14630.0  298.502333      59.000000
12    2018-05-01 03:00         0        0               0
13    2018-05-01 03:15         14350.0  298.502333      59.000000

但是在0的地方——用昨天的数据填充()表示前一天的数据或前一天的数据)

请提出建议。提前谢谢你

编辑

df = df.set_index(['Date_time']).asfreq('15T').ffill()
#df = df.set_index('Date_time').resample('15T').ffill() #as same 
#df = df.asfreq('15T').ffill()

df = df.asfreq('15T').fillna(df.shift(1, freq='d'))

为什么我收到 NaN ?请告诉我

                     current_demand  Temp_Mean  humidity_Mean
Date_time                                                    
2018-05-01 00:00:00             NaN        NaN            NaN
2018-05-01 00:15:00             NaN        NaN            NaN
2018-05-01 00:30:00             NaN        NaN            NaN
2018-05-01 00:45:00             NaN        NaN            NaN
2018-05-01 01:00:00             NaN        NaN            NaN

【问题讨论】:

    标签: python python-3.x python-2.7 pandas datetime


    【解决方案1】:

    使用pd.Grouper 和pd.Series.ffill 用前一天的数据填充空数据的一种方法:

    df = pd.DataFrame([['2018-05-01 00:00', 15951.0, 300.904267, 49.600000],
                       ['2018-05-01 00:15', 16075.0, 300.904267, 49.600000],
                       ['2018-05-01 00:30', 15977.0, 300.904267, 49.600000],
                       ['2018-05-01 01:00', 15868.0, 298.889333, 298.889333]],
                      columns=['Date_time', 'current_demand', 'Temp_Mean', 'humidity_Mean'])
    
    df['Date_time'] = pd.to_datetime(df['Date_time'])
    
    grouper = pd.Grouper(key='Date_time', freq='15T')
    
    res = df.groupby(grouper).first().ffill().reset_index()
    
    print(res)
    
                Date_time  current_demand   Temp_Mean  humidity_Mean
    0 2018-05-01 00:00:00         15951.0  300.904267      49.600000
    1 2018-05-01 00:15:00         16075.0  300.904267      49.600000
    2 2018-05-01 00:30:00         15977.0  300.904267      49.600000
    3 2018-05-01 00:45:00         15977.0  300.904267      49.600000
    4 2018-05-01 01:00:00         15868.0  298.889333     298.889333
    

    【讨论】:

      【解决方案2】:

      使用asfreq 或resample 进行正向填充:

      df = pd.read_csv(r'submission.csv', index_col=[1], parse_dates=[1], dayfirst=True)
      
      df = df.asfreq('15T').ffill()
      
      df = df.resample('15T').ffill()
      
      print (df)
      
                           current_demand   Temp_Mean  humidity_Mean
      Date_time                                                     
      2018-05-01 00:00:00         15951.0  300.904267      49.600000
      2018-05-01 00:15:00         16075.0  300.904267      49.600000
      2018-05-01 00:30:00         15977.0  300.904267      49.600000
      2018-05-01 00:45:00         15945.0  300.837600      50.333333
      2018-05-01 01:00:00         15868.0  298.889333      59.133333
      2018-05-01 01:15:00         15583.0  298.889333      59.133333
      2018-05-01 01:30:00         15470.0  298.756000      59.800000
      2018-05-01 01:45:00         15301.0  298.756000      59.800000
      2018-05-01 02:00:00         15301.0  298.756000      59.800000
      2018-05-01 02:15:00         14946.0  298.756000      59.800000
      2018-05-01 02:30:00         14736.0  298.756000      59.800000
      2018-05-01 02:45:00         14630.0  298.502333      59.000000
      2018-05-01 03:00:00         14630.0  298.502333      59.000000
      2018-05-01 03:15:00         14350.0  298.502333      59.000000
      

      如果您想用前几天的小时数替换 NaNs 解决方案是 fillna 与 shifted DataFrame:

      df = df.asfreq('15T').fillna(df.shift(1, freq='d'))
      

      【讨论】:

      • 亲爱的,谢谢,这里出现错误df = df.set_index(['Date_time']) KeyError: 'Date_time'
      • @user8866270 - 如果print (df.index) 是DatetimeIndex,则需要删除.set_index(['Date_time'])
      • @user8866270 - 为Datetimeindex添加了解决方案
      • 加 1 把asfreq 介绍给我!
      • 谢谢@jezrael,为什么我在其他专栏中得到NaN?
      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2018-08-17
      • 1970-01-01
      • 2017-12-12
      • 2018-01-16
      • 2018-05-03
      • 1970-01-01
      相关资源
      最近更新 更多