问题识别原因：ValueError: infinity or too large AFTER normalization

Question

我有一个我首先标准化的数据集，删除 na，现在，我尝试 df[col] = preprocessing.scale(df[col].values) 在这里我得到错误：ValueError: Input contains infinity或者对于 dtype('float64') 来说值太大。

这是我完成的步骤：

1- 通过删除 nan 确保数据表 (pandas) 没有 NAN 2- 使用 pct_change 标准化值 3- 在调用 pct_change 后立即删除 na

然后尝试缩放功能并得到错误

这里是代码 sn-p ：

来自主调用：

dataset = f"./Data/Original/{RATIO_TO_PREDICT}.csv"
df = pd.read_csv(dataset)
df.set_index("Timestamp", inplace = True)

#calculate volume candle type 1

#calculate volume candle type 2

#df['VC1_Future'] = df["VC1"].shift(-FUTURE_PERIOD_PREDICT)
#df['VC1_Target'] = list(map(classify,df["VC1"], df["VC1_Future"]))

#df['VC2_Future'] = df["VC2"].shift(-FUTURE_PERIOD_PREDICT)
#df['VC2_Target'] = list(map(classify,df["VC2"], df["VC2_Future"]))

df.fillna(method="ffill", inplace = True)
df.dropna(inplace=True)

df['Price_Future'] = df["Close"].shift(-FUTURE_PERIOD_PREDICT) # We go N number of time to the future, get that value and put it in this row's FUTURE PRICE value
df['Price_Target'] = list(map(classify,df["Close"], df["Price_Future"])) 
# Now we compare the current price with that future price to see if we went up, down or none, here we use the 0.015 or 1.5% spread to make sure we pass commision

# Now we want to separate part of the data for training and another part for testing
times = sorted(df.index.values)
last_5pct = times[-int(0.1 * len(times))]


# We get the final columns we want, making sure we are not including any of the High, Low, and Open values. Remember that Price Target is last. That is OUR GOAL !!!
#dfs = df[["Close", "Volume", "Price_Future", "Price_Target"]]#, "VC1", "VC2", "VC1_Future", "VC2_Future", "VC1_Target", "VC2_Target", "Price_Future", "Price_Target"]]


# We finally separate the data into two different lists
validation_df = df[(df.index >= last_5pct)]
training_df = df[(df.index < last_5pct)]

# We save each list into a file so that we don't need to make this process walk through again unless A) we get new data B) we loose previous data on hard drive
Message(name)
print(len(df), len(training_df), len(validation_df))
Message(len(df))
#training_df.dropna(inplace=True)
print(np.isfinite(training_df).all())

print('')

#validation_df.dropna(inplace=True)
print(np.isfinite(validation_df).all())


Train_X, Train_Y = preprocess(training_df)

现在，说到功能，这里是开始：

def preprocess(df) :
    df.drop('Price_Future', 1)
    #df.drop('VC1_Future', 1)
    #df.drop('VC2_Future', 1)
    for col in df.columns:
        if col != "Price_Target" and col != "VC1_Target" and col != "VC2_Target":
            df[col] = df[col].pct_change() # gets the percent change, other than the volume, the data now should sit between -1 and 1, the formula : (value[i] / value[i-1]) - 1
            df.dropna(inplace=True)
            df[col] = preprocessing.scale(df[col].values)

当我调用 main 时，您可能注意到了，我正在检查 nan，结果：

Open             True
High             True
Low              True
Close            True
Volume           True
Price_Future    False
Price_Target     True
dtype: bool

在函数的开头，我删除了 Price_Future 列，那么，为什么我在缩放线处收到此错误？

另外，上面的代码会引起很多警告：

试图在 DataFrame 的切片副本上设置一个值。 尝试改用 .loc[row_indexer,col_indexer] = value

但我是 python 和所有这些东西的新手，所以我不知道如何修复函数上的代码。

请有人帮忙。

谢谢

Answer 1

哎呀，找到主要问题了；

df[col] = preprocessing.scale(df[col].values)

错了

df[col] = preprocessing.scale(df[col])

注意 scale 调用中缺少 .values !!!

但是请有人帮我处理这些警告消息。