“PerformanceWarning：DataFrame高度碎片化。这通常是多次调用`frame.insert`的结果，性能很差。”

Question

基本上，我对df 数据框中的每一列进行排名，并将其添加到ranking 数据框中。我显然不是很有效地这样做，我想知道是否有人可以指出我正确的方向。

for x in range(1,num_sims+1):
    ranking[x] = df[x].rank(ascending=False, method='min')

完整的错误信息是：

PerformanceWarning: DataFrame is highly fragmented.  This is usually
the result of calling `frame.insert` many times, which has poor
performance.  Consider using pd.concat instead.  To get a
de-fragmented frame, use `newframe = frame.copy()`   ranking[x] =
df[x].rank(ascending=False, method='min')"

Answer 1

重现警告的示例：

import numpy as np
import pandas as pd

# Sample `df`
np.random.seed(5)
df = pd.DataFrame(np.random.randint(1, 100, (4, 5000)))
df.columns = df.columns + 1

num_sims = len(df.columns)  # Placeholder for `num_sims`
ranking = pd.DataFrame()  # Placeholder for `ranking`

for x in range(1, num_sims + 1):
    ranking[x] = df[x].rank(ascending=False, method='min')

PerformanceWarning：DataFrame 高度分散。这通常是多次调用frame.insert的结果，性能较差。考虑使用 pd.concat(axis=1) 一次连接所有列。要获得碎片整理的帧，请使用newframe = frame.copy() 排名[x] = df[x].rank(ascending=False, method='min')

---

改用DataFrame.rank 和concat 修复：

ranking = pd.DataFrame()  # Placeholder for `ranking`
ranking = pd.concat(
    [ranking, df[range(1, num_sims + 1)].rank(ascending=False, method='min')],
    axis=1
)

output 没有错误：

   1     2     3     4     5     6     ...  4995  4996  4997  4998  4999  5000
0   2.0   2.0   3.0   1.0   4.0   1.0  ...   3.0   3.0   3.0   4.0   4.0   1.0
1   1.0   1.0   4.0   2.0   3.0   2.0  ...   4.0   1.0   2.0   3.0   1.0   3.0
2   3.0   4.0   1.0   4.0   1.0   4.0  ...   1.0   4.0   1.0   1.0   3.0   4.0
3   4.0   3.0   2.0   3.0   2.0   3.0  ...   2.0   2.0   3.0   2.0   2.0   2.0

*当然，如果ranking 为空，我们可以直接从df 创建它：

ranking = df[range(1, num_sims + 1)].rank(ascending=False, method='min')

---

检查它们是否产生相同的结果：

import numpy as np
import pandas as pd

np.random.seed(5)
df = pd.DataFrame(np.random.randint(1, 100, (4, 5000)))
df.columns = df.columns + 1
ranking = pd.DataFrame()
num_sims = len(df.columns)

for x in range(1, num_sims + 1):
    ranking[x] = df[x].rank(ascending=False, method='min')

print(ranking.eq(pd.concat(
    [pd.DataFrame(),
     df[range(1, num_sims + 1)].rank(ascending=False, method='min')],
    axis=1
)).all(axis=None))  # True