【发布时间】:2021-03-03 09:31:30
【问题描述】:
以下示例代码包含三个for 循环:
import numpy as np
import pandas as pd
#Generating a sample (ndarray) of 25 particles with 3 random coordinates in the range between 0 and 3.
#Maybe think of the particles as contained in a cube of 3 x 3 x 3 units.
sample_data = np.random.uniform(0, 3, (25,3))
#Converting the narray into a dataframe
df = pd.DataFrame(data = sample_data, columns = ['A', 'B', 'C'])
print(df)
#Generating another narray which will store the number of particles in each cell of the cube
#Each cell has dimentions 1 x 1 x 1; total cells = 27
counts_in_cells = np.empty((3, 3, 3))
counts_in_cells[:] = np.NaN
#Three nested loops to count the number of particles in each cell
for i, x_low in enumerate(np.arange(0, 3, 1)):
for j, y_low in enumerate(np.arange(0, 3, 1)):
for k, z_low in enumerate(np.arange(0, 3, 1)):
#Specifying filtering conditions for three dimentions of the cells
x_condition = (df['A'] >= x_low) & (df['A'] < (x_low + 1))
y_condition = (df['B'] >= y_low) & (df['B'] < (y_low + 1))
z_condition = (df['C'] >= z_low) & (df['C'] < (z_low + 1))
#Applying the filtering conditions
df_select = df[x_condition & y_condition & z_condition]
#Counting the particles in cells (desired outcome)
counts_in_cells[i][j][k] = len(df_select)
#Paricles in each cell
print(counts_in_cells)
示例输入
期望的结果
快速运行
此示例代码可以立即在 Kaggle 上运行:https://www.kaggle.com/awaismirza/counting-particles-in-each-cell-of-a-cube。
问题
我想避免三个循环,因为此代码的实际版本需要几分钟才能运行。 (它有 90k 个粒子和一个更大的立方体。)此外,我必须运行实际代码 6k 次,这需要很多天。
有没有办法(Pandas 功能或 NumPy 掩码之类的)我可以避免循环并更快地运行代码?
原始代码
代码的实际版本可用here 但上面的例子 代码应该足以理解问题。
【问题讨论】:
-
请查看How to make good pandas examples,并在您的问题文本中提供示例输入和预期输出,而不是作为图像或外部链接,以制作minimal reproducible example,以便我们更好地理解如何提供帮助
-
另外,由于您的代码有效,但您正在寻求改进方面的帮助,您可能希望改为在 Code Review 发帖
-
@G.Anderson,我已经编辑了我的问题,以便在一个小示例代码的上下文中提供示例输入和预期输出。如果需要更多信息,请告诉我。感谢您让我了解代码审查。我也会在那里发帖。
标签: python pandas performance dataframe filtering