Python iloc 循环执行时间过长。技术分析计算

Question

我开发了一个代码来分析大量股票价格。基本上，它使用两个技术指标（MACD 和 EMA）并创建一个技术分析标志。

代码正在运行，这很好，但执行起来需要太多时间，很可能是因为使用 iloc 进行迭代。您对提高速度有什么建议吗？我在下面提供一个例子：

import pandas as pd
import numpy as np
import time

df = pd.DataFrame(np.random.uniform(low=2, high=5.5, size=(10000,)), columns=['Close'])
close = df['Close'].astype(float)

def MACD(first,second,signal):
    df['EMA'+str(first)] = close.ewm(span=first).mean()
    df['EMA'+str(second)] = close.ewm(span=second).mean()
    df['MACD']=df['EMA'+str(first)]-df['EMA'+str(second)]
    df['signal']=df.MACD.ewm(span=signal).mean()
    df['MACD_ind'] = 0
    
    for i in range (second+signal, len(df)):
        if df.MACD.iloc[i]>df.signal.iloc[i] and df.MACD.iloc[i-1]<df.signal.iloc[i-1]:
            df.loc[i,'MACD_ind']=1
        if df.MACD.iloc[i]<df.signal.iloc[i] and df.MACD.iloc[i-1]>df.signal.iloc[i-1]:
            df.loc[i,'MACD_ind']=-1
                
def EMA(first,second):
    df['EMA'+str(first)] = close.rolling(window=first).mean()
    df['EMA'+str(second)] = close.rolling(window=second).mean()
    df['EMAdif'] = df['EMA'+str(first)]-df['EMA'+str(second)]
    df['EMA_ind'] = 0
    for i in range (second, len(df)):
        if df.EMAdif.iloc[i]>0 and df.EMAdif.iloc[i-1]<0:
            df.loc[i,'EMA_ind']=1
        if df.EMAdif.iloc[i]<0 and df.EMAdif.iloc[i-1]>0:
            df.loc[i,'EMA_ind']=-1


split_time = time.time()
TA_ind=list()

MACD(12, 26, 9)
TA_ind.append('MACD_ind')
print("MACD--- %s seconds ---" % (time.time() - split_time))
split_time = time.time()    
        
EMA(20,50)
TA_ind.append('EMA_ind')
print("EMA--- %s seconds ---" % (time.time() - split_time))
split_time = time.time()

Answer 1

我发现这个名为 shift 的数据框函数对我有很大帮助。

def MACD(first,second,signal):
    df['EMA'+str(first)] = close.ewm(span=first).mean()
    df['EMA'+str(second)] = close.ewm(span=second).mean()
    df['MACD']=df['EMA'+str(first)]-df['EMA'+str(second)]
    df['signal']=df.MACD.ewm(span=signal).mean()
    df['dif']=df['MACD']-df['signal']
    df['dif_shift']=df.dif.shift(1)
    df['MACD_ind'] = 0
    df['MACD_ind']=np.where((df['dif']>0) & (df['dif_shift']<0),1,df['MACD_ind'])
    df['MACD_ind']=np.where((df['dif']<0) & (df['dif_shift']>0),-1,df['MACD_ind'])
                
def EMA(first,second):
    df['EMA'+str(first)] = close.rolling(window=first).mean()
    df['EMA'+str(second)] = close.rolling(window=second).mean()
    df['EMAdif'] = df['EMA'+str(first)]-df['EMA'+str(second)]
    df['EMAdif_shift'] = df.EMAdif.shift(1)
    df['EMA_ind'] = 0
    df['EMA_ind']=np.where((df['EMAdif']>0) & (df['EMAdif_shift']<0),1,df['EMA_ind'])
    df['EMA_ind']=np.where((df['EMAdif']<0) & (df['EMAdif_shift']>0),-1,df['EMA_ind'])

Answer 2

遍历字典比遍历DataFrame 快得多。因此，您可以将您的 DataFrame 转换为字典。但是您不能使用 pandas 内置的函数，如 MACD 或 ewm。所以你应该自己写这个函数。然后结果你有更快的程序。例如，我使用您的数据框执行此操作。首先我循环遍历DataFrame 本身，然后将其转换为字典并循环遍历它。

df = pd.DataFrame(np.random.uniform(low=2, high=5.5, size=(10000,2)), columns=['Close', 'Open'])
st = time.time()
for i in range(len(df)):
    a = df.iloc[i]['Close'] - df.iloc[i]['Open']
print(time.time() - st)

这个的执行时间是2.005403757095337。

df = pd.DataFrame(np.random.uniform(low=2, high=5.5, size=(10000,2)), columns=['Close', 'Open'])
df_dict = df.to_dict()
st = time.time()
for i in range(len(df)):
    a = df_dict['Close'][i] - df_dict['Open'][i]
print(time.time() - st)

但是这个的执行时间是0.0029413700103759766。这意味着第二种方法大约快一千倍！