迭代 2 个数组并执行操作的最快方法

Question

我有一个函数，它将 2 个零数组和一个数组作为输入，每个数组约 8000 个元素。我的函数 eps 计算这些数组的统计数据并返回输出。只需检查 0 并注意在数组中找到 0 的索引，操作很简单。我尽力优化速度，但我能得到的最好结果是使用 timeit 库 4.5 ~5 秒（对于 18k 数组对）。时间很重要，因为我需要在数十亿个数组对上运行这个函数。

    #e.g. inputs
    #ts_1 = [0,1,1,0,0,1,1,0,......]
    #ts_2 = [1,1,1,1,1,1,1,0,......]
    # tau = any integer or float
    
def eps(ts_1, ts_2, tau):  
    
    n1 = 0
    n2 = 0
    Q_tau = 0
    q_tau = 0

    event_index1 = [index for index, item in enumerate(ts_1) if item == 0]
    n1 = ts_1.count(0)
    event_index2 = [index for index, item in enumerate(ts_2) if item == 0]
    n2 = ts_2.count(0)


    # tried numpy based on @Ram comment below, no improvement
    event_index1, = np.where(np.array(ts_1) == 0)
    n1 = event_index1.shape[0]
    
    event_index2, = np.where(np.array(ts_2) == 0)
    n2 = event_index2.shape[0]
    # tried numpy based on @Ram comment below, no improvement   

    if (n1 == 0 or n2 == 0):
        Q_tau = 0
    else:
        c_ij = 0  
        matching_idx = set(event_index1).intersection(event_index2)
        c_ij = c_ij + (0.5 *len(matching_idx) )
        
        for x,y in product(event_index1,event_index2):
            if x-y > 0 and (x-y)<= tau:
                c_ij = c_ij +1  
        
        c_ji = 0            
        matching_idx_2 = set(event_index2).intersection(event_index1)         
        c_ji = c_ji + (0.5 *len(matching_idx_2) )
        
        for x,y in product(event_index2,event_index1):
            if x-y > 0 and (x-y)<= tau:
                c_ji = c_ji +1                       
                  
        Q_tau = (c_ij+c_ji)/math.sqrt( n1 * n2 )
        q_tau = (c_ij - c_ji)/math.sqrt( n1 * n2 )
    
    return Q_tau, q_tau

Answer 1

根据我之前的 cmets，考虑到在产品中排列两个列表会给您提供刚刚反转的相同元组，您可以将代码简化为：

def eps(ts_1, ts_2, tau):  
    Q_tau = 0
    q_tau = 0

    event_index1 = [index for index, item in enumerate(ts_1) if item == 0]
    n1 = len(event_index1)
    event_index2 = [index for index, item in enumerate(ts_2) if item == 0]
    n2 = len(event_index2)
    
    if (n1 != 0 and n2 != 0):
        matching_idx = set(event_index1).intersection(event_index2)
        c_ij = c_ji = 0.5 *len(matching_idx)
        
        for x,y in product(event_index1,event_index2):
            if x-y > 0 and (x-y)<= tau:
                c_ij += 1
            elif y-x > 0 and (y-x) <= tau:
                c_ji += 1                     
                  
        Q_tau = (c_ij+c_ji)/math.sqrt( n1 * n2 )
        q_tau = (c_ij - c_ji)/math.sqrt( n1 * n2 )
    
    return Q_tau, q_tau