【问题标题】:Counting Pattern Occurence in Pandas DataFrame Column在 Pandas DataFrame 列中计数模式出现
【发布时间】:2020-04-29 21:01:27
【问题描述】:

我有一个 Pandas DataFrame,其中每一行都包含一个列表。我想知道从列表列中识别模式的最有效方法/最佳实践是什么 - 例如,接受之前的平均拒绝数。 (见下例)

    sequence_of_selection
0   Accept,Reject,Reject,Reject,Reject,Accept,Reje...
1   Accept,Reject,Reject,Reject,Reject,Reject,Reje...
2   Reject,Accept,Accept,Reject,Reject,Reject,Acce...
3   Accept,Reject,Accept,Accept,Accept,Accept,Reje...
4   Reject,Accept,Reject,Accept,Reject,Reject,Acce...

我可以将数据转换为字符串并将它们拆分或在字符串中搜索子字符串等等,但我更愿意寻求一种更有效的方法,因为 Python 字符串是不可变的。

感谢任何建议/帮助。

Jupyter screenshot

【问题讨论】:

    标签: python pandas dataframe jupyter-notebook


    【解决方案1】:

    这是一个非常好的问题!我想强调事件时间间隔的力量——基于序列的行为和可预测性有很多洞察力就像张贴的一样。考虑到这一点,我写了一个长答案,希望能解释一些数据操作的核心原则。

    1.创建一个自定义函数来执行您的计算:
    (假设您只应用于一个列表 - 我实际上建议在调试或测试时提取一个列表) p>

    def event_metrics(my_list, look_for = "Accept", exclude_zeros=True, simple=True):
        """
        Simple mode:
            Returns the average number of `items` before `look_for` 
    
        Non-Simple mode:
            Returns a dictionary with the mean, median, and max number of `items`
            before `look_for` 
    
        --
        my_list: a list of values
    
        look_for: An item in the list which constitutes the "event"
                  Example: "accept" from a list of "accept" and "reject"
    
        exclude_zeros: exclude metrics for when `look_for` occurs back to back
    
        simple: operate in simple mode or non-simple mode
    
        """
    
        # Instantiate a counter list
        my_counter = []
        n = 0
    
        # Loop through the list
        for x in my_list:
    
            # If a match, add n to the list and reset
            if x==look_for:
                my_counter.append(n)
                n=0
    
            # Otherwise, continue
            else:
                n+=1
    
        # Sometimes you might want to append the final n at conclusion of the loop
        # You could do that with the following code:
        # if x!=look_for:
        #     my_counter.append(n)
    
    
        # You may not want to include back-to-back events
        if exclude_zeros:
            my_counter = [x for x in my_counter if x>0]
    
        # You can return a specific metric such as mean
        if simple:
            return np.mean(my_counter)
    
        # Or you can pass several metrics as a dictionary and convert to a series
        my_metrics = {
            "mean":np.mean(my_counter),
            "median":np.median(my_counter),
            "max":np.max(my_counter)
        }
        return my_metrics
    

    2。将此自定义函数应用到您的 df:

    • 简单模式:返回单个值的数组 - 像新列一样对待。
    • 非简单模式: 返回一个字典数组——使用pd.to_Series 转换为多列。使用pd.merge 添加到原始df。
    # Simple Mode
    df["sequence_of_selection"].apply(event_metrics, simple=True)
    
    # Non-Simple Mode
    temp_df = df["sequence_of_selection"].apply(event_metrics, simple=False)\
                .apply(pd.Series)\  # Convert to its own df
                .add_prefix("rej_") # Add a prefix to your column names
    
    df.merge(temp_df,left_index=True,right_index=True)
    

    【讨论】:

      【解决方案2】:

      由于它们是列表,您可以获取'Accept' 的index,然后取这些索引的平均值。如果索引为 0,则列表中的第一项为'Accept',因此其前面有零个'Reject',依此类推。

      df['sequence_of_selection'].apply(lambda x: x.index('Accept')).mean()
      

      【讨论】:

        猜你喜欢
        • 2023-02-06
        • 1970-01-01
        • 2016-09-01
        • 2016-06-30
        • 1970-01-01
        • 2018-07-05
        • 1970-01-01
        • 1970-01-01
        • 2022-11-13
        相关资源
        最近更新 更多