【问题标题】:delete row from dataframe, df.drop not removing rows从数据框中删除行,df.drop 不删除行
【发布时间】:2021-05-01 22:52:19
【问题描述】:

我正在制作一个表格,其中记录了类似产品的索引,然后将这些行中的数据粘贴到单个行中,该行汇总了所有数据。在此之后,该行被删除。代码如下:

matchedproducts_df = pd.read_sql_query("SELECT * from matchedproducts", conn)

print(len(matchedproducts_df.index))

def mergeduplicates(df, similarity_field='', databasetable='', similar_level=85):
    
    print(len(df.index))

    def check_simi(d):
        global dupl_indexes, dupl_originals
        dupl_originals = []
        dupl_indexes = []
        for i in range(len(d.values) - 1):
            for j in range(i + 1, len(d.values)):
                if fuzz.token_sort_ratio(d.values[i], d.values[j]) >= similar_level:
                    dupl_indexes.append(d.index[j])
                    dupl_originals.append(d.index[i])

       

    indexes = df.groupby([True]*len(df))[similarity_field].apply(check_simi)

    a = 0
    for i in dupl_indexes:
        if df.iloc[i, 5] == 'harveynichols':
            df.at[dupl_originals[a], 'pricehn'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilityhn'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storehn'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinkhn'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'houseoffraser':
            df.at[dupl_originals[a], 'pricehof'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilityhof'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storehof'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinkhof'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'selfridges':
            df.at[dupl_originals[a], 'pricesf'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilitysf'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storesf'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinksf'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'lookfantastic':
            df.at[dupl_originals[a], 'pricelf'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilitylf'] =  df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storelf'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinklf'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'superdrug':
            df.at[dupl_originals[a], 'pricesd'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilitysd'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storesd'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinksd'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'boots':
            df.at[dupl_originals[a], 'priceboots'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilityboots'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storeboots'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinkboots'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'allbeauty':
            df.at[dupl_originals[a], 'priceab'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilityab'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storeab'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinkab'] = df.loc[i, 'hyperlink']
            df.drop([i])
        elif df.iloc[i, 5] == 'asos':
            df.at[dupl_originals[a], 'pricea'] = df.loc[i, 'price']
            df.at[dupl_originals[a], 'availabilitya'] = df.loc[i, 'availability']
            df.at[dupl_originals[a], 'storea'] = df.loc[i, 'store']
            df.at[dupl_originals[a], 'hyperlinka'] = df.loc[i, 'hyperlink']
            df.drop([i])

        # for index_list in indexes:
        #     df.drop(index_list, inplace=True)

        a += 1

    print(len(df.index))

    df.to_csv('C:/Users/Judoo/Desktop/matchedproducts.csv')

    df.to_sql(databasetable, conn, if_exists="replace")
    
    conn.commit()

    conn.close()

mergeduplicates(matchedproducts_df, similarity_field='name', databasetable='matchedproducts')

该代码适用于查找重复项并将其数据复制到相关的原始行中,但不会在检查后删除/删除该行。脚本前后的表格长度相同。我不确定为什么会发生这种情况,任何帮助将不胜感激。

【问题讨论】:

  • 我认为你需要通过inplace=True,试试pls.df.drop([i],inplace=True)
  • @simpleApp inplace=True 不建议使用,我相信在未来的熊猫版本中会被弃用。但您本质上是正确的,只需将您的 df.drop 重新分配给您的数据框 df = df.drop(..)
  • 我已经实现了这个,但它现在给出了一个``` KeyError: 460```。有什么原因会发生这种情况吗?
  • 在这一行专门df.at[dupl_originals[a], 'priceboots'] = df.loc[i, 'price']

标签: python pandas dataframe fuzzywuzzy


【解决方案1】:

df.drop() 方法通常会删除您想要删除特定行的行。你有没有试过加inplace=True?因为它可能会保留旧索引。或者,您可以使用 df = df.drop([i]) 之类的新 df 来做同样的事情。 See here

如果这些不能解决您的问题,您必须使用 df.drop([i]) 的结构。

【讨论】:

  • 这不是您在这里真正回答的方式。使用他们的代码,然后进行相应的修改,然后向他们解释这里的主要问题是什么。
猜你喜欢
  • 2022-10-07
  • 1970-01-01
  • 1970-01-01
  • 2016-11-23
  • 1970-01-01
  • 2011-12-16
相关资源
最近更新 更多