Pandas Dataframe 迭代循环无限期加载

Question

我有以下数据框，其中包含 UserId 和该人在 Stackoverflow 上获得的徽章的 Name。现在，每个徽章都属于一个特定的类别，例如 Question、Answer、Participation、Moderation 和 Tag。我想创建一个名为Category 的列来存储每个徽章的类别。

如果数据少于 100 万用户，我编写的代码运行良好，对于更多数据，它只会不断加载。如何解决这个问题？

数据框（徽章）

UserId | Name
  1    | Altruist
  2    | Autobiographer
  3    | Enlightened
  4    | Citizen Patrol
  5    | python

代码

def category(df):  
  
  questionCategory = ['Altruist', 'Benefactor', 'Curious', 'Inquisitive', 'Socratic', 'Favorite Question', 'Stellar Question', 'Investor', 'Nice Question', 'Good Question', 'Great Question', 'Popular Question', 'Notable Question', 'Famous Question', 'Promoter', 'Scholar', 'Student']
  
  answerCategory = ['Enlightened', 'Explainer', 'Refiner', 'Illuminator', 'Generalist', 'Guru', 'Lifejacket', 'Lifeboat', 'Nice Answer', 'Good Answer', 'Great Answer', 'Populist', 'Revival', 'Necromancer', 'Self-Learner','Teacher', 'Tenacious', 'Unsung Hero']
  
  participationCategory = ['Autobiographer','Caucus', 'Constituent', 'Commentator', 'Pundit', 'Enthusiast', 'Fanatic', 'Mortarboard', 'Epic', 'Legendary', 'Precognitive', 'Beta', 'Quorum', 'Convention', 'Talkative', 'Outspoken', 'Yearling']
  
  moderationCategory = ['Citizen Patrol', 'Deputy', 'Marshal', 'Civic Duty', 'Cleanup', 'Constable', 'Sheriff', 'Critic', 'Custodian', 'Reviewer', 'Steward', 'Disciplined', 'Editor', 'Strunk & White', 'Copy Editor', 'Electorate', 'Excavator', 'Archaelogist', 'Organizer', 'Peer Pressure', 'Proofreader', 'Sportsmanship', 'Suffrage', 'Supporter', 'Synonymizer', 'Tag Editor', 'Research Assistant', 'Taxonomist', 'Vox Populi']

  #Tag Category will be represented as 0
  df['Category'] = 0

  for i in range(len(df)) : 
    if (df.loc[i, "Name"] in questionCategory):
      df.loc[i, 'Category'] = 1 

    elif (df.loc[i, "Name"] in answerCategory):
      df.loc[i, 'Category'] = 2 

    elif (df.loc[i, "Name"] in participationCategory):
      df.loc[i, 'Category'] = 3 

    elif (df.loc[i, "Name"] in moderationCategory):
      df.loc[i, 'Category'] = 4 

  return df   

category(stackoverflow_badges)

预期输出

UserId | Name           | Category
  1    | Altruist       |    1
  2    | Autobiographer |    3
  3    | Enlightened    |    2
  4    | Citizen Patrol |    4
  5    | python         |    0

Answer 1

如果您想更新超过 1M 行的数据框，那么您肯定希望尽可能避免 for 循环。更新您的 'Category' 列更容易，就像在 here 完成一样。

在您的情况下，您只需将带有徽章名称的 4 个列表转换为将徽章名称与其数字类别匹配的字典，例如：

category_dict = {
    **{key: 1 for key in questionCategory},
    **{key: 2 for key in answerCategory},
    **{key: 3 for key in participationCategory},
    **{key: 4 for key in moderationCategory},
}

然后你可以用这个命令替换你所有的for循环：

df['Category'] = df['Name'].map(category_dict).fillna(0)

这可能无法解决您的全部问题，但至少可以节省一些时间。