Webscraping：如何处理返回的数据框中的 NA 值？答案

【问题标题】：Webscraping: How to deal with NA values in a dataframe returned?Webscraping：如何处理返回的数据框中的 NA 值？
【发布时间】：2021-05-29 09:41:19
【问题描述】：

我正在抓取 www.oddsportal.com 并收到 object has no attribute 错误。

一些匹配没有得分，因此不返回值。因此我知道错误是因为这个。

我的代码在没有可用分数之前一直有效并返回错误

import pandas as pd
from selenium import webdriver
from bs4 import BeautifulSoup as bs

browser = webdriver.Chrome()

class GameData:
    def __init__(self):
        self.date = []
        self.time = []
        self.country = []
        self.league = []
        self.game = []
        self.home_odds = []
        self.draw_odds = []
        self.away_odds = []

    def append(self, score):
        pass


def get_urls(browser, landing_page):
    browser.get(landing_page)
    urls = [i.get_attribute('href') for i in
            browser.find_elements_by_css_selector(
                '.next-games-date > a:nth-child(1), .next-games-date > a:nth-child(n+3)')]

    return urls


def parse_data(html):
    df = pd.read_html(html, header=0)[0]
    html = browser.page_source
    soup = bs(html, "lxml")
    cont = soup.find('div', {'id': 'wrap'})
    content = cont.find('div', {'id': 'col-content'})
    content = content.find('table', {'class': 'table-main'}, {'id': 'table-matches'})
    main = content.find('th', {'class': 'first2 tl'})

    if main is None:
        return None

    count = main.findAll('a')
    country = count[0].text
    game_data = GameData()

    for row in df.itertuples():
        if not isinstance(row[1], str):
            continue
        elif ':' not in row[1]:
            country = row[1].split('»')[0]
            continue
        game_time = row[1]
        game_date = row[1].split('-')[0]
        score = row[3] #The error happens here. How do I construct 'if NA then NaN?'

        game_data.date.append(game_date)
        game_data.time.append(game_time)
        game_data.country.append(country)
        game_data.league.append(count[1].text)
        game_data.game.append(row[2])
        game_data.score.append(score) #This should be score if available else NaN
        game_data.home_odds.append(row[4])
        game_data.draw_odds.append(row[5])
        game_data.away_odds.append(row[6])



    return game_data


if __name__ == '__main__':

    start_url = "https://www.oddsportal.com/matches/soccer/"
    urls = []
    browser = webdriver.Chrome()
    results = None
    urls = get_urls(browser, start_url)
    urls.insert(0, start_url)

    for number, url in enumerate(urls):
        if number > 0:
            browser.get(url)
        html = browser.page_source
        game_data = parse_data(html)

        if game_data is None:
            continue

        result = pd.DataFrame(game_data.__dict__)

        if results is None:
            results = result
        else:
            results = results.append(result, ignore_index=True)

错误：

Traceback (most recent call last):
  File "C:/Users/harsh/AppData/Roaming/JetBrains/PyCharmCE2021.1/scratches/scratch_16.py", line 98, in <module>
    game_data = parse_data(html)
  File "C:/Users/harsh/AppData/Roaming/JetBrains/PyCharmCE2021.1/scratches/scratch_16.py", line 75, in parse_data
    game_data.score.append(score)
AttributeError: 'GameData' object has no attribute 'score'

如何在此处包含“If NA then NaN else get score”参数？

【问题讨论】：

标签： python pandas selenium selenium-webdriver web-scraping

【解决方案1】：

您可以使用hasattr 函数检查对象是否具有属性。它需要两个参数，第一个是 object 本身，第二个是您要查找的 attribute。在你的情况下，它会是这样的：

if hasattr(gamedata, 'score'): 
    gamedata.score.append(score)

问题

您的代码有很多问题。导致此错误的原因是您没有在GameData 类的初始化方法中定义 score 属性。

【讨论】：