【问题标题】:How to add exception to skip AttributeError in my code?如何在我的代码中添加异常以跳过 AttributeError?
【发布时间】:2019-12-08 23:51:29
【问题描述】:

我试图在我的代码中跳过 AttributeError,因为在抓取时总是存在属性错误,尤其是在尝试获取标题和内容的值时。我试图将“AttributeError 除外”放在某个地方,但从来没有工作过。有人可以帮助我吗?我正在使用 python 3.6

from bs4 import BeautifulSoup
import requests
import pymysql.cursors
urls2 = []
result = requests.get("http://desaku.bandungkab.go.id/desaonline/")
src = result.content
soup = BeautifulSoup(src, 'lxml')
links = soup.find_all('a')
urls = []
for link in links:
    if "www" in link.text:
        url = link.attrs['href']
        urls.append(url)

num1=len(urls)
b=0
while b<10: 
    result2 = requests.get(urls[b])
    src2 = result2.content
    soup = BeautifulSoup(src2, 'lxml')
    links2 = soup.find_all('a')
    for link in links2:
        if "selengkapnya" in link.text:
            url2 = link.attrs['href']
            urls2.append(url2)
    b+=1

num=len(urls2)
i=0
while i<num:
    html = requests.get(urls2[i])
    src = html.content
    soup = BeautifulSoup(src, 'lxml')
    recordList = soup.findAll("div", {"class": "artikel", })
    recordlist = soup.find_all('div', attrs={'class':'sampul2'})

    connection = pymysql.connect(host='localhost',
                                 user='root',
                                 password='',
                                 db='bs4-test',
                                 charset='utf8mb4',
                                cursorclass=pymysql.cursors.DictCursor)

    try:
        with connection.cursor() as cursor:
            for record in recordList:
    #except AttributeError:
                #continue #WHERE TO PUT THIS EXCEPTION,TO SKIP ATRRIBUTEERRROR?
                title = record.find("h2", {"class": "judul",      }).get_text().strip()
                date = record.find('i').next_sibling.next_sibling.next_sibling.replace('\t\t\t\t\t\t\t', '')
                content = record.find("div", {"class":"teks"}).get_text().strip()
                image = record.img['src']
                cover = record.img['src']
                sql = "INSERT INTO `artikel` (`jdl`, `tgl`, `kon`, `gambar`, `sampul`) VALUES (%s, %s, %s, %s, %s)"
                cursor.execute(sql, (title, date, content, image, cover))
        connection.commit()
        print ("Record inserted successfully into table")
    finally:
        connection.close()
        print("MySQL connection is closed")
    i+=1

【问题讨论】:

  • 什么是result2?它突然出现。
  • 对不起,我错过了一行,我不小心删除了它。我已经编辑了代码

标签: python-3.x exception beautifulsoup


【解决方案1】:

只是一个例子:如果是 url,则附加 url,如果不是,则附加 null。通常你会希望所有列表的长度相同,这样最后你就可以把它们放在dataframe中。

import numpy as np
links = soup.find_all('a')
urls = []
for link in links:
    try:
        url = link['href']
        urls.append(url)
    except:
        urls.append(np.nan)

【讨论】:

    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 2017-04-26
    • 1970-01-01
    • 1970-01-01
    • 2017-01-15
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多