【发布时间】:2019-12-08 23:51:29
【问题描述】:
我试图在我的代码中跳过 AttributeError,因为在抓取时总是存在属性错误,尤其是在尝试获取标题和内容的值时。我试图将“AttributeError 除外”放在某个地方,但从来没有工作过。有人可以帮助我吗?我正在使用 python 3.6
from bs4 import BeautifulSoup
import requests
import pymysql.cursors
urls2 = []
result = requests.get("http://desaku.bandungkab.go.id/desaonline/")
src = result.content
soup = BeautifulSoup(src, 'lxml')
links = soup.find_all('a')
urls = []
for link in links:
if "www" in link.text:
url = link.attrs['href']
urls.append(url)
num1=len(urls)
b=0
while b<10:
result2 = requests.get(urls[b])
src2 = result2.content
soup = BeautifulSoup(src2, 'lxml')
links2 = soup.find_all('a')
for link in links2:
if "selengkapnya" in link.text:
url2 = link.attrs['href']
urls2.append(url2)
b+=1
num=len(urls2)
i=0
while i<num:
html = requests.get(urls2[i])
src = html.content
soup = BeautifulSoup(src, 'lxml')
recordList = soup.findAll("div", {"class": "artikel", })
recordlist = soup.find_all('div', attrs={'class':'sampul2'})
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='bs4-test',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
for record in recordList:
#except AttributeError:
#continue #WHERE TO PUT THIS EXCEPTION,TO SKIP ATRRIBUTEERRROR?
title = record.find("h2", {"class": "judul", }).get_text().strip()
date = record.find('i').next_sibling.next_sibling.next_sibling.replace('\t\t\t\t\t\t\t', '')
content = record.find("div", {"class":"teks"}).get_text().strip()
image = record.img['src']
cover = record.img['src']
sql = "INSERT INTO `artikel` (`jdl`, `tgl`, `kon`, `gambar`, `sampul`) VALUES (%s, %s, %s, %s, %s)"
cursor.execute(sql, (title, date, content, image, cover))
connection.commit()
print ("Record inserted successfully into table")
finally:
connection.close()
print("MySQL connection is closed")
i+=1
【问题讨论】:
-
什么是result2?它突然出现。
-
对不起,我错过了一行,我不小心删除了它。我已经编辑了代码
标签: python-3.x exception beautifulsoup