python爬虫4猫眼电影的Top100

1 查看网页结构

(1)确定需要抓取的字段

电影名称

电影主演

电影上映时间

电影评分

(2) 分析页面结构

按住f12------->点击右上角(如下图2)---->鼠标点击需要观察的字段

python爬虫4猫眼电影的Top100

(3)BeautifulSoup解析源代码并设置过滤属性

1 soup = BeautifulSoup(htmll, 'lxml')

2 Movie_name = soup.find_all('div',class_='movie-item-info')

3 Movie_Score1=soup.find_all('p',class_='score')

(4)调试查看过滤属性是否正确

(5)提取对应字段

 1  for cate,score in zip(Movie_name,Movie_Score1):
 2         data={}
 3         movie_name1 = cate.find('a').text.strip('\n')
 4         data['title']=movie_name1
 5         movie_actor = cate.find_all("p")[1].text.replace("\n"," ").strip()
 6         data['actors']=movie_actor
 7         movie_time=cate.find_all("p")[2].text.strip('\n').strip()
 8         data['data']=movie_time
 9         movie_score1=score.find_all("i")[0].string
10         movie_score2=score.find_all("i")[1].string
11         movie_score=movie_score1+movie_score2
12         data['score'] = movie_score
13         name = movie_name1 + "\t"+movie_actor+"\t" + movie_time+"\t"+movie_score
14         DATA.append(name)
15         with open('Movie1.txt', 'a+') as f:
16             f.write("\n{}".format(name))

View Code