在python中循环以从URL获取标题标签答案

【问题标题】：Loop in python to get title tag from a URL在python中循环以从URL获取标题标签
【发布时间】：2020-10-07 15:50:56
【问题描述】：

import urllib.request as urllib2
from bs4 import BeautifulSoup

a = "https://player.vimeo.com/video/1234"

soup = BeautifulSoup(urllib2.urlopen(a))
print (a + soup.title.string)

我想通过循环获取 URL 的标题，每次获取 URL 标题时都会添加一个数字。

例如：我得到https://player.vimeo.com/video/1234 的标题，然后是https://player.vimeo.com/video/1235 等等......

【问题讨论】：

这能回答你的问题吗？ Iterating over arrays in Python 3

标签： python html loops beautifulsoup

【解决方案1】：

你可以这样做：

import urllib.request as urllib2
from bs4 import BeautifulSoup

start_idx, end_idx = 1234, 1245

for idx in range(start_idx, end_idx):
  a = f"https://player.vimeo.com/video/{idx}"
  soup = BeautifulSoup(urllib2.urlopen(a))
  print (f"for url:{a}, title: {soup.title.string}")

根据需要正确设置start_idx 和end_idx。

此外，您可能需要处理由于禁止访问某些 url 而可能出现的 HTTPError。

【讨论】：

【解决方案2】：

如果你有更多url，添加到lst。你得到了所有的标题。你可以试试下面的脚本：

import urllib.request as urllib2
from bs4 import BeautifulSoup

lst = ["https://player.vimeo.com/video/1234","https://player.vimeo.com/video/1235"]
title = []
for a in lst:    
    soup = BeautifulSoup(urllib2.urlopen(a), 'lxml')
    title.append(soup.title.string)

print(title)

输出将是：

['Diving catch from Chris Bodenner on Vimeo', 'Hit with box from Chris Bodenner on Vimeo']

或

import urllib.request as urllib2
from bs4 import BeautifulSoup

lst = ["https://player.vimeo.com/video/1234","https://player.vimeo.com/video/1235"]
title = []
for a in lst:    
    soup = BeautifulSoup(urllib2.urlopen(a), 'lxml')
    title.append(soup.title.string)
    print (a + " : " + soup.title.string)

输出将是：

https://player.vimeo.com/video/1234 : Diving catch from Chris Bodenner on Vimeo
https://player.vimeo.com/video/1235 : Hit with box from Chris Bodenner on Vimeo

【讨论】：