【发布时间】:2017-02-03 19:05:03
【问题描述】:
在一些在线 tuts (Bucky) 的帮助下,我设法编写了一个简单的网络爬虫,它只检查网页上是否有一些文本。然而,我想做的是让代码每小时运行一次。我假设我还需要托管代码,所以它会这样做吗?我做了一些研究,但似乎无法找到每小时运行它的正确方法。 这是我目前得到的代码:
import requests
from bs4 import BeautifulSoup
def odeon_spider(max_pages):
page = 1
while page <= max_pages:
url = "http://www.odeon.co.uk/films/rogue_one_a_star_wars_story/16038/" + str(page) #stores url in variable
source_code = requests.get(url) #gets url and sets it as source_code variable
plain_text = source_code.text #stores plain text in plain_text variable
soup = BeautifulSoup(plain_text, "lxml") #create beautifulsoup object
div_content = soup.findAll("div", {"class": "textComponent"}) #finds all divs with specific class
for x in div_content:
find_para = str(x.find('p').text) #finds all paragraphs and stores them in variable
text_to_search = "Register to be notified" #set text to search to variable
if text_to_search in find_para: #checks if text is in find_para
print("No tickets")
else:
print("Tickets")
page += 1
odeon_spider(1)
谢谢!
【问题讨论】:
-
Linux 机器使用
crontab -
在 Linux 上使用 cronjobs,在 windows 上你可以使用 taskscheduler
标签: python function web scraper