处理具有不同分页结构的链接时遇到问题

Question

我在 python 中编写了一个脚本来抓取位于其着陆页地图旁边右侧区域的不同项目的标题。我在脚本中使用了两个链接：一个有分页，另一个没有。

当我执行我的脚本时，它首先检查分页链接。如果找到，则将链接传递给get_paginated_info() 函数以在那里打印结果。但是，如果它找不到分页链接，则它将汤对象传递给get_info() 函数并在那里打印结果。此时脚本的工作方式与我描述的完全一样。

我怎样才能让我的脚本在get_info()函数中打印结果，无论链接是否有分页或不符合我已经尝试应用的逻辑，因为我希望踢出@987654324我的脚本中的@函数？

这是我迄今为止的尝试：

import requests 
from bs4 import BeautifulSoup
from urllib.parse import urljoin

urls = (
    'https://www.mobilehome.net/mobile-home-park-directory/maine/all',
    'https://www.mobilehome.net/mobile-home-park-directory/rhode-island/all',
)

def get_names(link):
    r = requests.get(link)
    soup = BeautifulSoup(r.text,"lxml")
    items = soup.select_one(".pagination a.next_page")
    if items:
        npagelink = items.find_previous_sibling().get("href").split("/")[-1]
        return [get_paginated_info(link + "/page/{}".format(page)) for page  in range(1,int(npagelink)+1)]

    else:
        return [get_info(soup)]

def get_info(soup):
    print("================links without pagination==============")
    for items in soup.select("td[class='table-row-price']"):
        item = items.select_one("h2 a").text
        print(item)

def get_paginated_info(url):
    r = requests.get(url)
    sauce = BeautifulSoup(r.text,"lxml")
    print("================links with pagination==============")
    for content in sauce.select("td[class='table-row-price']"):
        title = content.select_one("h2 a").text
        print(title)

if __name__ == '__main__':
    for url in urls:
        get_names(url)

任何能够处理不同点赞的更好设计都将受到高度赞赏。

Answer 1

我稍微改变了逻辑。所以现在无论是分页存在还是没有分页脚本都会调用get_names。但是在for循环的第二种情况下，只会执行一次迭代

import requests 
from bs4 import BeautifulSoup
from urllib.parse import urljoin

urls = (
    'https://www.mobilehome.net/mobile-home-park-directory/maine/all',
    'https://www.mobilehome.net/mobile-home-park-directory/rhode-island/all',
)

def get_names(link):
    r = requests.get(link)
    soup = BeautifulSoup(r.text,"lxml")
    items = soup.select_one(".pagination a.next_page")
    try:
        npagelink = items.find_previous_sibling().get("href").split("/")[-1]
    except AttributeError:
        npagelink = 1
    return [get_info(link + "/page/{}".format(page)) for page in range(1, int(npagelink) + 1)]


def get_info(url):
    r = requests.get(url)
    sauce = BeautifulSoup(r.text,"lxml")
    for content in sauce.select("td[class='table-row-price']"):
        title = content.select_one("h2 a").text
        print(title)

if __name__ == '__main__':
    for url in urls:
        get_names(url)

请仔细检查输出以确保一切正常