bs4 python网页抓取答案

【问题标题】：bs4 python web scrapingbs4 python网页抓取
【发布时间】：2020-09-06 23:59:13
【问题描述】：

我只想从这个特定的div 访问文本。结构如下：

<div class="edgtf-pli-text"><h4 class="edgtf-pli-title entry-title" itemprop="name">
Crash Landing on You</h4></div>

代码是：

import requests
from bs4 import BeautifulSoup
page = requests.get('https://kdramaclicks.com/kdrama/romantic-comedy/')
soup = BeautifulSoup(page.content,'html.parser')
names = soup.find_all('div',class_='edgtf-pli-text')
print(names)

我将如何塑造代码以便只显示文本，即“Crash Landing on You？”

我对抓取真的很陌生，所以请帮帮我，如果有任何好的 api 用于抓取 wiki 表格，也推荐给我一个

【问题讨论】：

标签： python web-scraping beautifulsoup

【解决方案1】：

使用get_text() 方法提取标签内的文本。

for name in names:
    print(name.get_text(strip=True))

Crash Landing on You
Meow, The Secret Boy
Seven First Kisses
What’s Wrong with Secretary Kim
Touch Your Heart
The Secret Life of My Secretary
Strong Girl Bong-soon
Suspicious Partner
Secret Garden
She Was Pretty
Shopping King Louis
Oh My Venus
My Love from the Star
My First First Love
Legend of the Blue Sea
The Big Hit
Her Private Life
Beating Again
Emergency Couple
Clean with Passion for Now
Be Melodramatic

【讨论】：

【解决方案2】：

import requests
from bs4 import BeautifulSoup


def main(url):
    r = requests.get(url)
    soup = BeautifulSoup(r.content, 'html.parser')
    target = [item.get_text(strip=True) for item in soup.select(
        "h4.edgtf-pli-title.entry-title")]
    print(target)


main("https://kdramaclicks.com/kdrama/romantic-comedy/")

输出：

['Crash Landing on You', 'Meow, The Secret Boy', 'Seven First Kisses', 'What’sWrong with Secretary Kim', 'Touch Your Heart', 'The Secret Life of My Secretary', 'Strong Girl Bong-soon', 'Suspicious Partner', 'Secret Garden', 'She Was Pretty', 'Shopping King Louis', 'Oh My Venus', 'My Love from the Star', 'My FirstFirst Love', 'Legend of the Blue Sea', 'The Big Hit', 'Her Private Life', 'Beating Again', 'Emergency Couple', 'Clean with Passion for Now', 'Be Melodramatic']

【讨论】：

【解决方案3】：

您可以使用 BeautifulSoup 标签的.text 属性，然后使用.strip() 它（删除每个韩剧名称中前面的“\n”（换行符）。

import requests
from bs4 import BeautifulSoup


page = requests.get('https://kdramaclicks.com/kdrama/romantic-comedy/')
soup = BeautifulSoup(page.content,'html.parser')
names = soup.find_all('div',class_='edgtf-pli-text')
for name in names:
    print(name.text.strip())

【讨论】：