BeautifulSoup 找不到 <h3> 标签

Question

我想使用 BeautfiulSoup 库在 reddit 页面中收集最受好评的文本，但每当我尝试运行代码时，它都找不到 h3 标签并返回一个空数组。我该如何解决？

import requests as re
from bs4 import BeautifulSoup

r=re.get("https://www.reddit.com/r/funfacts/")

soup=BeautifulSoup(r.content,"html.parser")

gelen=soup.find_all("h3")

print(gelen)

Answer 1

如果您将.json 添加到 Reddit URL，您将获得 Json 格式的所有数据。

例如：

import json
import requests


url = 'https://www.reddit.com/r/funfacts.json'  # <-- add .json here!
headers = {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0'}

data = requests.get(url, headers=headers).json()

# uncomment this to see all data:
# print(json.dumps(data, indent=4))
    
# print some data to screen:
print('{:<8} {:<8} {}'.format('UPS', 'DOWNS', 'TITLE'))
for c in data['data']['children']:
    print('{:<8} {:<8} {}'.format(c['data']['ups'], c['data']['downs'], c['data']['title']))

打印：

UPS      DOWNS    TITLE
38       0        People! Please remember to begin all posts with "Fun Fact:".
276      0        Fun Fact: Marathons are a tradition that originated from the ancient greeks where someone ran 26 miles and died.
165      0        Fun Fact: Andean condor in flight flaps its wings for just 1 % of its flight time.
347      0        fun fact i found on youtube..
0        0        Fun fact: planes only crash once
4        0        Fun fact dump
24       0        FUN FACT. DID YOU KNOW? An airplane mechanic invented Slinky while he was playing with engine parts and realized the possible secondary use for the springs. Seems like most invention occurred by just playing around sometimes.
119      0        Fun Fact: Walking the perimeter of this city = Walking from Philadelphia to Denver
0        0        Here's a funny fun fact!
3        0        Fun Fact: The Pygmy Marmoset is the World's Smallest Monkey
7        0        Fun Fact: Actor Christopher Lee was related to a king, a Confederate general, and the creator of James Bond, and was a soldier in WWII
111      0        Fun Fact: This is probably the most famous image on youtube. In just three days; over 4000 users had this image as their profile picture. The image appears to be a blood-covered creature thing.
5        0        Fun Fact: Walt Disney originally wanted to call his most famous creation Mortimer, but his wife convinced him to change the name to Mickey.
3        0        Fun fact: Sweden has the most islands of any country in the world, sitting on at least 220,000 islands. For comparison, The Phillipines has 7000 and Indonesia 17000 islands.
19       0        Fun Fact: Almost all humans have Neanderthal DNA.
197      0        Fun Fact: "The Anarchist Cookbook" is Riddled With Errors, and Publishers Won't Pull It, Even Though The Author Has Tried To Get It Off Shelves For Years (cross post from /r/Trivia)
0        0        Fun Fact: 23 Most Interesting Facts That You Have Never Heard Before
3        0        Fun Fact: Emily from Hannah Montanna and Cole from the sixth sense are siblings.
120      0        Fun Fact did you know that NBA player Lamarcus Aldridge is the cousin of NBA sideline reporter David Aldridge?
6        0        fun fact
12       0        Fun Fact: If you travel 69.169 miles in a direction, then you have traveled 1 degree of the entire earth's circumference.
113      0        Fun Fact: Rarest And Strangest Sharks Species Hidden In The Ocean
372      0        Putting scallions in water will grow more scallions
2        0        Fun Fact about the Ocean! Top 10
474      0        Wtf fun fact
4        0        Fun fact: Watch Dogs NPCs almost do not exist.