如何在python中将值转换为嵌套的JSON？

Question

我使用 beautifulsoup 抓取某些值。我想将其转换为嵌套 JSON 格式。 以下是我的价值结构。

category = development
heading =  Complete Python Bootcamp | Deep Learning Into Python Coding
image = https://i.udemycdn.com/course/750x422/3871500_3d01_3.jpg
link = https://www.udemy.com/course/complete-python-bootcamp-deep-learning-into-python-coding

category = development
heading = C++ Complete Course For Beginners
image = https://i.udemycdn.com/course/750x422/3847698_8547_2.jpg
link = https://www.udemy.com/course/c-complete-course-for-beginners/?couponCode=FREE2021

category = it-software
heading = TB0-116 TIBCO Enterprise Message Service 6 Practice Exam
image = https://i.udemycdn.com/course/750x422/2931054_d555.jpg
link = https://www.udemy.com/course/tb0-116-tibco-enterprise-message-service-6-practice-exam-t

预期的 json 输出：

[
   {
      "development":[
         {
            "heading":" Complete Python Bootcamp | Deep Learning Into Python Coding",
            "image":"https://i.udemycdn.com/course/750x422/3871500_3d01_3.jpg",
            "courselink":"https://www.udemy.com/course/complete-python-bootcamp-deep-learning"
         }
         {
            "heading":"C++ Complete Course For Beginners",
            "image":"https://i.udemycdn.com/course/750x422/3871500_3d01_3.jpg",
            "courselink":"https://www.udemy.com/course/complete-python-bootcamp-deep-learning"
         }
      ],
     "it-software":[
        {
         "heading" : "TB0-116 TIBCO Enterprise Message Service 6 Practice Exam",
         "image" : "https://i.udemycdn.com/course/750x422/2931054_d555.jpg"
         "courselink" : "https://www.udemy.com/course/tb0-116-tibco-enterprise-message-service"
        }
      ],
]

下面我附上了我的抓取代码

def scrapeData(category):
    
    base_url = "https://udemycoupon.learnviral.com/coupon-category/"+category+"/"
    print(base_url)
    source=requests.get(base_url,headers=headers).text

    soup = BeautifulSoup(source,'lxml')
    contents = soup.find_all('div',class_="item-holder")
    print()
    # print(contents)
    for item in contents:
        print(category)
        heading=item.find("h3",{"class":"entry-title"}).text.replace("[Free]","")
        print(heading)
        image=item.find("div",{"class":"store-image"}).find("img")['src']
        imagelink = image.replace('240x135', '750x422')
        print(imagelink)
        courselink = item.find("a", {"class":"coupon-code-link btn promotion"})

任何人都可以帮我将它转换成我在 python 中的预期格式。在此先感谢。

Answer 1

def scrape_category(name):
    base_url = 'https://udemycoupon.learnviral.com/coupon-category/' + name + '/'
    source = requests.get(base_url).text
    soup = BeautifulSoup(source, 'lxml')
    contents = soup.find_all('div', class_='item-holder')
    courses = []
    for item in contents:
        heading = item.find('h3', {'class': 'entry-title'}).text.replace('[Free]', '')
        image = item.find('div', {'class': 'store-image'}).find('img')['src']
        course_link = item.find('a', {'class': 'coupon-code-link btn promotion'})
        courses.append({
            'heading': heading,
            'image': image.replace('240x135', '750x422'),
            'courselink': course_link['href'],
        })

    return courses


result = {}
for category in ('development', 'it-software', ):
    result[category] = scrape_category(category)

print(result)  # or print([result])

Answer 2

您可以使用defaultdict 并更新抓取代码为每个新课程创建字典对象：

from collections import defaultdict
main_d = defaultdict(list)
for item in contents:
    print(category)
    heading=item.find("h3",{"class":"entry-title"}).text.replace("[Free]","")
    print(heading)
    image=item.find("div",{"class":"store-image"}).find("img")['src']
    imagelink = image.replace('240x135', '750x422')
    print(imagelink)
    courselink = item.find("a", {"class":"coupon-code-link btn promotion"})
    
    d = {"heading": heading, "image": image, "courselink": courselink}
    main_d[category].append(d)

main_d 将是一个具有以下结构的字典对象：

{
      "development":[
         {
            "heading":" Complete Python Bootcamp | Deep Learning Into Python Coding",
            "image":"https://i.udemycdn.com/course/750x422/3871500_3d01_3.jpg",
            "courselink":"https://www.udemy.com/course/complete-python-bootcamp-deep-learning"
         }
         {
            "heading":"C++ Complete Course For Beginners",
            "image":"https://i.udemycdn.com/course/750x422/3871500_3d01_3.jpg",
            "courselink":"https://www.udemy.com/course/complete-python-bootcamp-deep-learning"
         }
      ],
     "it-software":[
        {
         "heading" : "TB0-116 TIBCO Enterprise Message Service 6 Practice Exam",
         "image" : "https://i.udemycdn.com/course/750x422/2931054_d555.jpg"
         "courselink" : "https://www.udemy.com/course/tb0-116-tibco-enterprise-message-service"
        }
      ],
}

注意：这不是经过测试的代码，可能需要进行一些修改才能使其正常工作。