【问题标题】:Can you automate adding products from an online store to WooCommerce?您可以自动将产品从在线商店添加到 WooCommerce 吗?
【发布时间】:2019-11-29 09:50:05
【问题描述】:

我想从Dutch webshop(类似于 eBay/Amazon)的帐户中获取所有产品,并使用 WooCommerce 将它们添加到this WordPress webshop。大约 2 到 3 周前,我开始了 Web 开发,我了解 HTML、CSS、JavaScript、Nodejs 和 Express 的基础知识。我想我大概知道该怎么做了,那就是:

  • 遍历每页的所有产品。
  • 获取标题、描述、类别、价格和照片。
  • 将该信息与产品对象一起存储在数组中。
  • 获取对 WooCommerce API 的访问权限。
  • 遍历所有产品并将它们添加到 WooCommerce。

我的问题是

  1. 这可能吗?
  2. 我可以使用可用的语言吗?
  3. 您会使用哪些方法? (例如,您将如何抓取 HTML,是否有比我描述的步骤更简单的方法,您会使用代码还是使用一些自动化软件等)

这对我来说是一个大项目,所以欢迎任何帮助(关于如何开始)!

【问题讨论】:

    标签: javascript node.js web-scraping woocommerce automation


    【解决方案1】:

    您对这些步骤是正确的,是的,这是可能的。您可以使用 node.js 抓取数据,如您所知,我个人的偏好是 python 在数据抓取方面,但您可以在 node.js 中完成。 Node.js 有 HTML 解析器等等。我会建议你一些事情:

    • 使用解析器解析 HTML 数据,以便更好地访问元素以获取数据。
    • 使用某种数据结构来正确存储数据,例如:JSON、XML、CSV...
    • 如果获取数据是一个漫长的过程,请先获取数据,因为如果解析系统中的任何部分不适合,您可能会在解析时丢失所有数据,然后再解析数据。

    我将把我写的从你放的网站上获取数据的代码带到这里,它是用 python 语言编写的,但我在上面放了 cmets,这样你就可以更好地了解如何获取数据并用其他语言编写.您也可以使用split 从 HTML 数据中剪切部分,您甚至不需要使用解析器。

    示例:

    import requests, json
    from bs4 import BeautifulSoup
    from pprint import pprint
    
    endpoint = "http://johndevisser.marktplaza.nl/?p=1"
    
    # Send a get request to page to get the html.
    data = requests.get(endpoint).content
    
    # Parse the html via BeautifulSoup
    page = BeautifulSoup(data)
    
    # Find 'div' elements whose 'itemscope' attributes are 'itemscope'
    products = page.find_all("div", {"itemscope": "itemscope"})[1:]
    
    # Create an empty array to store prepared data.
    finalProductList = []
    
    # Iterate over the products.
    for i in products:
        # Create a dictionary object to store data properly.
        productData = {}
        # Get the title attribute from 'a' element on the current product.
        productData["title"] = i.find("a").get("title")
        # Get the href attribute from 'a' element on the current product because the real source can be useful in the future.
        productData["origin"] = i.find("a").get("href")
        # Get the image url from 'img' elements to download images.
        productData["imageURL"] = i.find("img").get("src")
        # This may look you complicated but it just finds 'span' elements value of 'class' attribute is 'subtext' and get the
        # inner text, split into two from ' '(space) to this ['€', '15,00'] and get the right part which is the second part
        # in the array which is the price and replace comma with dot to parse in float value.
        productData["price"] = float(i.find("span", {"class": "subtext"}).get_text().split(u"\xa0")[1].replace(",", "."))
        # Append the data to final data array.
        finalProductList.append(productData)
    
    # Get json representation of dictionary.
    print(json.dumps(finalProductList))
    

    输出:

    [
      {
        "title": "Sieb Posthuma  -  Mannetje Jas  (Hardcover/Gebonden) Kinderjury",
        "origin": "http://www.marktplaza.nl/boeken/kinderboeken/sieb-posthuma-mannetje-jas-hardcover-gebonden-kinderjury-92409632.html",
        "imageURL": "http://www.marktplaza.nl/M92409632/1/sieb-posthuma-mannetje-jas-hardcover-gebonden-kinderjury-92409632.jpg",
        "price": 12.5
      },
      {
        "title": "Estefhan Meijer  -  United Wraps    Wraps Uit De Hele Wereld",
        "origin": "http://www.marktplaza.nl/boeken/kookboeken/estefhan-meijer-united-wraps-wraps-uit-de-hele-wereld-92390218.html",
        "imageURL": "http://www.marktplaza.nl/M92390218/1/estefhan-meijer-united-wraps-wraps-uit-de-hele-wereld-92390218.jpg",
        "price": 15
      },
      {
        "title": "Daphne Deckers  -  De Verschrikkelijke Ijstaart  (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/kookboeken/daphne-deckers-de-verschrikkelijke-ijstaart-hardcover-gebonden-92390182.html",
        "imageURL": "http://www.marktplaza.nl/M92390182/1/daphne-deckers-de-verschrikkelijke-ijstaart-hardcover-gebonden-92390182.jpg",
        "price": 10
      },
      {
        "title": "Adelene Fletcher  -   Bomen Aquarelleren Van A Tot Z",
        "origin": "http://www.marktplaza.nl/boeken/hobby-techniek/adelene-fletcher-bomen-aquarelleren-van-a-tot-z-92390124.html",
        "imageURL": "http://www.marktplaza.nl/M92390124/1/adelene-fletcher-bomen-aquarelleren-van-a-tot-z-92390124.jpg",
        "price": 12.5
      },
      {
        "title": "Razorlight ‎– America  (2 Track CDSingle)",
        "origin": "http://www.marktplaza.nl/cd-vinyl/singles/razorlight-america-2-track-cdsingle-92390118.html",
        "imageURL": "http://www.marktplaza.nl/M92390118/1/razorlight-america-2-track-cdsingle-92390118.jpg",
        "price": 5
      },
      {
        "title": "Twarres ‎– Children  (2 Track CDSingle)",
        "origin": "http://www.marktplaza.nl/cd-vinyl/singles/twarres-children-2-track-cdsingle-92390078.html",
        "imageURL": "http://www.marktplaza.nl/M92390078/1/twarres-children-2-track-cdsingle-92390078.jpg",
        "price": 5
      },
      {
        "title": "Tower Of Power ‎– The Very Best Of Tower Of Power - The Warner Years  (CD)",
        "origin": "http://www.marktplaza.nl/cd-vinyl/pop/tower-of-power-the-very-best-of-tower-of-power-the-warner-years-cd-92389836.html",
        "imageURL": "http://www.marktplaza.nl/M92389836/1/tower-of-power-the-very-best-of-tower-of-power-the-warner-years-cd-92389836.jpg",
        "price": 10
      },
      {
        "title": "Red Hot Chili Peppers ‎– Dani California  (2 Track CDSingle)",
        "origin": "http://www.marktplaza.nl/cd-vinyl/singles/red-hot-chili-peppers-dani-california-2-track-cdsingle-92389742.html",
        "imageURL": "http://www.marktplaza.nl/M92389742/1/red-hot-chili-peppers-dani-california-2-track-cdsingle-92389742.jpg",
        "price": 5
      },
      {
        "title": "Seth Godin  -  Icarus Deception  (Engelstalig)",
        "origin": "http://www.marktplaza.nl/boeken/management-en-economie/seth-godin-icarus-deception-engelstalig-92389542.html",
        "imageURL": "http://www.marktplaza.nl/M92389542/1/seth-godin-icarus-deception-engelstalig-92389542.jpg",
        "price": 12.5
      },
      {
        "title": "Rob Gifford  -  De Chinese Weg",
        "origin": "http://www.marktplaza.nl/boeken/reizen/rob-gifford-de-chinese-weg-92389500.html",
        "imageURL": "http://www.marktplaza.nl/M92389500/1/rob-gifford-de-chinese-weg-92389500.jpg",
        "price": 12.5
      },
      {
        "title": "Bart Leeuwenburgh  -   Darwin In Domineesland",
        "origin": "http://www.marktplaza.nl/boeken/informatief/bart-leeuwenburgh-darwin-in-domineesland-92386128.html",
        "imageURL": "http://www.marktplaza.nl/M92386128/1/bart-leeuwenburgh-darwin-in-domineesland-92386128.jpg",
        "price": 12.5
      },
      {
        "title": "Per Olov Enquist  -  Het Record  (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/romans/per-olov-enquist-het-record-hardcover-gebonden-92386080.html",
        "imageURL": "http://www.marktplaza.nl/M92386080/1/per-olov-enquist-het-record-hardcover-gebonden-92386080.jpg",
        "price": 10
      },
      {
        "title": "Fred Vargas - Uit De Dood Herrezen (Hardcover/Gebonden) blauw/groene achtergrond",
        "origin": "http://www.marktplaza.nl/boeken/romans/fred-vargas-uit-de-dood-herrezen-hardcover-gebonden-blauw-groene-achtergrond-92385368.html",
        "imageURL": "http://www.marktplaza.nl/M92385368/1/fred-vargas-uit-de-dood-herrezen-hardcover-gebonden-blauw-groene-achtergrond-92385368.jpg",
        "price": 12.5
      },
      {
        "title": "Fred Vargas  -   De Omgekeerde Man   (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/romans/fred-vargas-de-omgekeerde-man-hardcover-gebonden-92385304.html",
        "imageURL": "http://www.marktplaza.nl/M92385304/1/fred-vargas-de-omgekeerde-man-hardcover-gebonden-92385304.jpg",
        "price": 15
      },
      {
        "title": "David Sandes  -  Sergei Bubka's Wondermethode  (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/romans/david-sandes-sergei-bubkas-wondermethode-hardcover-gebonden-92385090.html",
        "imageURL": "http://www.marktplaza.nl/M92385090/1/david-sandes-sergei-bubkas-wondermethode-hardcover-gebonden-92385090.jpg",
        "price": 10
      },
      {
        "title": "Sjoerd Kuyper  -  Sjaantje Doet Alsof  (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/kinderboeken/sjoerd-kuyper-sjaantje-doet-alsof-hardcover-gebonden-92384948.html",
        "imageURL": "http://www.marktplaza.nl/M92384948/1/sjoerd-kuyper-sjaantje-doet-alsof-hardcover-gebonden-92384948.jpg",
        "price": 10
      },
      {
        "title": "Het Piratenschip     Klap Open En Bekijk  (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/kinderboeken/het-piratenschip-klap-open-en-bekijk-hardcover-gebonden-92371996.html",
        "imageURL": "http://www.marktplaza.nl/M92371996/1/het-piratenschip-klap-open-en-bekijk-hardcover-gebonden-92371996.jpg",
        "price": 12.5
      },
      {
        "title": "John Topsell  -  Draken Trainen En Verzorgen (Hardcover/Gebonden)",
        "origin": "http://www.marktplaza.nl/boeken/kinderboeken/john-topsell-draken-trainen-en-verzorgen-hardcover-gebonden-92371928.html",
        "imageURL": "http://www.marktplaza.nl/M92371928/1/john-topsell-draken-trainen-en-verzorgen-hardcover-gebonden-92371928.jpg",
        "price": 15
      }
    ]
    

    【讨论】:

    • 还有一件事 - 我还必须添加描述(位于原始 URL 上)。我可以只遍历 JSON,转到原始 URL,然后从那里获取描述以稍后将其添加到对象中吗?还是我应该立即将这个额外的步骤放入您建议的 for 循环中?
    • 描述是什么意思?产品类型?你可以在 for 循环中搞乱它,因为网站并不复杂,故障点也更少。
    • 我的意思是当您点击product 时,在“Omschrijving”的小标题下方有一个产品描述。我将尝试在 for 循环中执行此操作,再次感谢!
    • 哦,您的意思是访问并获取数据,抱歉没有正确读取。您可能可以在该特定网站上的 for 循环中执行此操作,但要小心,许多网站都有请求速率限制。如果网站有速率限制,你可以设置一个 sleep 函数来避免这种情况,或者你可以添加一个 if else 处理程序来查看响应代码是否为 200(OK)。
    • 太棒了,非常感谢您的帮助。我会直截了当的!我的赞成票尚不可见,但我赞成您的答案并将其标记为最佳。 :)
    猜你喜欢
    • 1970-01-01
    • 2016-01-05
    • 1970-01-01
    • 2021-11-15
    • 1970-01-01
    • 1970-01-01
    • 2019-08-26
    • 2020-11-26
    • 2015-05-09
    相关资源
    最近更新 更多