获取空白抓取数据[关闭]答案

【问题标题】：Getting blank scrape data [closed]获取空白抓取数据[关闭]
【发布时间】：2013-10-01 23:00:38
【问题描述】：

一旦我从网页上抓取数据并获得空白值，就无法获得准确的数据。以下是代码：

require 'nokogiri'
require 'open-uri'

number=1
url="http://www.jabong.com/109F/"

doc = Nokogiri::HTML(open(url))
puts doc.at_css("title").text
products=doc.css('.narrow')
products.each do |item|
product_name = item.at_css('.itm-title').text unless item.at_css('.itm-title').nil?
product_price = item.at_css('.itm-priceBox').text unless item.at_css('.itm-priceBox').nil?
puts product_name
puts product_price    
puts number
number+=1
end
puts "it is the end of code"

【问题讨论】：

你应该放一个输出示例而不是 url，因为 url 内容可能会改变（也供将来参考）
我正在从 URL jabong.com/109F 抓取数据输出需要像衬衫200 裤子300 tshirt900

标签： javascript ruby web-scraping screen-scraping nokogiri

【解决方案1】：

我认为您应该使用 xpath 而不是 CSS。 xpath 非常强大，可以让您在 DOM 中导航更容易抓取数据。

例如，获取提供的 URL 中的所有服装名称并输出一个数组：

doc.xpath("//ul[@id = 'productsCatalog']//li//a//span[@class='itm-title']").children.map{|x| x.text.gsub("\r\n                                                                                    ","").strip}

 => ["Ruffle Sleeves Self Pattern Green Top", "Cap Sleeve Solid Red Top", "3/4Th Sleeve Embroidered Blue Tunic", "Ruffle Sleeves Embroidered Beige Tunic", "3/4Th Sleeve Stripe Blue Tunic", "Ruffle Sleeves Printed Beige/Pink Tunic", "Short Sleeve Embroidered Black Tunic", "Sleeve Less Solid Black Top", "Short Sleeve Solid Black Top", "Puffed Sleeve Embroidered Black Top", "Sleeve Less Printed Cream Top", "Sleeve Less Solid Yellow Top", "3/4Th Sleeve Embroidered Black Top", "Sleeve Less Solid Off White Top", "Sleeve Less Solid Fuschia Top", "Mega Sleeves Embroidered Green T-Shirt", "Sleeve Less Solid Orange Tunic", "Puffed Sleeve Embroidered Black Top", "Mega Sleeves Printed Beige Top", "Mega Sleeves Printed Beige T-Shirt", "Puffed Sleeve Stripe Red Dress", "Puffed Sleeve Stripe Blue Top", "Puffed Sleeve Printed Yellow Top", "Cap Sleeves Printed Multi Dress", "Short Sleeve Solid Mustard Yellow Tunic", "Mega Sleeves Embroidered Red Tunic", "Short Sleeve Stripe Red Dress", "Puffed Sleeve Embroidered Beige Top", "Short Sleeve Solid Black Top", "Butterfly Sleeve Check Beige Top", "Short Sleeve Solid Black tunic", "Sleeve Less Embroidered Green Top", "Mega Sleeves Printed Cream Tunic", "Mega Sleeves Embroidered Black Tunic", "Ruffle Sleeves Printed Orange Tunic", "Mega Sleeves Solid Pink Top", "Ruffle Sleeves Solid Rust Dress", "Sleeve Less Solid Navy Blue Dress", "Sleeve Less Self Pattern White Top", "Puffed Sleeve Stripe Red Top", "Mega Sleeves Solid Pink Tunic", "Roll Up Sleeve Solid Off White Tunic", "Sleeve Less Pintucks White Top", "Puffed Sleeve Embroidered Off White Top", "Puffed Sleeve Printed Pink Top", "Raglan Sleeve Solid Black Top", "Mega Sleeves Printed Beige Tunic", "Sleeve Less Solid Yellow Top", "Puffed Sleeve Self Pattern Beige Top", "Cap Sleeve Printed Yellow Dress", "Sleeve Less Solid Pink Dress"]

【讨论】：

Sean 它正确地给出了产品的名称，但同时我无法获得产品的价格。您如何获得 xpath 代码，因为我不知道如何使用 xpath 可以请您通过代码并告诉我工具和教程，我可以从中了解如何创建 xpath。此外，如果应用了 ajax 页面或分页，那么如何在其中使用 xpath
正确我只提供了一个数据来说明如何使用 xpath 搜索 DOM。我会查看 Nokogiri 的文档或在线查找有关如何将 xpath 与 Nokogiri 一起使用的教程。在我的代码中，我所做的一切都列出了所用元素的“映射”。 This link 将是您可以开始的一个很好的例子。您可能会发现您要么编写一行复杂的代码，要么编写多行代码来获取您要查找的所有数据。