使用scrapy在python中使用css选择访问网络抓取中的数据

Question

我试图在 python 中使用 Scrapy 和 css select 从网站上抓取 BHK 详细信息。 谁能告诉我如何从下面的输出中访问文本- 3 BHK Residential Apartment,3,4 BHK Independent House/Villa-。

输出：

{'bhk': ['<tr class="NpsrpTuple__subHead"><td>3 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4,5 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->8518<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->4226<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>1,2 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->4700<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->4199<!-- --> / Sq.Ft. </span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->7500<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ '
         '<!-- -->4947<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>4 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->7229<!-- --> / Sq.Ft. </span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->6572<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>New '
         'Launch</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->6453<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>New '
         'Launch</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->6618<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->5900<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Under Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td><span class="NpsrpTuple__webRupee">₹ <!-- '
         '-->5233<!-- --> / Sq.Ft. <!-- -->Onwards</span></td><td '
         'class="undefined"><span>Ready To Move</span></td></tr']

我尝试运行代码：

bhk = response.css('.NpsrpTuple__subHead').extract()

Answer 1

不确定这是否是您想要实现的目标：

from scrapy import Selector

item = {'bhk': ['<tr class="NpsrpTuple__subHead"><td>3 BHK Residential Apartment<!-- '
         '--> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>3,4 BHK Independent '
         'House/Villa<!-- --> </td><td></td><td class="undefined"><span>Under '
         'Construction</span></td></tr>',
         '<tr class="NpsrpTuple__subHead"><td>2,3 BHK Residential '
         'Apartment<!-- --> </td><td></td><td class="undefined"><span>Ready To '
         'Move</span></td></tr>']
    }


items = ""
for i in item['bhk']:
    items+=i

response = Selector(text=items)
for elem in response.css('tr > td::text').getall():
    print(elem)

输出：

3 BHK Residential Apartment
3,4 BHK Independent House/Villa
2,3 BHK Residential Apartment