【发布时间】:2019-02-18 14:22:54
【问题描述】:
当我运行这段代码时,我得到了这个错误:
ValueError: 信号只在主线程中起作用
我正在使用 ActiveMQ。
RealTor 是我的 Spider(selogerSpider) 的名称,它帮助我进行抓取。
import stomp
from RealTor import selogerSpider
from scrapy.crawler import CrawlerProcess
from scrapy.settings import Settings
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
settings = Settings()
settings.set("USER_AGENT",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_4)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36")
settings.set("LOG_ENABLED", False)
crawler = CrawlerProcess(settings)
crawler.crawl(selogerSpider)
selogerSpider.signals.engine_started()
print("STARTING scraping")
crawler.start()
print("Scraping STOPPED")
try:
conn = stomp.Connection()
conn.set_listener('', MyListener())
conn.start()
""" conn = stomp.Connection([('0.0.0.0', 61613)])"""
conn.connect('admin', 'password', wait=True)
conn.subscribe(destination='/queue/test', id=1, ack='auto')
print('subscripe')
"""conn.send(body=' '.join(sys.argv[1:]), destination='/queue/test')"""
input("coucou")
"""conn.disconnect()"""
print('end')
except IOError as e:
print("error message")
【问题讨论】:
标签: python web-scraping