【发布时间】:2013-07-03 06:52:35
【问题描述】:
有没有办法查询当前Gevent进程中greenlets的数量,以及它们的状态?
例如,我想用任意 greenlet 抓取任意网站,然后我运行另一个 greenlet 以查询正在运行的有多少是已完成/异常。
或者我应该将一个全局变量设置为计数器? Gevent 有类似的内置功能吗?
【问题讨论】:
有没有办法查询当前Gevent进程中greenlets的数量,以及它们的状态?
例如,我想用任意 greenlet 抓取任意网站,然后我运行另一个 greenlet 以查询正在运行的有多少是已完成/异常。
或者我应该将一个全局变量设置为计数器? Gevent 有类似的内置功能吗?
【问题讨论】:
Gevent 没有这样的东西。但您可以使用bool(g)、g.ready() 和g.successful() 来检查其状态。我会以这种方式轮询greenlets的状态:
import gevent
import random
def _get_status(greenlets):
total = 0
running = 0
completed = 0
successed = 0
yet_to_run = 0
failed = 0
for g in greenlets:
total += 1
if bool(g):
running += 1
else:
if g.ready():
completed += 1
if g.successful():
successed += 1
else:
failed += 1
else:
yet_to_run += 1
assert yet_to_run == total - completed - running
assert failed == completed - successed
return dict(total=total,
running=running,
completed=completed,
successed=successed,
yet_to_run=yet_to_run,
failed=failed)
def get_greenlet_status(greenlets):
while True:
status = _get_status(greenlets)
print status
if status['total'] == status['completed']:
return
gevent.sleep(5)
def crawl(url):
r = random.randint(0, 10)
gevent.sleep(r)
err = random.randint(0, 4)
if err == 0:
raise Exception
greenlets = [gevent.spawn(crawl, each) for each in xrange(100)]
get_greenlet_status(greenlets)
【讨论】:
bool(g) 为真,它仍在运行。