Python：让迭代器在不同线程上迭代日志消息

Question

我有 2 个线程在运行：

# Thread 1: In model
logging.getLogger('ui').info("Sit still, I'm computing...")
more_stuff = compute_stuff(stuff)
logging.info("Ok, I'm done.")

...

# Thread 2: In view, streaming messages to client
for message in log_iterator('ui'):
    send_to_client(message)

我希望 log_iterator 挂起，直到出现日志消息，然后当出现日志消息时，对其进行处理。问题是，你如何制作 log_iterator？或者有没有更好的方法来实现这个目标？

Answer 1

您可以通过 1 个线程将日志写入文件。第二个线程可以读取文件并显示日志。

这是日志配置：

import logging
logger = logging.getLogger('simple_example') # set logger
logger.setLevel(logging.INFO) # set logger level

fh = logging.FileHandler('path-to-log-file') #set Handler
fh.setLevel(logging.INFO) #set Handler level

formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') # set log format
fh.setFormatter(formatter) # add format to Handler

logger.addHandler(fh) # add Handler to logger

在我的示例函数中 rewrite 记录到另一个文件：

def print_message():
    for i in range(0,1000):     
        f = open('path-to-log-file', 'r')
        s = f.read()
        f.close()
        f2 = open('path-to-log-file2', 'r+')
        f2.write(s)
        f2.close()
        time.sleep(.2)

import threading
t = threading.Thread(target=print_message)
t.start()
logger.error('error message1')
logger.error('error message2')
logger.error('error message3')
t.join()

所以记录器通过第一个线程将日志写入文件，第二个线程从文件中读取日志。 我想这就是你想要的。

此外，如果您想避免将日志写入文件，您可以制作自己的处理程序，将日志准确发送到第二个线程。我认为它可能很复杂，但我对线程知之甚少。

这里是日志文档：https://docs.python.org/2/howto/logging.html

Answer 2

好吧，在玩了一些之后，这是我想出的解决方案：

def log_iterator(logger_names):
    '''
    Get an iterator that returns LogRecord objects whenever a log message is made.
    
    logger_names can be:
        a string identifying which logger you want to get messages iterate from
        a list identifying all the loggers you want to get messages from
        a dict<name: level> where level identifies the minimum loggling level 
            you want (default is INFO)
    '''
    if isinstance(logger_names, str):
        logger_names = {logger_names: logging.INFO}
    elif isinstance(logger_names, (list, tuple)):
        logger_names = {name: logging.INFO for name in logger_names}
    
    log_message_event = threading.Event()

;你应该使用 items() 因为 iteritems() 已被弃用

    for logger_name, log_level in logger_names.iteritems():
        logger = logging.getLogger(logger_name)
        logger.addHandler(FireOnLog(log_message_event, log_level))
    
    while True:
        log_message_event.wait()
        record = log_message_event.record
        log_message_event.clear()
        yield record
        
        
class FireOnLog(logging.StreamHandler):
    '''
    Custom loghandler that fires an event every time a log message 
    comes in.  To use, you need to add this handler to a logger.
    '''

    def __init__(self, event, log_level = logging.INFO):
        '''
        Event is a threading.Event object
        log_level is the minimum logging level you want
        '''
        logging.StreamHandler.__init__(self)
        self._event = event
        self._log_level = log_level

    def emit(self, record):
        if record.levelno >= self._log_level:
            self._event.record = record
            self._event.set()

使用以下代码测试时表现正确：

def _print_messages():
    print 'Started Printing loop'
    for log_record in log_iterator(['testlog', 'anotherlog']):
        print '%s:%s:%s' % (log_record.levelname, log_record.name, log_record.message)
    

def test_log_handling():
    
    t = threading.Thread(target = _print_messages)
    t.start()
    
    time.sleep(.1)
    logging.getLogger('testlog').critical('Message!')
    logging.getLogger('anotherlog').critical('Another Message right away')
    logging.getLogger('badlog').critical("Someething useless that you don't want to hear")
    time.sleep(.1)
    logging.getLogger('testlog').critical('Another Message after some wait')

这似乎是 python 内置的非常通用的功能，所以如果有人知道如何使用内置函数执行此操作，请回复，我会给你一个点！