【问题标题】:python multiprocessing - sending child process logging to GUI running in parentpython multiprocessing - 将子进程日志记录发送到在父进程中运行的 GUI
【发布时间】:2019-04-16 18:01:21
【问题描述】:

我正在编写的一些分析代码之上构建一个界面,该代码执行一些 SQL 并处理查询结果。在我想向用户公开的分析代码中,围绕着一些事件进行了日志记录。因为分析代码运行时间较长,而且我不希望 UI 阻塞,所以到目前为止,我已经通过将分析函数放入其自己的线程来完成此操作。

我现在拥有的简化示例(完整脚本):

import sys
import time
import logging
from PySide2 import QtCore, QtWidgets

def long_task():
    logging.info('Starting long task')
    time.sleep(3) # this would be replaced with a real task
    logging.info('Long task complete')

class LogEmitter(QtCore.QObject):
    sigLog = QtCore.Signal(str)

class LogHandler(logging.Handler):
    def __init__(self):
        super().__init__()
        self.emitter = LogEmitter()
    def emit(self, record):
        msg = self.format(record)
        self.emitter.sigLog.emit(msg)

class LogDialog(QtWidgets.QDialog):
    def __init__(self, parent=None):
        super().__init__(parent)
        log_txt = QtWidgets.QPlainTextEdit(self)
        log_txt.setReadOnly(True)
        layout = QtWidgets.QHBoxLayout(self)
        layout.addWidget(log_txt)
        self.setWindowTitle('Event Log')
        handler = LogHandler()
        handler.emitter.sigLog.connect(log_txt.appendPlainText)
        logger = logging.getLogger()
        logger.addHandler(handler)
        logger.setLevel(logging.INFO)

class Worker(QtCore.QThread):
    results = QtCore.Signal(object)

    def __init__(self, func, *args, **kwargs):
        super().__init__()
        self.func = func
        self.args = args
        self.kwargs = kwargs

    def run(self):
        results = self.func(*self.args, **self.kwargs)
        self.results.emit(results)

class MainWindow(QtWidgets.QMainWindow):

    def __init__(self):
        super().__init__()
        widget = QtWidgets.QWidget()
        layout = QtWidgets.QHBoxLayout(widget)
        start_btn = QtWidgets.QPushButton('Start')
        start_btn.clicked.connect(self.start)
        layout.addWidget(start_btn)
        self.setCentralWidget(widget)

        self.log_dialog = LogDialog()
        self.worker = None

    def start(self):
        if not self.worker:
            self.log_dialog.show()
            logging.info('Run Starting')
            self.worker = Worker(long_task)
            self.worker.results.connect(self.handle_result)
            self.worker.start()

    def handle_result(self, result=None):
        logging.info('Result received')
        self.worker = None

if __name__ == '__main__':
    app = QtWidgets.QApplication()
    win = MainWindow()
    win.show()
    sys.exit(app.exec_())

这很好用,只是我需要能够允许用户停止执行分析代码。我读过的所有内容都表明没有办法很好地中断线程,所以使用multiprocessing 库似乎是要走的路(没有办法重写分析代码以允许定期轮询,因为大多数的时间只是等待查询返回结果)。使用multiprocessing.Poolapply_async 以不阻塞UI 的方式执行分析代码很容易获得相同的功能。

例如将上面的 MainWindow 替换为:

class MainWindow(QtWidgets.QMainWindow):

    def __init__(self):
        super().__init__()
        widget = QtWidgets.QWidget()
        layout = QtWidgets.QHBoxLayout(widget)
        start_btn = QtWidgets.QPushButton('Start')
        start_btn.clicked.connect(self.start)
        layout.addWidget(start_btn)
        self.setCentralWidget(widget)

        self.log_dialog = LogDialog()
        self.pool = multiprocessing.Pool()
        self.running = False

    def start(self):
        if not self.running:
            self.log_dialog.show()
            logging.info('Run Starting')
            self.pool.apply_async(long_task, callback=self.handle_result)

    def handle_result(self, result=None):
        logging.info('Result received')
        self.running = False

但我似乎无法弄清楚如何从子进程检索日志输出并将其传递给父进程以更新日志对话框。我已经阅读了几乎所有关于此的 SO 问题以及如何处理从多个进程写入单个日志文件的食谱示例,但我无法理解如何使这些想法适应我​​的想法我想在这里做。

编辑

所以试图弄清楚为什么我看到的行为与我添加的@eyllanesc 不同:

logger = logging.getLogger()
print(f'In Func: {logger} at {id(logger)}')

logger = logging.getLogger()
print(f'In Main: {logger} at {id(logger)}')

分别发给long_taskMainwindow.start。当我运行main.py 时,我得到:

In Main: <RootLogger root (INFO)> at 2716746681984
In Func: <RootLogger root (WARNING)> at 1918342302352

这似乎是this SO question中描述的内容

使用QueueQueueHandler 作为解决方案的想法似乎类似于@eyllanesc 的原始解决方案

【问题讨论】:

    标签: python multithreading logging multiprocessing pyside2


    【解决方案1】:

    信号不会在进程之间传输数据,因此对于这种情况,必须使用管道然后发出信号:

    # other imports
    import threading
    # ...
    
    class LogHandler(logging.Handler):
        def __init__(self):
            super().__init__()
            self.r, self.w = multiprocessing.Pipe()
            self.emitter = LogEmitter()
            threading.Thread(target=self.listen, daemon=True).start()
    
        def emit(self, record):
            msg = self.format(record)
            self.w.send(msg)
    
        def listen(self):
            while True:
                try:
                    msg = self.r.recv()
                    self.emitter.sigLog.emit(msg)
                except EOFError:
                    break
    
    # ...
    

    【讨论】:

    • 我在这里仍然遗漏了一些东西。来自子进程的日志数据如何进入管道?创建子进程时,我是否需要为其提供管道的工作端(在这种情况下,我仍然不确定如何在子进程中设置目标函数)?只需将其放入我当前拥有的代码中就会产生相同的行为(即,父进程按预期记录,但子日志不会在父进程中结束)。
    • @user3014097 真奇怪,我试过了,它工作正常,你是在问题代码中使用我的代码还是在另一个个性化代码中?
    • @user3014097 在下面的链接中你会找到我的测试代码:gist.github.com/eyllanesc/72a67e76350691012580f2207f1dd12b,试试看告诉我它是否有效。
    • 嗯...是的,运行你的 main.py 我在脚本中看到了相同的行为(即只有父级记录到对话框中)。这是在 Windows 上运行的,这里解释的问题是否仍然适用? stackoverflow.com/questions/26167873/…
    • 我将把它标记为已回答,因为您的解决方案显然有效。鉴于平台限制,我仍然不确定如何在 Windows 上运行它,但至少这是一个开始的地方
    【解决方案2】:

    万一有人在路上徘徊,使用QueueHandlerQueueListener 会导致一个也适用于Windows 的解决方案。大量借用this answer to a similar question:

    import logging
    import sys
    import time
    import multiprocessing
    from logging.handlers import QueueHandler, QueueListener
    from PySide2 import QtWidgets, QtCore
    
    def long_task():
        logging.info('Starting long task')
        time.sleep(3) # this would be replaced with a real task
        logging.info('Long task complete')
    
    def worker_init(q):
        qh = QueueHandler(q)
        logger = logging.getLogger()
        logger.setLevel(logging.INFO)
        logger.addHandler(qh)
    
    class LogEmitter(QtCore.QObject):
        sigLog = QtCore.Signal(str)
    
    class LogHandler(logging.Handler):
        def __init__(self):
            super().__init__()
            self.emitter = LogEmitter()
        def emit(self, record):
            msg = self.format(record)
            self.emitter.sigLog.emit(msg)
    
    class LogDialog(QtWidgets.QDialog):
        def __init__(self, parent=None):
            super().__init__(parent)
            self.log_txt = QtWidgets.QPlainTextEdit(self)
            self.log_txt.setReadOnly(True)
            layout = QtWidgets.QHBoxLayout(self)
            layout.addWidget(self.log_txt)
            self.setWindowTitle('Event Log')
    
    class MainWindow(QtWidgets.QMainWindow):
    
        def __init__(self):
            super().__init__()
            widget = QtWidgets.QWidget()
            layout = QtWidgets.QHBoxLayout(widget)
            start_btn = QtWidgets.QPushButton('Start')
            start_btn.clicked.connect(self.start)
            layout.addWidget(start_btn)
            self.setCentralWidget(widget)
    
            self.log_dialog = LogDialog()
            self.running = False
    
            # sets up handler that will be used by QueueListener
            # which will update the LogDialoag
            handler = LogHandler()
            handler.emitter.sigLog.connect(self.log_dialog.log_txt.appendPlainText)
    
            self.q = multiprocessing.Queue()
            self.ql = QueueListener(self.q, handler)
            self.ql.start()
    
            # main process should also log to a QueueHandler
            self.main_log = logging.getLogger('main')
            self.main_log.propagate = False
            self.main_log.setLevel(logging.INFO)
            self.main_log.addHandler(QueueHandler(self.q))
    
            self.pool = multiprocessing.Pool(1, worker_init, [self.q])
    
        def start(self):
            if not self.running:
                self.log_dialog.show()
                self.main_log.info('Run Starting')
                self.pool.apply_async(long_task, callback=self.handle_result)
    
        def handle_result(self, result=None):
            time.sleep(2)
            self.main_log.info('Result received')
            self.running = False
    
        def closeEvent(self, _):
            self.ql.stop()
    
    if __name__ == '__main__':
        app = QtWidgets.QApplication()
        win = MainWindow()
        win.show()
        sys.exit(app.exec_())
    

    【讨论】:

      猜你喜欢
      • 2020-01-12
      • 2014-03-24
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2019-10-17
      相关资源
      最近更新 更多