Python 多线程日志记录

Question

我正在尝试编写一个简单的端口检查程序来测试 VMware Horizo​​n 网络端口。端口检查本身工作得很好，但是我的“多线程”日志在第 71 行和第 91 行之间不足。我知道事实上我做错了，因为我将多个帖子中的日志记录部分拼凑在一起在线的。我没有运行记录器的经验，而且我的 Python 在我最好的一天是中级的。无论如何，有人可以帮我清理日志记录部分并使其按预期工作吗？

我在多个线程中运行端口检查以提高速度，随后我想准确记录每个线程的结果。我想我已经让这个工作了一段时间，但是我对我的日志记录代码的开头做了一些修改以“清理”它并且我破坏了一些东西。不幸的是，当时我在 git repo 中没有这个，所以我不能再恢复我的更改了。如果可以的话，请给我更多关于如何修复/重写日志记录部分以完成干净的多线程日志记录的详细信息。我什至有兴趣将线程日志记录分组在一起以提高可读性，但这不是必须的。提前感谢您的帮助。

这是一个抛出错误的示例，整个清理过的代码都在下面。

错误信息：

THREAD INFORMATION PRINT
<Worker(Thread-1, initial)>
THREADRESULT IS:
None
logging_threads is:
[<Worker(Thread-1, started 140445767894784)>]
INFO    : pod1.azvd.private.example.com # Checking port: 80 on pod1.azvd.private.example.com
INFO    : pod1.azvd.private.example.com # Checking port: 443 on pod1.azvd.private.example.com
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "./horizon_troubleshooter.py", line 41, in run
    self.logger.info('Created Log for {}'.format(self.info['pod_name']))
  File "/usr/lib/python2.7/logging/__init__.py", line 1455, in info
    self.logger.info(msg, *args, **kwargs)
AttributeError: 'NoneType' object has no attribute 'info'

完整代码：

#!/usr/bin/python

# VMware Horizon troubleshooter
# Test port connectivity and call
# and response, then output to file.

import sys
import logging
import socket
import time
import threading
import traceback
from contextlib import closing

logging.basicConfig(level=logging.DEBUG,
                    format= '%(asctime)-15s %(levelname)-8s: %(thread)x -- %(pod_name)s - %(funcName)s:%(lineno)d # %(message)s',
                    datefmt='%m-%d %H:%M',
                    filename='/tmp/horizon-troubleshooter.log',
                    filemode='w')

console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(levelname)-8s: %(pod_name)-12s # %(message)s')
console.setFormatter(formatter)
logger = logging.getLogger()
logger = logging.getLogger('').addHandler(console)

class Worker(threading.Thread):
    def __init__(self, info):
        self.running=False
        self.info=info
        self.logger=logging.LoggerAdapter(logger, self.info)
        super(Worker, self).__init__()
    def start(self):
        self.running=True
        super(Worker, self).start()
    def stop(self):
        self.running=False
    def run(self):
        while self.running:
            self.logger.info('Created Log for {}'.format(self.info['pod_name']))
            time.sleep(0.5)

def check_port(host, port):
    try:
        ip = socket.gethostbyname(host) # get IP address of host
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)  # TCP
        #sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)  # UDP
        socket.setdefaulttimeout(2.0) # floating integer for seconds
        result = sock.connect_ex((host,port))
        if result == 0:
            logging.info('Port: %s OPEN', port, extra={'pod_name': host})
            return True
        else:
            logging.info('Port: %s CLOSED', port, extra={'pod_name': host})
            return False
        sock.close()
        logging.debug('Result: %s for port check on: %s', result, port, extra={'pod_name': host}) 
    except:
        logging.error('check_port failed for %s', port, extra={'pod_name': host })

# set variables for ports and hosts to check
port_list = [80, 443, 4172, 8443]
domain_name = ".azvd.private.example.com"
hostname_prefix = "pod"
num_pods = 22
logging_threads = []
max_threads = 50    # maximum thread count for multi-threaded port check

# iterate through pods and test ports
for pod_num in range(1, num_pods + 1):
    try:
        pod_name = (hostname_prefix + str(pod_num) + domain_name)
        logging.debug('PODNAME IS %s', pod_name, extra={'pod_name': pod_name})
        kwargs = ({'pod_name': pod_name})
        logging.debug('KWARGS is %s', kwargs, extra=kwargs)
        thread = Worker({'pod_name': pod_name})
        print("THREAD INFORMATION PRINT")
        print(thread)
        threadresult = thread.start()
        print('THREADRESULT IS:')
        print(threadresult)
        logging_threads.append(thread)
        print('logging_threads is: ')
        print(logging_threads)
        logging.debug("Thread started and appended to logging_threads", extra=kwargs)
    except NameError:
        exc_type, exc_value, exc_traceback = sys.exc_info()
        lines = traceback.format_exception(exc_type, exc_value, exc_traceback)
        logging.error('\nFailed to create logging thread on %s', pod_name, extra=kwargs)
        logging.exception("exception data: %s", lines, extra=kwargs)

    for port in port_list:
        try:
            logging.info('Checking port: %s on %s', port, pod_name, extra={'pod_name': pod_name})
            threading.Thread(target=check_port, args=[str(pod_name), port]).start()
        except:
            logging.error('PORT_LIST FOR-LOOP FAILURE', extra={'pod_name': pod_name})

    while threading.active_count() > max_threads :
        time.sleep(2)

for lt in logging_threads:
    lt.stop()
for lt in logging_threads:
    lt.join()

Answer 1

我从来没有设法让日志记录变得更干净，比如将日志文件中每个线程的日志输出分组，但这对我来说并不是一个显示停止器。但是，我确实让应用程序正常工作。我的 OP 试图解决的这个错误的修复：

错误

 Traceback (most recent call last):
 File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
   self.run()
 File "./horizon_troubleshooter.py", line 41, in run
   self.logger.info('Created Log for {}'.format(self.info['pod_name']))
 File "/usr/lib/python2.7/logging/__init__.py", line 1455, in info
   self.logger.info(msg, *args, **kwargs) 
 AttributeError: 'NoneType' object has no attribute 'info'

解决方案是对引用记录器变量分配的语句进行小幅更改。更改在下面的代码 sn-ps 中突出显示。

破解密码

logging.basicConfig(level=logging.DEBUG,
                    format= '%(asctime)-15s %(levelname)-8s: %(thread)x -- %(pod_name)s - %(funcName)s:%(lineno)d # %(message)s',
                    datefmt='%m-%d %H:%M',
                    filename='/tmp/horizon-troubleshooter.log',
                    filemode='w')

console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(levelname)-8s: %(pod_name)-12s # %(message)s')
console.setFormatter(formatter)
logger = logging.getLogger()                        # here is where the problem starts
logger = logging.getLogger('').addHandler(console)  # here is where the problem ends

更正的代码

logging.basicConfig(level=logging.INFO,
                    format='%(asctime)-15s %(levelname)-8s: %(thread)x -- %(pod_name)s - %(funcName)s:%(lineno)d # %(message)s',
                    datefmt='%m-%d %H:%M',
                    filename='/home/gecko/Desktop/horizon-troubleshooter.log',
                    filemode='w')

console = logging.StreamHandler()
console.setLevel(logging.INFO)
formatter = logging.Formatter('%(levelname)-8s: %(pod_name)-12s # %(message)s')
console.setFormatter(formatter)
logger = logging.getLogger()
logging.getLogger('').addHandler(console)     # **Here is the correction**

长话短说，我正在使用损坏代码中的最后一条语句重新分配我的 logger 变量。我想做的只是为我的日志写入添加第二个输出（即控制台），我需要做的就是更改最后一行代码，如上面的“更正代码”部分所示。