Python 的 SyslogHandler 和 TCP

Question

我试图理解为什么 Python 的日志框架 (logging.handlers) 中的 SyslogHandler 类没有实现 RFC 6587 描述的任何框架机制：

1. 八位组计数：它将消息长度“添加”到系统日志帧：

2. 非透明框架：用于分隔消息的拖车字符。这是大多数服务器所理解的。

这个“问题”可以通过在消息末尾添加一个 LF 字符来轻松解决，但是我希望 SyslogHandler 默认会处理这个问题：

sHandler = logging.handlers.SysLogHandler(address=(address[0], address[1]), socktype = socket.SOCK_STREAM)
sHandler.setFormatter(logging.Formatter(fmt=MSG_SYSLOG_FORMAT, datefmt=DATE_FMT))
self.addHandler(sHandler)

这既不适用于 Fluentd，也不适用于 rsyslog。正如我所说，我暂时将其添加到 handlers.py 的第 855 行（只是为了测试）：

msg = prio + msg + '\n'

现在正在工作。

---

我的问题：

1. Python SyslogHandler 类是否应该提供设置打开/关闭八位字节计数或尾部字符的可能性。目前它什么都不做...
2. 程序员的工作是了解服务器如何工作并重写 Handler 以解决消息帧问题？

现在，我现在正在做的是重写 emit() 方法，子类化 SyslogHandler。

Answer 1

logging 中的 Syslog 支持早于 RFC，在 RFC 之前，几乎没有标准。

确切地说：SysLogHandler 处理程序在 first added to the Python standard library in 2002 时是 logging 的一部分，此后基本保持不变（TCP 支持为 added in 2009，RFC5424 支持在 2011 年得到改进）；原始代码基于this syslog module from 1997。

other bug reports 很明显，维护者希望在此处的代码中保持最广泛的向后兼容性，因此如果您需要来自较新 RFC 的特定功能，您有两种选择：

• 扩展类并自己实现该功能
• 提交功能请求和/或补丁以改进logging 模块中的功能；考虑向后兼容性要求。

Answer 2

感谢@Martijn Pieters♦ 的回答，我的回答扩展了他的回答。

我实现了一个继承自 SyslogHandler 类并覆盖 emit 函数的类。 我还为此问题打开了一个拉取请求： https://github.com/python/cpython/pull/24556

python2：

import socket
import logging.handlers as handlers


class TcpSyslogHandler(handlers.SysLogHandler):
    """
    This class override the python SyslogHandler emit function.
    It is needed to deal with appending of the nul character to the end of the message when using TCP.
    Please see: https://stackoverflow.com/questions/40041697/pythons-sysloghandler-and-tcp/40152493#40152493
    """
    def __init__(self, message_separator_character, address=('localhost', handlers.SYSLOG_UDP_PORT),
                 facility=handlers.SysLogHandler.LOG_USER,
                 socktype=None):
        """
        The user of this class must specify the value for the messages separator.
        :param message_separator_character: The value to separate between messages.
                                            The recommended value is the "nul character": "\000".
        :param address: Same as in the super class.
        :param facility: Same as in the super class.
        :param socktype: Same as in the super class.
        """
        super(SfTcpSyslogHandler, self).__init__(address=address, facility=facility, socktype=socktype)

        self.message_separator_character = message_separator_character

    def emit(self, record):
        """
        SFTCP addition:
        To let the user to choose which message_separator_character to use, we override the emit function.
        ####
        Emit a record.

        The record is formatted, and then sent to the syslog server. If
        exception information is present, it is NOT sent to the server.
        """
        try:
            msg = self.format(record) + self.message_separator_character

            """
            We need to convert record level to lowercase, maybe this will
            change in the future.
            """
            prio = '<%d>' % self.encodePriority(self.facility, self.mapPriority(record.levelname))
            # Message is a string. Convert to bytes as required by RFC 5424
            if type(msg) is unicode:
                msg = msg.encode('utf-8')
            msg = prio + msg
            if self.unixsocket:
                try:
                    self.socket.send(msg)
                except socket.error:
                    self.socket.close()  # See issue 17981
                    self._connect_unixsocket(self.address)
                    self.socket.send(msg)
            elif self.socktype == socket.SOCK_DGRAM:
                self.socket.sendto(msg, self.address)
            else:
                self.socket.sendall(msg)
        except (KeyboardInterrupt, SystemExit):
            raise
        except Exception:
            self.handleError(record)

python3：

import socket
import logging.handlers as handlers


class SfTcpSyslogHandler(handlers.SysLogHandler):
    """
    This class override the python SyslogHandler emit function.
    It is needed to deal with appending of the nul character to the end of the message when using TCP.
    Please see: https://stackoverflow.com/questions/40041697/pythons-sysloghandler-and-tcp/40152493#40152493
    """
    def __init__(self, message_separator_character, address=('localhost', handlers.SYSLOG_UDP_PORT),
                 facility=handlers.SysLogHandler.LOG_USER,
                 socktype=None):
        """
        The user of this class must specify the value for the messages separator.
        :param message_separator_character: The value to separate between messages.
                                            The recommended value is the "nul character": "\000".
        :param address: Same as in the super class.
        :param facility: Same as in the super class.
        :param socktype: Same as in the super class.
        """
        super(SfTcpSyslogHandler, self).__init__(address=address, facility=facility, socktype=socktype)

        self.message_separator_character = message_separator_character

    def emit(self, record):
        """
        SFTCP addition:
        To let the user to choose which message_separator_character to use, we override the emit function.
        ####
        Emit a record.

        The record is formatted, and then sent to the syslog server. If
        exception information is present, it is NOT sent to the server.
        """
        try:
            msg = self.format(record) + self.message_separator_character
            if self.ident:
                msg = self.ident + msg

            # We need to convert record level to lowercase, maybe this will
            # change in the future.
            prio = '<%d>' % self.encodePriority(self.facility,
                                                self.mapPriority(record.levelname))
            prio = prio.encode('utf-8')
            # Message is a string. Convert to bytes as required by RFC 5424
            msg = msg.encode('utf-8')
            msg = prio + msg
            if self.unixsocket:
                try:
                    self.socket.send(msg)
                except OSError:
                    self.socket.close()
                    self._connect_unixsocket(self.address)
                    self.socket.send(msg)
            elif self.socktype == socket.SOCK_DGRAM:
                self.socket.sendto(msg, self.address)
            else:
                self.socket.sendall(msg)
        except Exception:
            self.handleError(record)

Answer 3

鉴于问题被标记为fluentd，您是否尝试过使用fluent.handler.FluentHandler 代替logging.handlers.SysLogHandler - 请参阅https://github.com/fluent/fluent-logger-python？