unpack_from 需要至少 4 个字节的缓冲区

Question

我正在接收来自客户端的数据包，其中包含许多字段。我成功读取了所有字段，但是当涉及到最后一个字段 tag_end 时，python 给了我一个错误：

unpack_from 需要至少 4 个字节的缓冲区，但未找到。

这是代码：

def set_bin(self, buf):
    """Reads a vector of bytes (probably received from network or
    read from file) and tries to construct the packet structure
    from it, by reading each packet member from the buffer.  This
    is somehow like deserializing the packet.
    """
    assert isinstance(buf, bytearray), 'buffer type is not valid'
    offset = 0

    print("$$$$$$$$$$$$$$$$ set bin $$$$$$$$$$$$$$$$$")

    try:
        (self._tag_start, self._version, self._checksum, self._connection_id,
         self._packet_seq) = Packet.PACKER_1.unpack_from(str(buf), offset)
    except struct.error as e:
        print(e)
        raise DeserializeError(e)
    except ValueError as e:
        print(e)
        raise DeserializeError(e)
          #I=4 H=2 B=1
    offset = Packet.OFFSET_GUID     #14 correct
    self._guid = buf[offset:offset+Packet.UUID_SIZE] #14-16 correct

    offset = Packet.OFFSET_GUID + Packet.UUID_SIZE

    print("$$$$$$$$$$$$$$$$ GUID read successfully  $$$$$$$$$$$$$$$$$")

    try:
       (self._timestamp_sec, self._timestamp_microsec, self._command,
        self._command_seq, self._subcommand, self._data_seq,
        self._data_length) = Packet.PACKER_3.unpack_from(str(buf), offset)
    except struct.error as e:
        print(e)
        raise DeserializeError(e)
    except ValueError as e:
        print(e)
        raise DeserializeError(e)

    print("$$$$$$$$$$$$$$$$ timestamps read successfully $$$$$$$$$$$$$$$$$")
   offset = Packet.OFFSET_AUTHENTICATE
    self._username = buf[offset:offset + self.USERNAME_SIZE]        #Saman
    offset += self.USERNAME_SIZE

    print("$$$$$$$$$$$$$$$$ username read successfully  $$$$$$$$$$$$$$$$$")

    self._password = buf[offset:offset+self.USERNAME_SIZE]
    offset += self.PASSWORD_SIZE

    print("$$$$$$$$$$$$$$$$ password read successfully $$$$$$$$$$$$$$$$$")

    self._data = buf[offset:offset+self._data_length]
    offset = offset + self._data_length

    print("$$$$$$$$$$$$$$$$ data read successfully  $$$$$$$$$$$$$$$$$")

    try:
        (self._tag_end,) = Packet.PACKER_4.unpack_from(str(buf), offset)
    except struct.error as e:
        print(e)
        raise DeserializeError(e)
    except ValueError as e:
        print(e)
        raise DeserializeError(e)

    print("$$$$$$$$$$$$$$$$ tag end read successfully $$$$$$$$$$$$$$$$$")


    if len(buf) != Packet.PACKER.size + self._data_length:
        print('failed to deserialize binary data correctly and construct the packet due to extra data')
    else:
        print('@@@@@@@@@@@@@@@ Deserialized Successfully')

这是代码中使用的一些常量：

STRUCT_FORMAT_STR   = r'=IHIHH 16B IIHHHHH I 6c 9c' #Saman
STRUCT_FORMAT_STR_1 = r'=IHIHH'
STRUCT_FORMAT_STR_2 = r'=16B'
STRUCT_FORMAT_STR_3 = r'=IIHHHHH'
STRUCT_FORMAT_STR_4 = r'=I'
STRUCT_FORMAT_STR_5 = r'=6c'
STRUCT_FORMAT_STR_6 = r'=9c'
UUID_SIZE   = 16
OFFSET_GUID = 14
#OFFSET_DATA = 48     #shifting offset data by 15 char      
OFFSET_AUTHENTICATE = 48
PACKER   = struct.Struct(str(STRUCT_FORMAT_STR))     #Saman
PACKER_1 = struct.Struct(str(STRUCT_FORMAT_STR_1))
PACKER_2 = struct.Struct(str(STRUCT_FORMAT_STR_2))
PACKER_3 = struct.Struct(str(STRUCT_FORMAT_STR_3))
PACKER_4 = struct.Struct(str(STRUCT_FORMAT_STR_4))
PACKER_5 = struct.Struct(str(STRUCT_FORMAT_STR_5))
PACKER_6 = struct.Struct(str(STRUCT_FORMAT_STR_6))
BYTES_TAG_START = PACKER_4.pack(TAG_START)
BYTES_TAG_END   = PACKER_4.pack(TAG_END)

以及数据包对象的初始化，它在其中初始化字段：

def init(self, **kwargs): 如果 kwargs 中的“buf”： self.set_bin(kwargs['buf']) 别的： 在 Packet.RTCINET_COMMANDS.values() 中断言 kwargs['command'] 和在 Packet.RTCINET_COMMANDS.values() 中断言 kwargs['subcommand']，'未定义的协议命令' 断言 isinstance(kwargs['data'], bytearray), '数据字段的类型无效' 对于（“命令”，“子命令”，“数据”）中的字段： setattr(self, '_' + field, kwargs[field])

    self._tag_start = Packet.TAG_START
    self._version = Packet.VERSION_CURRENT % (Packet.USHRT_MAX + 1)
    self._checksum = Packet.CRC_INIT
    self._connection_id = kwargs.get('connection_id', 0) % (Packet.USHRT_MAX + 1)
    self._packet_seq = Packet.PACKET_SEQ
    Packet.PACKET_SEQ = (Packet.PACKET_SEQ + 1) % (Packet.USHRT_MAX + 1)
    self._guid = uuid.uuid4().bytes
    dt = datetime.datetime.now()
    self._timestamp_sec = int(time.mktime(dt.timetuple()))
    self._timestamp_microsec = dt.microsecond
    # self._command = kwargs['command']
    self._command_seq = kwargs.get('command_seq', 0)
    # self._subcommand = kwargs['subcommand']
    self._data_seq = kwargs.get('data_seq', 0)
    self._data_length = len(kwargs['data'])

    self._username = Packet.USERNAME            #Saman
    self._password = Packet.PASSWORD

我已确保以正确的顺序读取所有字段，因为它是由客户端程序写入数据包中的。但我仍然无法解决这个问题。

你知道如何解决这个问题吗？

Answer 1

问题似乎是你无缘无故地将东西转换为str。

在某些地方，例如PACKER_1 = struct.Struct(str(STRUCT_FORMAT_STR_1))，它会降低您的代码的可读性和可理解性，但不会影响实际输出。例如，STRUCT_FORMAT_STR_1 已经是str，所以str(STRUCT_FORMAT_STR_1) 是相同的str。

但在其他地方，情况远比这糟糕。特别是，查看像Packet.PACKER_1.unpack_from(str(buf), offset) 这样的所有行。在那里，buf 是一个bytearray。 （它必须是，因为你 assert 它。）在 bytearray 上调用 str 会为你提供 bytearray 的字符串表示形式。例如：

>>> b = bytearray(b'abc')
>>> len(b)
3
>>> s = str(b)
>>> s
"bytearray(b'abc')"
>>> len(s)
17

该字符串表示显然通常不会与您所表示的实际缓冲区具有相同的长度。因此，难怪您会收到有关长度错误的错误。 （如果你真的很不走运并且没有任何此类错误，那么你将改为读取垃圾值。）

那么，您应该如何将bytearray 转换为struct 模块可以处理的东西？没有！正如the docs 所说：

几个 struct 函数（和 Struct 的方法）采用缓冲区参数。这是指实现缓冲区协议并提供可读或可读写缓冲区的对象。用于此目的的最常见类型是 bytes 和 bytearray...