Netty 解码器在写入字符串时抛出 IndexOutOfBounsException答案

【问题标题】：Netty decoder throws IndexOutOfBounsException while writing stringNetty 解码器在写入字符串时抛出 IndexOutOfBounsException
【发布时间】：2020-11-02 18:04:25
【问题描述】：

堆栈颠覆者。我目前正在使用具有以下架构的 netty 编写一些网络内容：

集群
- 客户端 1
- 客户端 2
- 客户端 3

当我启动客户端时，它连接到一个集群，并且集群将我的对象（称为 ServerDataPacket）发送到所有连接的客户端。所有客户端都成功接收到 ServerDataPacket 对象。但是，当我在其中一个客户端中执行某些操作时，问题就开始了。例如，我在客户端 1 中做了一些事情，它将另一个对象发送到集群，集群接收对象 ServerDataPacket 并将其再次发送给所有客户端。所有客户端都会收到此对象，但客户端 1 不会收到此对象，而不是完整的信息会引发此错误：

io.netty.handler.codec.DecoderException: java.lang.IndexOutOfBoundsException: index: 501, length: 15 (expected: range(0, 512))
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IndexOutOfBoundsException: index: 501, length: 15 (expected: range(0, 512))
        at io.netty.buffer.AbstractByteBuf.checkIndex0(AbstractByteBuf.java:1359)
        at io.netty.buffer.AbstractByteBuf.checkIndex(AbstractByteBuf.java:1354)
        at io.netty.buffer.PooledUnsafeDirectByteBuf.internalNioBuffer(PooledUnsafeDirectByteBuf.java:331)
        at io.netty.buffer.ByteBufUtil.decodeString(ByteBufUtil.java:614)
        at io.netty.buffer.AbstractByteBuf.toString(AbstractByteBuf.java:1213)
        at io.netty.buffer.AbstractByteBuf.getCharSequence(AbstractByteBuf.java:492)
        at io.netty.buffer.AbstractByteBuf.readCharSequence(AbstractByteBuf.java:497)
        at net.novaplay.common.network.bungee.ServerDataPacket.read(ServerDataPacket.java:45)
        at net.novaplay.common.netty.packet.PacketDecoder.decode(PacketDecoder.java:29)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        ... 16 more

解码字节缓冲区中的字符串导致错误

    @Override
    public void read(ByteBuf buf){
            int length = buf.readInt();
            String id = (String)buf.readCharSequence(length, Charsets.UTF_8); //here i got an error
    }

我的网络解码器：

public class PacketDecoder extends ByteToMessageDecoder {
    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf byteBuf, List<Object> output) throws Exception {
        while(byteBuf.readableBytes() > 0) {
            int id = byteBuf.readInt();
            if(NettyHandler.DEBUG) {
                System.out.println("Packet ID: " + id);
            }
            Class<? extends Packet> packetClass = PacketHandler.PACKETS.get(id);
            if(packetClass == null) {
                throw new NullPointerException("Couldn't find packet by id " + id);
            }
            Packet packet = packetClass.newInstance();
            int length = byteBuf.readInt();
            packet.uniqueId = UUID.fromString( (String) byteBuf.readCharSequence(length, Charsets.UTF_8)); //memory leak here, view UPD.2
            packet.read(byteBuf); //memory leak here, view UPD.2
            output.add(packet);
            if(NettyHandler.DEBUG) {
                System.out.println("Packet successfully has been decoded");
            }
        }
    }
}

和编码器：

public class PacketEncoder extends MessageToByteEncoder<Packet> {
    @Override
    protected void encode(ChannelHandlerContext ctx, Packet packet, ByteBuf byteBuf) throws Exception {
        int id = PacketHandler.PACKETS.indexOf(packet.getClass());
        if(NettyHandler.DEBUG) {
            System.out.println("Packet ID: " + id);
        }
        if(id == -1) {
            throw new NullPointerException("Couldn't find id of packet " + packet.getClass().getSimpleName());
        }
        UUID uuid = packet.getUniqueId();
        byteBuf.writeInt(id);
        byteBuf.writeInt(uuid.toString().length());
        byteBuf.writeCharSequence(uuid.toString(), Charsets.UTF_8);
        packet.write(byteBuf);
        if(NettyHandler.DEBUG) {
            System.out.println("Packet successfully has been encoded");
        }
    }
}

客户端 1 收到此错误后，当我从集群发送内容时，它会接收除客户端 1 之外的所有客户端。

你知道为什么客户端会表现得如此奇怪吗？

UPD.：我忘了说，我的对象是由数字 ID 标识的，这在解码和编码方面是相同的。但是当这个错误出现时，这个数据包会得到一些奇怪的 ID，比如 1113941079

UPD.2.：现在我启用了更深层次的调试器并将这个对象发送给 2 个客户端。第二个客户端正常接收，但第一个客户端出现内存泄漏：

io.netty.handler.codec.DecoderException: java.lang.IndexOutOfBoundsException: readerIndex(512) + length(4) exceeds writerIndex(512): PooledUnsafeDirectByteBuf(ridx: 512, widx: 512, cap: 512)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IndexOutOfBoundsException: readerIndex(512) + length(4) exceeds writerIndex(512): PooledUnsafeDirectByteBuf(ridx: 512, widx: 512, cap: 512)
        at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1395)
        at io.netty.buffer.AbstractByteBuf.readInt(AbstractByteBuf.java:766)
        at net.novaplay.common.network.bungee.ServerDataPacket.read(ServerDataPacket.java:45)
        at net.novaplay.common.netty.packet.PacketDecoder.decode(PacketDecoder.java:29)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        ... 16 more
Packet ID: 9
Got io.netty.handler.codec.DecoderException: java.lang.OutOfMemoryError: Java heap space at
io.netty.handler.codec.DecoderException: java.lang.OutOfMemoryError: Java heap space
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:442)
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:351)
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:373)
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:359)
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926)
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129)
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:651)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:574)
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:488)
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:450)
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:873)
        at io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:144)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
        at java.nio.HeapCharBuffer.<init>(HeapCharBuffer.java:57)
        at java.nio.CharBuffer.allocate(CharBuffer.java:335)
        at io.netty.buffer.ByteBufUtil.decodeString(ByteBufUtil.java:605)
        at io.netty.buffer.AbstractByteBuf.toString(AbstractByteBuf.java:1213)
        at io.netty.buffer.AbstractByteBuf.getCharSequence(AbstractByteBuf.java:492)
        at io.netty.buffer.AbstractByteBuf.readCharSequence(AbstractByteBuf.java:497)
        at net.novaplay.common.netty.packet.PacketDecoder.decode(PacketDecoder.java:28)
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:411)
        ... 16 more

【问题讨论】：

标签： java netty

【解决方案1】：

这里的问题是由数据如何通过网络发送引起的。 ByteToMessageDecoder 收到的数据不能保证完全由完整的数据包组成——您可能只收到一个数据包的一部分，或者多个完整的数据包，后面跟着一个不完整的数据包。有两种简单的方法可以解决这个问题：

在您的解码器中添加一个检查（作为while 循环主体中的第一件事）以确保缓冲区包含完整的数据包，如果没有足够的可读性，则退出而不修改缓冲区的readerIndex字节。
在数据包解码器之前将LengthFieldBasedFrameDecoder 添加到管道中。这将自动读取并丢弃长度前缀，并为每个以长度为前缀的部分传递解码器一个缓冲区。请注意，这将要求您在数据包的开头添加另一个长度前缀，其中包含整个数据包的长度（包括 ID、UUID 以及实际的数据包正文）。

奖励：您编码字符串的方法不正确。 ByteBuf#writeCharSequence 返回写入的字节数，ByteBuf#readCharSequence 要求字符串的大小以 bytes 为单位，而不是字符。它适用于您的用例，因为 UUID 的文本表示仅由单字节字符组成，但通常您应该使用以下内容：

int lengthPrefixIndex = buffer.writerIndex();
int length = buffer.writeInt(-1).writeCharSequence(text, StandardCharsets.UTF_8);
buffer.setInt(lengthPrefixIndex, length);

此外，ByteBuf#readCharSequence 不能保证返回 String，您应该删除演员表并将其替换为对 toString() 的调用。

【讨论】：