对内容块使用压缩时，发布内容流获得缓冲

Question

首先介绍什么有效：（我的问题的描述进一步向下）

客户端应用程序向服务器 WebAPI 发送一个长数据流。 请求被分块 (request.Headers.TransferEncodingChunked = True)，数据被“即时”写入流。

数据由序列化对象组成，每个对象代表自定义类Chunk。

一切正常，服务器可以在客户端完成写入之前开始读取流（控制器禁用服务器输入缓冲）。

开始请求：

Public Async Function Transfer(authCookie As Cookie) As Task

    Using handler As New HttpClientHandler()
        handler.CookieContainer = New Net.CookieContainer
        handler.CookieContainer.Add(authCookie)

        'Client instance
        Using client As New HttpClient(handler), content As New Content(AddressOf WriteToStream)

            'Send stream asynchronously
            Dim request As New HttpRequestMessage(HttpMethod.Post, "https://myserver.net/api/datastream")
            request.Content = content
            request.Headers.TransferEncodingChunked = True

            Await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).ContinueWith(
                Async Function(requestTask)
                    Dim response As HttpResponseMessage = Nothing
                    Try
                        response = Await requestTask
                    Catch ex As Exception
                        'Exception writing the stream
                        Exit Function
                    End Try
                    Try
                        response.EnsureSuccessStatusCode()
                    Catch ex As Exception
                        'Exception on server ocurred
                    End Try

                    response.RequestMessage.Dispose()
                    response.Dispose()
                End Function)

        End Using
    End Using

End Function

Public Sub WriteToStream(str As IO.Stream)

    'Head-chunk
    head.attachedChunks.CompleteAdding()

    'Write head-chunk to stream
    Task.Run(
        Async Function()
            Await head.WriteToStream(str)
        End Function).Wait()

End Sub

HttpContent 类：

Public Class Content
    Inherits Net.Http.HttpContent

    Protected ReadOnly actionOfStream As Action(Of IO.Stream)

    Public Sub New(action As Action(Of IO.Stream))
        If action Is Nothing Then Throw New ArgumentNullException("action")
        actionOfStream = action
    End Sub

    Protected Overrides Function SerializeToStreamAsync(stream As IO.Stream, context As Net.TransportContext) As Task

        Return Task.Factory.StartNew(
            Sub(obj)
                Dim target As IO.Stream = DirectCast(obj, IO.Stream)
                actionOfStream(target)
            End Sub,
            stream)

    End Function

    Protected Overrides Function TryComputeLength(ByRef length As Long) As Boolean
        length = -1
        Return False
    End Function

End Class

WriteToStream() 方法调用 Head-chunks WriteToStream()，它为其子块调用相同的方法：

Public Class Chunk

    'More properties here

    Public Property attachedChunks As BlockingCollection(Of Chunk)

    Public Overridable Async Function WriteToStream(str As IO.Stream) As Task

        Using serializationStream As New IO.MemoryStream

            'Serialize
            Serializer.Serialize(Of Chunk)(serializationStream, Me)

            'Write length
            Dim cnt As Integer = CInt(serializationStream.Length)
            Dim cntBuffer() As Byte = BitConverter.GetBytes(cnt)
            Await str.WriteAsync(cntBuffer, 0, cntBuffer.Length)

            'Write chunk
            serializationStream.Seek(0, IO.SeekOrigin.Begin)
            Await serializationStream.CopyToAsync(str)

        End Using

        Await str.FlushAsync()

        'Clearing and disposing stuff 
        '...

        'Write sub-chunks
        If attachedChunks IsNot Nothing Then
            For Each chunk As Chunk In attachedChunks.GetConsumingEnumerable
                Await chunk.WriteToStream(str)
            Next

            attachedChunks.Dispose()
        End If

        'Write ending mark
        If attachedChunksIndefiniteCount Then
            Await str.WriteAsync({0, 0, 0, 0}, 0, 4)
        End If

    End Function

End Class

到目前为止一切顺利。

问题出现在我开始使用 GZipStream 之类的压缩工具来压缩我的 Chunk 时：请求在客户端被缓冲，因此会出现接缝。

这是Chunk-class 的WriteToStream() 方法：

Public Class Chunk

    'More properties here

    Public Property attachedChunks As BlockingCollection(Of Chunk)

    Public Overridable Async Function WriteToStream(str As IO.Stream) As Task

        Using serializationStream As New IO.MemoryStream

            'Serialization and compression
            Using gZipStream As New IO.Compression.GZipStream(serializationStream, IO.Compression.CompressionMode.Compress, True)
                Using bufferStream As New IO.BufferedStream(gZipStream, 64 * 1024)

                    'Serialize
                    Serializer.Serialize(Of Chunk)(bufferStream, Me)

                End Using
            End Using

            'Write length
            Dim cnt As Integer = CInt(serializationStream.Length)
            Dim cntBuffer() As Byte = BitConverter.GetBytes(cnt)
            Await str.WriteAsync(cntBuffer, 0, cntBuffer.Length)

            'Write chunk
            serializationStream.Seek(0, IO.SeekOrigin.Begin)
            Await serializationStream.CopyToAsync(str)

        End Using

        Await str.FlushAsync()

        'Clearing and disposing stuff 
        '...

        'Write sub-chunks
        If attachedChunks IsNot Nothing Then
            For Each chunk As Chunk In attachedChunks.GetConsumingEnumerable
                Await chunk.WriteToStream(str)
            Next

            attachedChunks.Dispose()
        End If

        'Write ending mark
        If attachedChunksIndefiniteCount Then
            Await str.WriteAsync({0, 0, 0, 0}, 0, 4)
        End If

    End Function

End Class

在写入所有数据之前它不会发送请求。请求是否缓冲流？ GZipStream 对某种底层上下文有影响吗？ 我无法弄清楚问题是什么......

Answer 1

我发现了困扰我的地方： 用大量数据进行测试，请求在几个块之后开始发送数据。

请求具有一定的缓冲区（您不能将其设置为使用HttpClient）。未压缩的内容很快就达到了这个限制，压缩的内容显然更晚了。

看起来整个请求都被缓冲了，因为我的测试数据太短了。