java.nio.ScoketChannel 忽略 Content-Length 并使用 Transfer-Encoding: chunked based on User-Agent

Question

我想在 javax.servlet.Filter 中压缩响应体。这是我的代码

byte[] bytes =  // compressing response body
response.addHeader("Content-Encoding", "gzip");
response.addHeader("Content-Length", String.valueOf(bytes.length));
response.setContentLength(bytes.length);
response.setBufferSize(bytes.length * 2);
ServletOutputStream output = response.getOutputStream();
output.write(bytes);
output.flush();
output.close();

但我在 Chrome 开发工具中看到的实际响应是

Accept-Ranges: bytes
Cache-Control: max-age=2592000
Content-Type: application/javascript;charset=UTF-8
Date: Fri, 14 Dec 2018 15:34:25 GMT
Last-Modified: Tue, 09 Oct 2018 13:42:54 GMT
Server: Apache-Coyote/1.1
Transfer-Encoding: chunked

我不希望 Transfer-Encoding: 分块，因为我声明了“Content-Length”。我在java上写了一个简单的测试

URLConnection connection = new URL("http://127.0.0.1:8081/js/ads.js").openConnection();
connection.addRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8");
connection.addRequestProperty("Accept-Encoding", "gzip, deflate");
connection.addRequestProperty("Accept-Language", "ru-RU,ru;q=0.9,en-US;q=0.8,en;q=0.7");
connection.addRequestProperty("Cache-Control", "no-cache");
connection.addRequestProperty("Connection", "keep-alive");
connection.addRequestProperty("Host", "127.0.0.1:8081");
connection.addRequestProperty("Pragma", "no-cache");
connection.addRequestProperty("Upgrade-Insecure-Requests", "1");
connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36"); 
connection.connect();
connection.getHeaderFields().forEach((s, strings) ->
        System.out.println(s + ":" + String.join(",", strings)));

这是我发现的：

• 如果我评论设置“User-Agent”标头或将“User-Agent”更改为任何其他值，那么我会收到“Content-Length”的响应
• 如果 Chrome 上的“用户代理”点然后我得到 Transfer-Encoding: chunked。

我调试了 sun.nio.ch.SocketChannel#write 方法，它得到了正确的 ByteBuffers 和 Content-Length 标头值。

我无法理解这种对分块的神奇转变发生在哪里？

更新

奇怪的是，我将压缩字节写入 Socket（我确信当我调试到调用 SocketChannel 实现中的本机方法写入时）。但是如果我没有指定 User-Agent 标头或放置一些随机字符串，URLConnection 将返回我的 unzipped 字节数组和 Chrome 的 User-Agent 和正确的 gzip 字节数组。 SO 似乎在 Windows 套接字实现的某个地方发生了魔法。

Answer 1

显示代码

我会假设您显示的代码有效并且问题出在其他地方。

设置

• Windows 10
• Tomcat 7.0.92
• 铬 71.0.3578.98

测试用例

我尝试创建一个小型过滤器示例来试用您的测试代码。

顺便说一句，更适合生产用途的压缩过滤器可以在 Tomcat 提供的示例 (webapps\examples\WEB-INF\classes\compressionFilters)。

import java.io.*;
import java.util.zip.GZIPOutputStream;
import javax.servlet.*;
import javax.servlet.http.*;

public class CompressionFilter  implements Filter {

    public void init(FilterConfig filterConfig) { }
    public void destroy() { }

    public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain filterChain) throws IOException, ServletException {
        final HttpServletRequest request = (HttpServletRequest) servletRequest;
        final HttpServletResponse response = (HttpServletResponse) servletResponse;

        ResponseWrapper wrapper = new ResponseWrapper(response);
        filterChain.doFilter(request, wrapper);
        byte[] uncompressed = wrapper.getBytes();

        byte[] bytes = compress(uncompressed);
        response.addHeader("Content-Encoding", "gzip");
        response.addHeader("Content-Length", String.valueOf(bytes.length));
        response.setContentLength(bytes.length);
        //response.setBufferSize(bytes.length * 2);
        ServletOutputStream output = response.getOutputStream();
        output.write(bytes);
        output.flush();
        output.close();

        System.out.println("request to:" +  request.getServletPath()
                + " size changed from: " + uncompressed.length
                + " to " + bytes.length);
    }

    private byte[] compress(byte[] bytes) throws IOException {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        GZIPOutputStream gzipOutputStream = new GZIPOutputStream(baos);
        gzipOutputStream.write(bytes);
        gzipOutputStream.close();
        return baos.toByteArray();
    }


    public class ResponseWrapper extends HttpServletResponseWrapper {
        private ByteArrayOutputStream output = new ByteArrayOutputStream();
        private PrintWriter printWriter = null;

        ResponseWrapper(HttpServletResponse response) {
            super(response);
        }

        byte[] getBytes() {
            if (printWriter != null)
                printWriter.flush();
            return output.toByteArray();
        }

        public PrintWriter getWriter() {
            if (printWriter == null)
                printWriter = new PrintWriter(output);
            return printWriter;
        }

        public ServletOutputStream getOutputStream() {
            return new ServletOutputStream() {
                private WriteListener writeListener;
                public boolean isReady() { return true; }
                public void setWriteListener(WriteListener writeListener) { this.writeListener  = writeListener; }
                public void write(int b) {
                    output.write(b);
                    if(writeListener != null)
                        writeListener.notify();
                }
            };
        }
    }

}

结果

三个带有静态 html 的测试用例、一个 JSP 生成的页面和一个带有一些虚拟内容的 Servlet 生成的页面在 Chrome 的开发者工具中显示如下：

a) 使用静态 html

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Accept-Ranges: bytes
ETag: W/"108-1545775482914"
Last-Modified: Tue, 25 Dec 2018 22:04:42 GMT
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 97
Date: Tue, 25 Dec 2018 22:34:41 GMT

b) JSP 生成

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 38
Date: Tue, 25 Dec 2018 22:49:17 GMT

c) Servlet 生成

HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Encoding: gzip
Content-Type: text/html
Content-Length: 65
Date: Tue, 25 Dec 2018 22:49:43 GMT

使用此设置没有Transfer-Encoding: chunked。那么也许这个分块头的原因可以在其他地方找到？