Httpie 无法解码我的 Bottle API Gzipped 响应

Question

感谢 Bottle Web 框架，我有一个用 Python 实现的 HTTP/JSON Restful 服务器。我想压缩发送给客户端的数据。

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# curl -H "Content-Type: application/json" -X POST -d '{"key1": 1, "key2": 2}' http://localhost:6789/post
#

from bottle import run, request, post, route, response
from zlib import compress
import json

data = {'my': 'json'}


@post('/post')
def api_post():
    global data
    data = json.loads(request.body.read())
    return(data)


@route('/get')
def api_get():
    global data
    response.headers['Content-Encoding'] = 'identity'
    return(json.dumps(data).encode('utf-8'))


@route('/getgzip')
def api_get_gzip():
    global data
    if 'gzip' in request.headers.get('Accept-Encoding', ''):
        response.headers['Content-Encoding'] = 'gzip'
        ret = compress(json.dumps(data).encode('utf-8'))
    else:
        response.headers['Content-Encoding'] = 'identity'
        ret = json.dumps(data).encode('utf-8')
    return(ret)


run(host='localhost', port=6789, debug=True)

当我用 Curl 测试我的服务器时，结果很好（如果我使用 --compressed 选项标签）：

$ curl  -H "Accept-encoding: gzip, deflated" -v --compressed http://localhost:6789/getgzip
*   Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 6789 (#0)
> GET /getgzip HTTP/1.1
> Host: localhost:6789
> User-Agent: curl/7.47.0
> Accept: */*
> Accept-encoding: gzip, deflated
> 
* HTTP 1.0, assume close after body
< HTTP/1.0 200 OK
< Date: Sun, 12 Nov 2017 09:09:09 GMT
< Server: WSGIServer/0.1 Python/2.7.12
< Content-Length: 22
< Content-Encoding: gzip
< Content-Type: text/html; charset=UTF-8
< 
* Closing connection 0
{"my": "json"}

但不适用于 HTTPie（或 Firefox，或 Chrome...）：

$ http http://localhost:6789/getgzipHTTP/1.0 200 OK
Content-Encoding: gzip
Content-Length: 22
Content-Type: text/html; charset=UTF-8
Date: Sun, 12 Nov 2017 09:10:10 GMT
Server: WSGIServer/0.1 Python/2.7.12

http: error: ContentDecodingError: ('Received response with content-encoding: gzip, but failed to decode it.', error('Error -3 while decompressing: incorrect header check',))

有什么想法吗？

Answer 1

我尼科拉戈，

根据 Httpie 的文档，默认编码设置为Accept-Encoding: gzip, deflate，但您使用的是zlib 模块的compress Python 函数，该函数实现了Lempel–Ziv–Welch Compression Algorithm（Gzip 基于DEFLATE Algorithm）。

或者，根据 Bottle (https://bottlepy.org/docs/dev/recipes.html#gzip-compression-in-bottle) 的文档，您将需要一个自定义中间件来执行 gzip 压缩（参见此处的示例：http://svn.cherrypy.org/tags/cherrypy-2.1.1/cherrypy/lib/filter/gzipfilter.py）。

编辑：

zlib 模块的compress 函数执行 gzip 兼容压缩。

我认为它与数据的header 更相关（如错误提及）。在http://svn.cherrypy.org/tags/cherrypy-2.1.1/cherrypy/lib/filter/gzipfilter.py 中有一个write_gzip_header 的用法，也许你可以试试这个。

Answer 2

感谢 Guillaume 的编辑部分，它现在可以与 Httpie 和 Curl 一起使用。

完整代码如下：

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
# curl -H "Content-Type: application/json" -X POST -d '{"key1": 1, "key2": 2}' http://localhost:6789/post
#

from bottle import run, request, post, route, response
import zlib
import json
import struct
import time

data = {'my': 'json'}


@post('/post')
def api_post():
    global data
    data = json.loads(request.body.read())
    return(data)


@route('/get')
def api_get():
    global data
    response.headers['Content-Encoding'] = 'identity'
    return(json.dumps(data).encode('utf-8'))


@route('/getgzip')
def api_get_gzip():
    global data
    ret = json.dumps(data).encode('utf-8')
    if 'gzip' in request.headers.get('Accept-Encoding', ''):
        response.headers['Content-Encoding'] = 'gzip'
        ret = gzip_body(ret)
    else:
        response.headers['Content-Encoding'] = 'identity'
    return(ret)


def write_gzip_header():
    header = '\037\213'      # magic header
    header += '\010'         # compression method
    header += '\0'
    header += struct.pack("<L", long(time.time()))
    header += '\002'
    header += '\377'
    return header


def write_gzip_trailer(crc, size):
    footer = struct.pack("<l", crc)
    footer += struct.pack("<L", size & 0xFFFFFFFFL)
    return footer


def gzip_body(body, compress_level=6):
    yield gzip_header()
    crc = zlib.crc32("")
    size = 0
    zobj = zlib.compressobj(compress_level,
                            zlib.DEFLATED, -zlib.MAX_WBITS,
                            zlib.DEF_MEM_LEVEL, 0)
    size += len(data)
    crc = zlib.crc32(data, crc)
    yield zobj.compress(data)
    yield zobj.flush()
    yield gzip_trailer(crc, size)


run(host='localhost', port=6789, debug=True)

这有点复杂，但它可以完成工作......