GAE - urlfetch 多部分帖子不适用于大文件答案

【问题标题】：GAE - urlfetch multipart post not working with large filesGAE - urlfetch 多部分帖子不适用于大文件
【发布时间】：2014-01-13 21:35:48
【问题描述】：

我正在尝试将位于 Blobstore 中的文件从 GAE 发布到另一个 HTTP 服务（病毒总数）。

当前的 sn-p 适用于小文件 (1 - 2 mb)：

import base64
import json
import logging
from poster.encode import multipart_encode, MultipartParam
from google.appengine.api import urlfetch


def post_file(url, file_name, file_type, file_size, file_obj, options=dict(), username=None, password=None):

    # Input checks
    if url is None:
        raise ValueError('url')

    if file_name is None:
        raise ValueError('file_name')

    if file_type is None:
        raise ValueError('file_type')

    if file_size is None:
        raise ValueError('file_size')

    if file_obj is None:
        raise ValueError('file_obj')

    if options is None:
        raise ValueError('options')

    logging.debug('Preparing file {0}'.format(file_name))

    # This is the post arguments section
    options['file'] = MultipartParam('file', filename=file_name, filetype=file_type, filesize=file_size, fileobj=file_obj)

    data, headers = multipart_encode(options)

    logging.debug('Submitting the file to {0}'.format(url))

    # For authorization (optional)
    if username is not None and password is not None:
        headers['Authorization'] = generate_authorization_header(username, password)

    fetch = urlfetch.fetch(url=url, payload="".join(data), method=urlfetch.POST, headers=headers)
    response = fetch.content

    result = json.loads(response)

    return result

当我将此 sn-p 用于大文件 (18 mb) 时，我得到以下异常：

Traceback (most recent call last):
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\api_server.py", line 162, in _handle_POST
    api_response = _execute_request(request).Encode()
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\api_server.py", line 120, in _execute_request
    make_request()
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\api_server.py", line 115, in make_request
    request_id)
  File "C:\Program Files (x86)\Google\google_appengine\google\appengine\api\apiproxy_stub.py", line 109, in MakeSyncCall
    'The request to API call %s.%s() was too large.' % (service, call))
RequestTooLargeError: The request to API call urlfetch.Fetch() was too large.

谢谢！

【问题讨论】：

标签： python google-app-engine multipart urlfetch

【解决方案1】：

App Engine 的 urlfetch 服务limits the request size to 10 megabytes。如果您有付费应用，则可以使用套接字（我建议使用 httplib 来简化您的代码，但请确保使用configure httplib to use sockets instead of urlfetch）。

【讨论】：